Crack detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a crack detection method and device, electronic equipment and a storage medium, and relates to the technical field of image recognition, and the method comprises the steps: obtaining a plurality of crack images, and marking a bounding box for a crack in each crack image; a YOLOv5 model is obtained, a Wise-IoU algorithm, a CA attention mechanism module, a GhostNet module and an SPPFCSPC module are fitted to the YOLOv5 model, and a crack detection model is obtained; training a crack detection model by using each crack image marked with the bounding box as training data; and performing crack detection on the crack image by using the trained crack detection model. On the basis of a YOLOv5 model, a Wise-IoU algorithm, a CA attention mechanism module, a GhostNet module and an SPPFCSPC module are introduced, the YOLOv5 model is modified to obtain a crack detection model, and the crack detection accuracy and efficiency of the crack detection model can be improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a crack detection method, device, electronic device and storage medium. Background Art

[0002] Crack detection is a very important engineering technology, which is widely used in the maintenance and safety monitoring of various infrastructures such as building structures, bridges, roads, pipelines, etc. The presence of cracks may cause structural damage and functional problems, and may even cause serious safety hazards. Therefore, early detection and timely repair of cracks are crucial to ensure public safety and extend the service life of facilities. However, the detection accuracy and efficiency of existing technologies are still low. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to provide a crack detection method, device, electronic device and storage medium to accurately and efficiently perform crack detection.

[0004] To achieve the above object, an embodiment of the present application provides a crack detection method, which comprises the following steps:

[0005] Acquire multiple crack images, and mark a boundary box for the crack in each of the crack images;

[0006] Obtain a YOLOv5 model, fit the Wise-IoU algorithm, the CA attention mechanism module, the GhostNet module, and the SPPFCSPC module to the YOLOv5 model, and obtain the modified YOLOv5 model as a crack detection model;

[0007] Training the crack detection model using each of the crack images annotated with the bounding box as training data;

[0008] The trained crack detection model is used to perform crack detection on the crack image.

[0009] In some embodiments, acquiring a plurality of crack images and marking a boundary box for the crack in each of the crack images comprises the following steps:

[0010] Under different weather conditions, different regions and different time conditions, multiple images of concrete surface cracks are collected as the first image set;

[0011] Deleting the images whose resolution is lower than a set threshold and the images that are blocked or repeated from among the first images to obtain a second image set;

[0012] Marking the cracks in the second image set with bounding boxes, and determining the coordinates of the upper left corner point and the lower right corner point of each protected bounding box;

[0013] Convert the coordinates of the upper left corner point and the coordinates of the lower right corner point of each of the bounding boxes into corresponding center point coordinates, width length, and height length, and normalize each of the center point coordinates, width length, and height length to obtain a third image set;

[0014] Rotating, scaling, flipping, cropping and splicing each image in the third image set to obtain a fourth image set;

[0015] The pixels of each image in the fourth image set are normalized to obtain each crack image.

[0016] In some embodiments, fitting the Wise-IoU algorithm, the CA attention mechanism module, the GhostNet network, and the SPPFCSPC module to the YOLOv5 model to obtain the modified YOLOv5 model as a crack detection model includes the following steps:

[0017] Using the Wise-IoU algorithm to adjust the gradient gain of the YOLOv5 model according to the outlier degree of the bounding box;

[0018] Using the CA attention mechanism module to weight the feature points according to the spatial position information of the feature points of the crack image;

[0019] Splicing the GhostNet module with the convolution module of the YOLOv5 model as the backbone network of the YOLOv5 model;

[0020] The SPPFCSPC module is spliced ​​with the feature extraction module of the YOLOv5 model.

[0021] In some embodiments, the method of training the crack detection model using the crack images annotated with the bounding boxes as training data comprises the following steps:

[0022] Clustering each of the bounding boxes to obtain target bounding boxes of multiple categories;

[0023] Randomly scaling and cropping the crack image annotated with the target bounding box to obtain a plurality of training images;

[0024] Extracting texture features, geometric features, shape and category features of each of the training images;

[0025] Performing dimension alignment and feature fusion on the texture features, the geometric features, and the shape and category features by using convolution and bilinear interpolation methods to obtain a fused feature map;

[0026] The fused feature map is used as a training sample, the target bounding box is used as a training label, the crack detection model is trained, and the Wise-IoU algorithm is used as a loss function to adjust the weight parameters of the crack detection model.

[0027] In some embodiments, clustering the bounding boxes to obtain target bounding boxes of multiple categories includes the following steps:

[0028] Normalizing the width and height of each of the bounding boxes, and converting the normalized width and height into two-dimensional data points;

[0029] Using the K-means algorithm to cluster the data points into K clusters, so that the sum of the distances from the data points to the center point of the cluster to which they belong is minimized; wherein K is a positive integer;

[0030] The K center points obtained by clustering are converted into widths and heights, and then the converted K widths and heights are denormalized and restored to the original scale to obtain the target bounding boxes of K categories.

[0031] In some embodiments, the adjusting the weight parameters of the crack detection model using the Wise-IoU algorithm as a loss function comprises the following steps:

[0032] Initializing the weight parameters of the crack detection model;

[0033] Inputting the fused feature map marked with the target bounding box into the crack detection model to obtain a prediction result;

[0034] Calculate the classification loss value and bounding box regression loss value of the result using the Wise-IoU algorithm;

[0035] Calculating the gradients of the classification loss value and the bounding box regression loss value relative to each of the weight parameters using a back propagation algorithm;

[0036] The weight parameters of the crack detection model are adjusted according to the gradient to reduce the classification loss value and the bounding box regression loss value.

[0037] In some embodiments, the method further comprises the step of calculating performance parameters of the trained crack detection model, wherein the step of calculating performance parameters of the trained crack detection model comprises:

[0038] Calculating the precision, recall, average accuracy, parameter quantity and computational complexity of the trained crack detection model as the performance parameters;

[0039] The calculation formula of the accuracy is:

[0040]

[0041] The calculation formula of the recall rate is:

[0042]

[0043] The calculation formula of the average accuracy is:

[0044]

[0045] The calculation formula of the parameter quantity is:

[0046]

[0047] The calculation formula of the calculation amount is:

[0048]

[0049] Wherein, P represents the precision rate, TP represents the number of samples that are judged as positive samples by the trained crack detection model and are actually positive samples, FP represents the number of samples that are judged as positive samples by the trained crack detection model and are actually negative samples; R represents the recall rate, FN represents the number of samples that are judged as negative samples by the trained crack detection model and are actually positive samples; AP represents the average precision; Parameters represents the parameter quantity, L represents the total number of network layers of the trained crack detection model, and P i represents the number of parameters of the i-th layer network; FLOPs represents the amount of calculation, N represents the total number of operations in the crack detection model after training, Ops i represents the number of times the i-th operation is executed, F i Indicates the amount of computation required for the i-th operation.

[0050] To achieve the above object, another aspect of the embodiment of the present application provides a crack detection device, the device comprising:

[0051] An image acquisition unit, used for acquiring a plurality of crack images and marking a boundary box for the crack in each of the crack images;

[0052] A model modification unit, used for obtaining a YOLOv5 model, fitting a Wise-IoU algorithm, a CA attention mechanism module, a GhostNet module, and a SPPFCSPC module to the YOLOv5 model, and obtaining the modified YOLOv5 model as a crack detection model;

[0053] A model training unit, configured to train the crack detection model using each of the crack images annotated with the bounding box as training data;

[0054] A crack detection unit is used to perform crack detection on the crack image using the trained crack detection model.

[0055] To achieve the above objective, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above crack detection method when executing the computer program.

[0056] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above crack detection method is implemented.

[0057] The embodiments of the present application include at least the following beneficial effects:

[0058] This application can obtain multiple crack images, and mark the bounding box of the cracks in each crack image; obtain the YOLOv5 model, fit the Wise-IoU algorithm, CA attention mechanism module, GhostNet module and SPPFCSPC module to the YOLOv5 model, and obtain the modified YOLOv5 model as a crack detection model; use each crack image marked with a bounding box as training data to train the crack detection model; use the trained crack detection model to perform crack detection on the crack image. This application is based on the YOLOv5 model, introduces the Wise-IoU algorithm, CA attention mechanism module, GhostNet module and SPPFCSPC module, optimizes and modifies the YOLOv5 model to obtain a crack detection model, which can improve the accuracy and efficiency of the crack detection model for crack detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0060] Figure 1 A schematic diagram of a crack detection method provided in an embodiment of the present application;

[0061] Figure 2 An example structural diagram of a CA attention mechanism module provided in an embodiment of the present application;

[0062] Figure 3An example structural diagram of a GhostNet module combined with a convolution module provided in an embodiment of the present application;

[0063] Figure 4 An example structural diagram of an SPPFCSPC module provided in an embodiment of the present application;

[0064] Figure 5 An example diagram of a clustering result provided in an embodiment of the present application;

[0065] Figure 6 An exemplary flow chart of a crack detection method provided in an embodiment of the present application;

[0066] Figure 7 An example structural diagram of a crack detection model provided in an embodiment of the present application;

[0067] Figure 8 An example diagram of crack detection results provided in an embodiment of the present application;

[0068] Fig. 9 A structural schematic diagram of a crack detection device provided in an embodiment of the present application;

[0069] Fig.10 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0071] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0072] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0073] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0074] Before describing the embodiments of the present application in detail, some terms and related technologies that may be involved in the embodiments of the present application are first described. Some terms and related technologies that may be involved in the embodiments of the present application are applicable to the following explanations:

[0075] Some nouns are explained as follows:

[0076] CNN (Convolutional Neural Network): A deep learning model that is specifically designed to process data with a grid structure, such as images. It extracts and learns features through components such as convolutional layers, pooling layers, and fully connected layers.

[0077] YOLOv5: An object detection algorithm model whose name is the abbreviation of "You Only Look Once" that achieves real-time target detection through a single neural network model.

[0078] LabelImg: An open source image annotation tool for creating and editing labels on images, often used to create datasets for training deep learning models.

[0079] OpenCV (Open Source Computer Vision Library): An open source library for computer vision and image processing tasks that provides many functions and tools for processing images, videos, and feature extraction.

[0080] Wise-IoU: An indicator used to evaluate the performance of object detectors. It is an improved version of IoU (Intersection over Union) and is used to measure the degree of overlap between the bounding box predicted by the detector and the true bounding box.

[0081] IoU (Intersection over Union): A metric used to evaluate the performance of object detectors that measures the degree of overlap between the bounding box predicted by the detector and the true bounding box.

[0082] CA (Channel Attention): A technique used to enhance the performance of convolutional neural networks by weighting feature maps of different channels to increase the model's attention to important features.

[0083] GhostNet: A lightweight convolutional neural network architecture designed to improve performance while keeping the model lightweight.

[0084] SPPFCSPC: A spatial pyramid pooling structure that avoids image distortion caused by cropping and scaling operations on image regions, and solves the problem of convolutional neural networks extracting repeated features related to graphs.

[0085] Kmeans (K-means clustering): A commonly used clustering algorithm used to classify data points into K different categories to minimize the variance within the category.

[0086] Backbone: In deep learning models, it refers to the backbone network structure, which is responsible for extracting features of input data.

[0087] Precision: In a binary classification problem, it refers to the ratio of the number of samples correctly predicted as positive by the model to the number of samples predicted as positive by the model.

[0088] Recall: In a binary classification problem, it refers to the ratio of the number of samples correctly predicted by the model as positive to the number of samples of all true positive categories.

[0089] AP (Average Precision): An indicator used to evaluate the performance of object detectors, measured by calculating the area under the precision-recall curve at different confidence thresholds.

[0090] Parameters: Learnable variables in a deep learning model whose values ​​are optimized through backpropagation during training.

[0091] FLOPs (Floating Point Operations): An indicator used to measure the computational workload of a deep learning model, indicating the number of floating-point operations performed by the model during inference or training.

[0092] The relevant technical description is as follows:

[0093] Crack detection is a very important engineering technology, which is widely used in the maintenance and safety monitoring of various infrastructures such as building structures, bridges, roads, pipelines, etc. The existence of cracks may cause structural damage and functional problems, and may even cause serious safety hazards. Therefore, early detection and timely repair of cracks are crucial to ensure public safety and extend the service life of facilities. There are currently three main crack detection methods:

[0094] Traditional methods: For a long time, people mainly relied on visual inspection and manual testing to find cracks. This method is subjective and uncertain, and also requires a lot of manpower and time. Therefore, with the increasing requirements for structural safety and quality, traditional methods have gradually revealed their limitations.

[0095] Nondestructive testing technology: In order to overcome the defects of traditional methods, nondestructive testing technology has become one of the main directions of crack detection. Ultrasonic testing is a widely used method that uses the characteristics of ultrasonic waves propagating in materials to detect cracks. X-ray testing uses the principle that X-rays penetrate materials and form images on the back to detect cracks. Magnetic particle testing displays cracks by coating magnetic powder on the surface and using a magnetic field to attract the powder. Eddy current testing detects surface defects and cracks by sensing changes in eddy currents. These methods are non-destructive, highly sensitive and high-resolution, and can be tested without damaging the integrity of the structure.

[0096] Image processing and computer vision: The application of image processing and computer vision technology in the field of crack detection has been an important trend in recent years. Through digital image processing algorithms and machine learning techniques, crack detection systems can automatically identify cracks from images or videos and perform quantitative analysis. These algorithms and techniques can accurately classify and locate cracks based on their shape, size, location and other features, helping engineers quickly understand the health of the structure. With the development of deep learning models, especially convolutional neural networks (CNNs), the performance of crack detection systems has been significantly improved. Deep learning models such as CNNs can learn feature representations in images, thereby achieving higher accuracy and robustness in crack detection. In addition, image processing and computer vision technologies can also be combined with other sensor data, such as acoustic wave data or laser scanning data, to provide a more comprehensive and accurate assessment of structural health. Therefore, the continuous advancement and application of image processing and computer vision technologies have brought new possibilities to the field of crack detection, making crack detection more automated, precise and efficient.

[0097] The prior art has the following problems:

[0098] 1. There is still room for optimization of the number of parameters in the existing models. A more lightweight model is needed to run in resource-limited environments such as mobile devices, embedded systems, and IoT devices.

[0099] 2. Existing methods are all tested on a certain number of data sets in a certain scenario. When new scenarios and data appear, they cannot be continuously optimized to adapt to changing situations.

[0100] Based on this, the purpose of this application is to provide a crack detection solution based on improved YOLOv5.

[0101] This application has the following beneficial effects:

[0102] 1. The model is lightweight, and crack detection can be performed quickly and reliably even on devices with limited computing resources, such as mobile devices or embedded systems, while ensuring that the user's privacy data will not be easily leaked. The use of lightweight models in this application not only provides an efficient solution for crack detection tasks, but also provides strong support for other real-time applications in resource-limited environments.

[0103] 2. This application can continuously train new data and new scenes based on certain detection images. When new data is added, the images with better detection results will be automatically generated with annotation files, and the new image data and annotation files will be trained to continuously optimize the detection capabilities of the model.

[0104] 3. It is not limited to a single scene and improves the efficiency of image annotation.

[0105] Therefore, the embodiments of the present application provide a crack detection method, device, electronic device and storage medium. The technical solution of the present application includes: obtaining multiple crack images, and marking the bounding box of the cracks in each crack image; obtaining the YOLOv5 model, fitting the Wise-IoU algorithm, CA attention mechanism module, GhostNet module and SPPFCSPC module to the YOLOv5 model, and obtaining the modified YOLOv5 model as the crack detection model; using each crack image marked with a bounding box as training data to train the crack detection model; using the trained crack detection model to perform crack detection on the crack image. The present application is based on the YOLOv5 model, introduces the Wise-IoU algorithm, CA attention mechanism module, GhostNet module and SPPFCSPC module, optimizes and modifies the YOLOv5 model to obtain the crack detection model, which can improve the accuracy and efficiency of the crack detection model for crack detection.

[0106] The embodiment of the present application provides a crack detection method, which relates to the field of image recognition technology. The crack detection method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the crack detection method, etc., but is not limited to the above forms.

[0107] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0108] Reference Figure 1 The present application embodiment provides a crack detection method, which may include but is not limited to S100 to S130, as follows:

[0109] S100: Acquire multiple crack images, and mark a boundary box for the crack in each crack image.

[0110] For example, in this embodiment, a boundary box may be marked on the crack in the crack image manually or using a marking tool.

[0111] Furthermore, S100 may include S101 to S106:

[0112] S101: collecting a plurality of images of concrete surface cracks under different weather conditions, different regions, and different time conditions as a first image set;

[0113] S102: deleting the images whose resolution is lower than a set threshold and the images that are blocked or repeated in the first images to obtain a second image set;

[0114] S103: marking boundary boxes for the cracks in the second image set, and determining the coordinates of the upper left corner point and the lower right corner point of each protected boundary box;

[0115] S104: converting the coordinates of the upper left corner point and the coordinates of the lower right corner point of each of the bounding boxes into corresponding center point coordinates, width length and height length, and normalizing each of the center point coordinates, width length and height length to obtain a third image set;

[0116] S105: rotating, scaling, flipping, cropping and splicing each image in the third image set to obtain a fourth image set;

[0117] S106: Normalizing the pixels of each image in the fourth image set to obtain each crack image.

[0118] Specifically, in this embodiment, the bounding box can be marked as follows:

[0119] Step 1: Data collection: Under different weather conditions, different regions, and different time conditions, image datasets of concrete surface cracks are collected to improve the generalization ability of the crack detection model (referred to as the “model”) to various environmental conditions.

[0120] Step 2: Data cleaning: Delete the unqualified crack images in the crack data set, such as those with blurred (resolution lower than the set threshold), blocked, repeated, etc., such as those with dead branches and leaves on the collected cracks, to ensure image quality.

[0121] Step 3: Data annotation: Use Labelimg image annotation tool to annotate the cracks in the image and generate an .xml suffix file containing the annotation box information of the cracks, including the coordinates of the upper left corner and the lower right corner.

[0122] Step 4: Annotation format conversion: Convert the coordinates of the upper left corner and lower right corner of the annotation box in the .xml suffix file into the format of center point coordinates, width length, and height length and normalize them, and finally save the annotation file with the suffix .txt.

[0123] Step 5: Dataset division: The labeled data is divided into training set and validation set in a ratio of 8:2. At the same time, the division of the data set is ensured to be random, and each sub-dataset contains various types of images to avoid bias.

[0124] Step 6: Data enhancement: Enhance the training set images by rotating, scaling, flipping, cropping, splicing, etc. The training data set is expanded by randomly combining different enhancement methods.

[0125] Step 7: Data loading and preprocessing: Use the image processing library OpenCV to load image data, normalize the image data, and scale the pixel values ​​to between 0 and 1.

[0126] S110: Obtain a YOLOv5 model, fit the Wise-IoU algorithm, the CA attention mechanism module, the GhostNet module, and the SPPFCSPC module to the YOLOv5 model, and obtain the modified YOLOv5 model as a crack detection model.

[0127] It can be understood that in order to improve the performance of the YOLOv5 model, this embodiment can fit the Wise-IoU algorithm, the CA attention mechanism module, the GhostNet module, and the SPPFCSPC module to the YOLOv5 model to achieve crack detection with higher accuracy and efficiency.

[0128] Furthermore, S110 may include S111 to S114:

[0129] S111: Using the Wise-IoU algorithm, adjust the gradient gain of the YOLOv5 model according to the outlier degree of the bounding box.

[0130] Specifically, the Wise-IoU algorithm combines a dynamic non-monotonic focusing mechanism, which can dynamically adjust the gradient gain of the YOLOv5 model according to the outlier degree of the bounding box. Whether the outlier degree is too large or too small, the gradient gain will be reduced. Only when the outlier degree is equal to the set hyperparameter, the highest gradient gain will be obtained, which reduces the competitiveness of high-quality bounding boxes while reducing the harmful gradients generated by low-quality examples. Then, the Wise-IoU algorithm can focus on bounding boxes of normal quality and improve the overall performance of the modified YOLOv5 model (i.e., the crack detection model).

[0131] S112: Utilize the CA attention mechanism module to weight the feature points according to the spatial position information of the feature points of the crack image.

[0132] Specifically, with respect to the features of cracks, this embodiment can use the CA attention mechanism module to weight the features of cracks according to the spatial position information of feature points, so that the modified YOLOv5 model pays more attention to features related to crack targets, thereby improving the perception and discrimination capabilities of the modified YOLOv5 model.

[0133] For example, Figure 2 An example structural diagram of a CA attention mechanism module provided for this embodiment. The specific workflow of the CA attention mechanism module may include: the convolution of the input feature map is decomposed into feature encodings in two directions, horizontal and vertical, and one-dimensional global average pooling is used to aggregate features in the horizontal and vertical directions respectively to generate feature maps perceived in two directions. The horizontal and vertical feature maps obtained above are spliced ​​along the spatial dimension, and an intermediate feature map is generated through a shared 1x1 convolution transformation function. The intermediate feature map is batch normalized and layer normalized, and then the segmented intermediate feature map is converted to the same size as the number of input channels through two 1x1 convolution transformations. Use the sigmoid function for normalization. Finally, the generated weights are applied to the input feature map and output to enhance the saliency of the object representation in the feature map.

[0134] S113: splicing the GhostNet module with the convolution module of the YOLOv5 model as the backbone network of the YOLOv5 model.

[0135] Specifically, in order to make the modified YOLOv5 model more lightweight, the present embodiment can combine the GhostNet module with the convolution module in the original YOLOv5 model (splicing or replacing to obtain the Ghost Bottleneck module) to obtain a lightweight backbone network. The modified YOLOv5 model has lower computational complexity and model parameter quantity while maintaining detection accuracy, and is suitable for deployment on mobile terminals and the cloud. Figure 3 This is a structural example diagram of a combination of a GhostNet module and a convolution module provided in this embodiment.

[0136] S114: Connect the SPPFCSPC module with the feature extraction module of the YOLOv5 model.

[0137] For example, Figure 4 This is an example structural diagram of an SPPFCSPC module provided in this embodiment.

[0138] Reference Figure 4In this embodiment, the SPPFCSPC module (SPPFCSPC multi-scale feature pyramid) can be spliced ​​with the feature extraction module of the YOLOv5 model: through a 1x1 convolution, a 3x3 convolution and a 1x1 convolution layer, preliminary features are extracted and the number of channels is reduced, and then the maximum pooling layer is used to generate feature maps of different scales. After splicing the feature maps of different scales, a 1x1 convolution layer and a 3x3 convolution are passed to generate a fusion feature map y1; at the same time, the input feature map directly generates a feature map y2 through a 1x1 convolution layer, and finally y1 and y2 are spliced, integrated through a 1x1 convolution layer, and the final feature map is output. The multi-scale feature pyramid effectively avoids problems such as image distortion caused by cropping and scaling operations on image regions; solves the problem of repeated feature extraction of image-related features by convolutional neural networks, greatly improves the speed of generating candidate frames, and saves computing costs. The modified YOLOv5 model is more stable when processing scenes with large scale changes, thereby improving the model's ability to detect cracks of different scales.

[0139] S120: Using each of the crack images annotated with the bounding box as training data to train the crack detection model.

[0140] It can be understood that this embodiment can improve the performance of the YOLOv5 model by optimizing and modifying the YOLOv5 model, and use the modified YOLOv5 model as a crack detection model. Then, the crack image is used as a training sample and the bounding box is used as a training label to train the crack detection model.

[0141] Furthermore, S120 may include S121 to S125:

[0142] S121: Clustering the bounding boxes to obtain target bounding boxes of multiple categories.

[0143] Furthermore, S121 may include S1211 to S1213:

[0144] S1211: Normalizing the width and height of each of the bounding boxes, and converting the normalized width and height into two-dimensional data points;

[0145] S1212: clustering each of the data points using a K-means algorithm, and clustering each of the data points into K clusters, so that the sum of the distances from each of the data points to the center point of the cluster to which it belongs is minimized; wherein K is a positive integer;

[0146] S1213: Convert the K center points obtained by clustering into widths and heights, and then denormalize the converted K widths and heights to restore them to the original scale, so as to obtain the target bounding boxes of K categories.

[0147] Specifically, the width and height of all bounding boxes are collected from the training data set, the width and height of the bounding boxes are normalized, and then the normalized width and height are used as two-dimensional data points, each bounding box corresponds to a data point, and illustratively, the number of clusters K=9 is determined, and the K-means algorithm is used to cluster the data points, and the data points are assigned to 9 clusters so that the sum of the distances from each data point to the center point of the cluster to which it belongs is minimized. Figure 5 An example diagram of clustering results provided for this embodiment. Then, the 9 center points obtained by clustering are denormalized back to the original scale to obtain the width and height of 9 representative bounding boxes (i.e., target bounding boxes). The target bounding box generated by K-means clustering can better match the distribution of bounding boxes in the training data, so it can improve the matching degree of the crack detection model, thereby improving the crack detection accuracy. At the same time, the predefined target bounding box size and shape are closer to the real bounding box, which can reduce the problem of imbalance between positive and negative samples, thereby simplifying the training process of the model and improving the convergence speed. In addition, setting target bounding boxes of different sizes for feature maps of different depths is to effectively process targets of different scales. This embodiment can use the spatial information retained by shallower feature maps to detect larger cracks, and use the semantic information retained by deeper feature maps to detect smaller cracks.

[0148] S122: Randomly scale and crop the crack image annotated with the target bounding box to obtain a plurality of training images.

[0149] First, the pre-trained model of the COCO dataset is used, and the weight parameters of the pre-trained model are used as the initial values. By fine-tuning on the training dataset, the convergence of the model is accelerated and the detection performance is improved.

[0150] This embodiment may adopt a multi-scale training method. During the training process, the input image is randomly scaled and cropped to simulate cracks of different scales, and multi-scale feature maps are used for detection during training, thereby improving the generalization ability of the model.

[0151] S123: Extracting texture features, geometric features, shape and category features of each of the training images.

[0152] Specifically, multi-level features are extracted and fused to improve the detection capability of the model by combining features at different levels.

[0153] Low-level features: These features usually correspond to the early layers of the crack detection model, which can capture basic visual information such as edges, textures, and colors in the input image. Low-level features are sensitive to image details and local features.

[0154] Mid-level features: As the network layers of the crack detection model increase, features begin to combine low-level features to form more complex patterns and structures. Mid-level features are able to capture higher-level visual information, such as partial shapes of objects and simple geometric structures.

[0155] High-level feature layers: At deeper levels of the network, features become more abstract and can represent more complex objects and scene information. High-level features are usually closely related to the semantic information of the input data and can capture the overall shape and category of the object.

[0156] Exemplarily, the feature map obtained by the 4th layer "Ghost Bottleneck" feature extraction module is fused with the feature map after upsampling by the 15th layer convolution module; the feature map obtained by the 6th layer "C3" feature extraction module is fused with the feature map after upsampling by the 11th layer convolution module; the feature maps extracted by the 15th layer and the 19th layer convolution modules are fused; the feature maps extracted by the 22nd layer and the 11th layer convolution modules are fused.

[0157] S124: Dimensionally aligning and fusing the texture features, the geometric features, and the shape and category features through convolution and bilinear interpolation methods to obtain a fused feature map.

[0158] Specifically, 1×1 convolution and bilinear interpolation methods are used to upsample the features, 2×2 convolution with a stride of 2 is applied to achieve 2 times of downsampling, and 4×4 convolution with a stride of 4 is applied to achieve 4 times of downsampling, thereby aligning the dimensions for feature fusion.

[0159] S125: Using the fused feature map as a training sample and the target bounding box as a training label, training the crack detection model, and using the Wise-IoU algorithm as a loss function to adjust the weight parameters of the crack detection model.

[0160] Further, using the Wise-IoU algorithm as a loss function to adjust the weight parameters of the crack detection model may include S1251 to S1255:

[0161] S1251: Initialize the weight parameters of the crack detection model;

[0162] S1252: Inputting the fused feature map marked with the target bounding box into the crack detection model to obtain a prediction result;

[0163] S1253: Calculate the classification loss value and bounding box regression loss value of the result using the Wise-IoU algorithm;

[0164] S1254: Calculate the gradients of the classification loss value and the bounding box regression loss value relative to each of the weight parameters using a back propagation algorithm;

[0165] S1255: Adjust the weight parameters of the crack detection model according to the gradient to reduce the classification loss value and the bounding box regression loss value.

[0166] Specifically, the weight parameters of the crack detection model are initialized using the pre-trained model, and the Wise-IoU algorithm is used to calculate the loss function value, including classification loss and bounding box regression loss: the input feature map is passed to the crack detection model, and the model generates a predicted bounding box and category probability (i.e., prediction result). The loss value is calculated based on the model's prediction result and the true annotation. The gradient of the loss relative to each weight parameter is calculated using the back propagation algorithm. The Adam optimizer is used to update the model's weight parameters according to the calculated gradient, so that the loss function value gradually decreases. The above process is repeated many times on the entire training data set until the model converges, that is, the loss function no longer decreases significantly or reaches the predetermined number of training rounds. The final detection model weight parameters are saved, and then crack detection is performed.

[0167] S130: Perform crack detection on the crack image using the trained crack detection model.

[0168] Exemplarily, this embodiment can deploy the trained crack detection model to a mobile terminal or a cloud to perform crack detection.

[0169] Deploy crack detection model on mobile terminals: Deploy the optimized crack detection model on mobile terminals to monitor concrete surface cracks in real time anytime and anywhere. Users only need to use the camera of mobile devices such as mobile phones or tablets to take crack images, and then use the model to detect cracks and take corresponding maintenance measures in time.

[0170] Deploy the crack detection model on the cloud server: Deploy the crack detection model on the cloud server to process large-scale crack images and support multiple users to perform crack detection simultaneously. Users can upload images or take real-time photos to send data to the cloud for processing, thereby realizing the monitoring and analysis of cracks in multiple locations or large areas.

[0171] Combined with the crack detection model deployed on the mobile terminal or cloud, real-time monitoring of cracks can be achieved. When in a new scene, image data with high detection and recognition accuracy will be automatically selected as retraining samples, and annotation files will be automatically generated. Retraining can be performed to generate model files that can handle new scene detection, and the detection capability can be continuously optimized. Regarding the detection and optimization of new scenes, Figure 6 An example flow chart of crack detection provided in this embodiment.

[0172] In order to evaluate the performance of the crack detection model, the embodiment of the present application may further include S140, a step of calculating the performance parameters of the trained crack detection model. S140 may include:

[0173] Calculating the precision, recall, average accuracy, parameter quantity and computational complexity of the trained crack detection model as the performance parameters;

[0174] The calculation formula of the accuracy is:

[0175]

[0176] The calculation formula of the recall rate is:

[0177]

[0178] The calculation formula of the average accuracy is:

[0179]

[0180] The calculation formula of the parameter quantity is:

[0181]

[0182] The calculation formula of the calculation amount is:

[0183]

[0184] Wherein, P represents the precision, TP represents the number of samples that are judged as positive samples by the trained crack detection model and are actually positive samples, FP represents the number of samples that are judged as positive samples by the trained crack detection model and are actually negative samples; R represents the recall, FN represents the number of samples that are judged as negative samples by the trained crack detection model and are actually positive samples; AP represents the average precision; Parameters represents the parameter quantity, L represents the total number of network layers of the trained crack detection model, and P i represents the number of parameters of the i-th layer network; FLOPs represents the amount of calculation, N represents the total number of operations in the crack detection model after training, Ops i represents the number of times the i-th operation is executed, F i Indicates the amount of computation required for the i-th operation.

[0185] Exemplarily, referring to Table 1, this embodiment provides a performance parameter table of the YOLOv5 model combined with different modules or algorithms.

[0186] Table 1

[0187] Precision Recall AP Parameters FLOPs YOLOv5s 0.748 0.639 0.686 7022326 15.9G YOLOv5n+CA 0.788 0.667 0.731 1771950 4.2G YOLOv5n+GhostBottleneck 0.691 0.611 0.673 1742966 3.9G YOLOv5n+SPPFCSPC 0.812 0.653 0.754 3373718 5.5G YOLOv5n+WIoU 0.727 0.819 0.755 1765270 4.2G YOLO_Crack 0.816 0.708 0.774 3358094 5.2G

[0188] The embodiment of the present application improves the YOLOv5 model to better adapt to crack detection, and the modified YOLOv5 model can be installed on a cloud server or a mobile terminal as a crack detection model, and has the ability to automatically screen and annotate new scene data and optimize existing models.

[0189] The embodiments of the present application have the following beneficial effects:

[0190] High accuracy: Based on the YOLOv5 algorithm, which is currently more mature in industrial applications, Wise-IOU, CA attention mechanism and SPPFCSPC spatial pyramid pooling structure are introduced to improve the accuracy of crack detection. Through large-scale data training and continuous updating of optimization models, the crack detection model can better detect cracks, thereby reducing the error rate and improving the accuracy of prediction.

[0191] High efficiency: By optimizing algorithms and workflows, training and testing can be performed at a faster speed. Modern computing resources can be fully utilized to quickly perform testing tasks. This high efficiency not only saves time costs, but also improves productivity and work efficiency, allowing users to obtain the desired results faster.

[0192] Low footprint: By reducing the depth of the YOLOv5 model and replacing one of its convolutional modules with GhostBottleneck, the computing system resources are reduced. This allows the crack detection model to maintain high performance while reducing the demand for hardware resources. Low footprint makes the crack detection model more portable and flexible, and can run in resource-limited environments, while reducing operation and maintenance costs and energy consumption.

[0193] Continuous optimization: It can perform real-time detection based on the detected data. When a new scene appears, it can screen detection pictures with higher accuracy and generate annotation files for the corresponding pictures. It can also conduct re-training on the original basis, so that the crack detection model can continuously improve its own model to adapt to the ever-changing environment and needs.

[0194] Next, the solution of the embodiment of the present application will be introduced and explained in conjunction with specific application examples:

[0195] Reference Figure 7 , this embodiment provides an example structural diagram of a crack detection model. Figure 8 Figure 2 is an example of the corresponding crack detection results.

[0196] Specifically, the workflow of each layer network of the crack detection model of this embodiment is described as follows:

[0197] Layer 0: Input layer, using convolution module, input channels are 3 (RGB images), output channels are 16, convolution kernel size is 6, stride is 2, 2 (indicates padding and stride during convolution).

[0198] Layer 1: The first convolutional layer with 16 input channels, 32 output channels, a kernel size of 3, and a stride of 2.

[0199] Layer 2: Use the “C3” feature extraction module with 32 input channels, 32 output channels, and a stride of 1.

[0200] Layer 3: The second convolutional layer with 32 input channels, 64 output channels, a kernel size of 3, and a stride of 2.

[0201] Layer 4: Use the “Ghost Bottleneck” feature extraction module with 64 input channels and 64 output channels.

[0202] Layer 5: The third convolutional layer, with 64 input channels, 128 output channels, a convolution kernel size of 3, and a stride of 2.

[0203] Layer 6: Use the “C3” feature extraction module with 128 input channels, 128 output channels, and a stride of 3.

[0204] Layer 7: The fourth convolutional layer, with an input channel of 128, an output channel of 256, a convolution kernel size of 3, and a stride of 2.

[0205] Layer 8: Uses the “CA” attention mechanism module with 256 input channels and 256 output channels.

[0206] Layer 9: Use the “C3” feature extraction module with 256 input channels, 256 output channels, and a stride of 1.

[0207] Layer 10: Use the spatial pyramid pooling module with 256 input channels, 256 output channels, and a spatial pyramid pooling kernel size of 5.

[0208] Layer 11: Use a convolutional module with 256 input channels, 128 output channels, a kernel size of 1, and a stride of 1.

[0209] Layer 12: Use upsampling to increase the resolution of the feature map.

[0210] Layer 13: Concatenates the feature maps of the `C3` feature extraction module in layer 6.

[0211] Layer 14: Use the “C3” feature extraction module with 256 input channels, 128 output channels, and a stride of 1.

[0212] Layer 15: Use a convolutional module with 128 input channels, 64 output channels, a kernel size of 1, and a stride of 1.

[0213] Layer 16: Use upsampling to increase the resolution of the feature map.

[0214] Layer 17: Concatenates the feature maps of the 4th layer's `Ghost Bottleneck` feature extraction module.

[0215] Layer 18: Use the “C3” feature extraction module with 128 input channels, 64 output channels, and a stride of 1.

[0216] Layer 19: Use a convolutional module with 64 input channels, 64 output channels, a kernel size of 3, and a stride of 2.

[0217] Layer 20: Concatenate the feature maps of layer 15 after the convolution module.

[0218] Layer 21: Use the “C3” feature extraction module with 128 input channels, 128 output channels, and a stride of 1.

[0219] Layer 22: Use a convolutional module with 128 input channels, 128 output channels, a kernel size of 3, and a stride of 2.

[0220] Layer 23: Concatenate the feature maps of layer 11 after convolution module.

[0221] Layer 24: Use the “C3” feature extraction module with 256 input channels, 256 output channels, and a stride of 1.

[0222] Layer 25: Detection is performed using three anchor box sizes of different scales and the number of channels of their corresponding feature maps.

[0223] Reference Fig. 9 The embodiment of the present application further provides a crack detection device, which can implement the above crack detection method, and the device includes:

[0224] An image acquisition unit, used for acquiring a plurality of crack images and marking a boundary box for the crack in each of the crack images;

[0225] A model modification unit, used for obtaining a YOLOv5 model, fitting a Wise-IoU algorithm, a CA attention mechanism module, a GhostNet module, and a SPPFCSPC module to the YOLOv5 model, and obtaining the modified YOLOv5 model as a crack detection model;

[0226] A model training unit, configured to train the crack detection model using each of the crack images annotated with the bounding box as training data;

[0227] A crack detection unit is used to perform crack detection on the crack image using the trained crack detection model.

[0228] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0229] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above crack detection method when executing the computer program. The electronic device can be any smart terminal including a tablet computer, a car computer, etc.

[0230] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0231] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0232] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0233] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the crack detection method of the embodiment of this application;

[0234] Input / output interface 1003, used to implement information input and output;

[0235] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0236] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0237] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0238] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned crack detection method is implemented.

[0239] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0240] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0241] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0242] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0243] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0244] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0245] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0246] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0247] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0248] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0249] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0250] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0251] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A crack detection method, characterized in that: The method comprises the following steps: Acquire multiple crack images, and mark a boundary box for the crack in each of the crack images; Obtain a YOLOv5 model, fit the Wise-IoU algorithm, the CA attention mechanism module, the GhostNet module, and the SPPFCSPC module to the YOLOv5 model, and obtain the modified YOLOv5 model as a crack detection model; Training the crack detection model using each of the crack images annotated with the bounding box as training data; The trained crack detection model is used to perform crack detection on the crack image.

2. A crack detection method according to claim 1, characterized in that: The step of acquiring a plurality of crack images and marking a boundary box for a crack in each of the crack images comprises the following steps: Under different weather conditions, different regions and different time conditions, multiple images of concrete surface cracks are collected as the first image set; Deleting the images whose resolution is lower than a set threshold and the images that are blocked or repeated from among the first images to obtain a second image set; Marking the cracks in the second image set with bounding boxes, and determining the coordinates of the upper left corner point and the lower right corner point of each protected bounding box; Convert the coordinates of the upper left corner point and the coordinates of the lower right corner point of each of the bounding boxes into corresponding center point coordinates, width length, and height length, and normalize each of the center point coordinates, width length, and height length to obtain a third image set; Rotating, scaling, flipping, cropping and splicing each image in the third image set to obtain a fourth image set; The pixels of each image in the fourth image set are normalized to obtain each crack image.

3. A crack detection method according to claim 1, characterized in that: The step of fitting the Wise-IoU algorithm, the CA attention mechanism module, the GhostNet network, and the SPPFCSPC module to the YOLOv5 model to obtain the modified YOLOv5 model as a crack detection model includes the following steps: Using the Wise-IoU algorithm to adjust the gradient gain of the YOLOv5 model according to the outlier degree of the bounding box; Using the CA attention mechanism module to weight the feature points according to the spatial position information of the feature points of the crack image; Splicing the GhostNet module with the convolution module of the YOLOv5 model as the backbone network of the YOLOv5 model; The SPPFCSPC module is spliced ​​with the feature extraction module of the YOLOv5 model.

4. A crack detection method according to claim 1, characterized in that: The method of training the crack detection model by using each of the crack images marked with the bounding box as training data comprises the following steps: Clustering each of the bounding boxes to obtain target bounding boxes of multiple categories; Randomly scaling and cropping the crack image annotated with the target bounding box to obtain a plurality of training images; Extracting texture features, geometric features, shape and category features of each of the training images; Performing dimension alignment and feature fusion on the texture features, the geometric features, and the shape and category features by using convolution and bilinear interpolation methods to obtain a fused feature map; The fused feature map is used as a training sample, the target bounding box is used as a training label, the crack detection model is trained, and the Wise-IoU algorithm is used as a loss function to adjust the weight parameters of the crack detection model.

5. A crack detection method according to claim 4, characterized in that: The step of clustering the bounding boxes to obtain target bounding boxes of multiple categories includes the following steps: Normalizing the width and height of each of the bounding boxes, and converting the normalized width and height into two-dimensional data points; Using the K-means algorithm to cluster the data points into K clusters, so that the sum of the distances from the data points to the center point of the cluster to which they belong is minimized; wherein K is a positive integer; The K center points obtained by clustering are converted into widths and heights, and then the converted K widths and heights are denormalized and restored to the original scale to obtain the target bounding boxes of K categories.

6. A crack detection method according to claim 4, characterized in that: The method of using the Wise-IoU algorithm as a loss function to adjust the weight parameters of the crack detection model includes the following steps: Initializing the weight parameters of the crack detection model; Inputting the fused feature map marked with the target bounding box into the crack detection model to obtain a prediction result; Calculate the classification loss value and bounding box regression loss value of the result using the Wise-IoU algorithm; Calculating the gradients of the classification loss value and the bounding box regression loss value relative to each of the weight parameters using a back propagation algorithm; The weight parameters of the crack detection model are adjusted according to the gradient to reduce the classification loss value and the bounding box regression loss value.

7. A crack detection method according to any one of claims 1 to 6, characterized in that: The method further comprises the step of calculating the performance parameters of the trained crack detection model, wherein the step of calculating the performance parameters of the trained crack detection model comprises: Calculating the precision, recall, average accuracy, parameter quantity and computational complexity of the trained crack detection model as the performance parameters; The calculation formula of the accuracy is: The calculation formula of the recall rate is: The calculation formula of the average accuracy is: The calculation formula of the parameter quantity is: The calculation formula of the calculation amount is: Wherein, P represents the precision rate, TP represents the number of samples that are judged as positive samples by the trained crack detection model and are actually positive samples, FP represents the number of samples that are judged as positive samples by the trained crack detection model and are actually negative samples; R represents the recall rate, FN represents the number of samples that are judged as negative samples by the trained crack detection model and are actually positive samples; AP represents the average precision; Parameters represents the parameter quantity, L represents the total number of network layers of the trained crack detection model, and P i represents the number of parameters of the i-th layer network; FLOPs represents the amount of calculation, N represents the total number of operations in the crack detection model after training, Ops i represents the number of times the i-th operation is executed, F i Indicates the amount of computation required for the i-th operation.

8. A crack detection device, characterized in that: The device comprises: An image acquisition unit, used for acquiring a plurality of crack images and marking a boundary box for the crack in each of the crack images; A model modification unit, used for obtaining a YOLOv5 model, fitting a Wise-IoU algorithm, a CA attention mechanism module, a GhostNet module, and a SPPFCSPC module to the YOLOv5 model, and obtaining the modified YOLOv5 model as a crack detection model; A model training unit, configured to train the crack detection model using each of the crack images annotated with the bounding box as training data; A crack detection unit is used to perform crack detection on the crack image using the trained crack detection model.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements a crack detection method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a crack detection method according to any one of claims 1 to 7 is implemented.

Images