Crack detection method and device applied to unmanned aerial vehicle and medium

Abstract

The invention relates to a crack detection method and device applied to an unmanned aerial vehicle, and a medium, and the method comprises the steps: inputting a to-be-detected image obtained by an unmanned aerial vehicle camera into a crack detection algorithm network model, and obtaining a crack detection result, the backbone network module comprises a single convolution sub-module, a twice and multi-scale convolution sub-module and a spatial pyramid pooling fast module which are in alternative linear connection, and the twice and multi-scale convolution sub-modules of a first number are connected with the feature fusion sub-modules of a second number of the neck network module; the two-time and multi-scale convolution submodule comprises a two-time convolution module and a multi-scale convolution module, the two-time convolution module takes the output of the single-time convolution submodule as the input to carry out gradient shunt convolution, and the multi-scale convolution module takes the gradient shunt convolution result of the two-time convolution module as the input to carry out multi-scale convolution; obtaining two times of convolution and a multi-scale convolution result; the feature fusion sub-module is constructed to only comprise a bottom-up network structure and a top-down network structure without convolution operation after up-sampling.

Description

Technical Field

[0001] The present disclosure relates to the technical field of building exterior wall defect recognition, and in particular to a crack detection method, device and medium applied to an unmanned aerial vehicle. Background Art

[0002] Building maintenance is a major issue that needs to be taken seriously. As one of the most important safety hazards of buildings, the intelligent detection of exterior wall cracks is of great significance to the control of high-rise building use risks and building maintenance. Traditional building appearance inspections usually require professionals to bring special tools to the inspection site and use visual observation, hammering and other techniques for evaluation, or use electromagnetic radiation and stress wave diffusion to detect structural and building defects (such as cracks in buildings). For cement surface inspections, it can be judged directly by vision.

[0003] However, crack detection methods are only detected through human vision, which may lead to subjective errors because the wall often contains different types of materials, various surface textures and uneven cracks. Therefore, in order to reduce human errors, edge detection, wavelet transform and other image-based filtering methods can be used for crack detection. However, crack detection using edge detection, wavelet transform and other image-based filtering methods is still very labor-intensive, especially in dealing with background elimination and noise reduction in complex lighting environments, which will take a lot of time. In addition, small grooves on the wall may be mistaken for cracks or cracks, increasing the possibility of misjudgment. Summary of the invention

[0004] The purpose of the present invention is to provide a crack detection method, device and medium applied to unmanned aerial vehicles, aiming to reduce model parameters and calculation amount, and improve the accuracy of wall crack detection.

[0005] In order to achieve the above-mentioned purpose, a first aspect of an embodiment of the present disclosure provides a crack detection method applied to a drone, the method comprising:

[0006] In response to the crack detection operation of the drone starting, an image to be detected is acquired through a camera configured for the drone;

[0007] Input the image to be detected acquired by the camera configured by the unmanned aerial vehicle into the crack detection algorithm network model to obtain the crack detection result output by the crack detection algorithm network model, wherein the backbone network module of the crack detection algorithm network model includes a plurality of linearly connected single convolution submodules, two-time and multi-scale convolution submodules and spatial pyramid pooling fast modules, wherein the single convolution submodules and the two-time and multi-scale convolution submodules are alternately configured, and a first number of the two-time and multi-scale convolution submodules are respectively connected to a second number of feature fusion submodules of the neck network module of the crack detection algorithm network model, and the first number is less than the second number;

[0008] Among them, the double and multi-scale convolution submodule includes a double convolution module and a multi-scale convolution module, the double convolution module uses the output of the single convolution submodule as input to perform gradient shunt convolution to obtain a gradient shunt convolution result, the multi-scale convolution module uses the gradient shunt convolution result output by the double convolution module as input to perform multi-scale convolution to obtain double convolution and multi-scale convolution results corresponding to the double and multi-scale convolution submodules, and the spatial pyramid pooling fast module aggregates features of different receptive fields through multiple sequential maximum pooling operations;

[0009] The feature fusion submodule in the neck network module is constructed to not include the convolution operation after upsampling, and only includes bottom-up and top-down network structures.

[0010] According to a second aspect of the present disclosure, an electronic device is provided, including:

[0011] a memory having a computer program stored thereon;

[0012] A processor is used to execute the computer program in the memory to implement the steps of any one of the methods in the first aspect.

[0013] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0014] The present invention provides a crack detection method, device and medium for use in drones. Compared with the prior art, it has the following beneficial effects:

[0015] By alternately configuring single convolution submodules and double and multi-scale convolution submodules, combined with the spatial pyramid pooling fast module, it is possible to extract richer multi-scale features and improve the model's ability to identify cracks. The design of the multi-level feature fusion extraction module helps to reduce model parameters and computation while ensuring feature extraction capabilities. The AFPN structure further reduces model parameters and computation, making the model more lightweight. In this way, by optimizing the backbone network module, neck network module, and detection head of the algorithm network model, the effect of enhancing the network feature extraction capability, reducing model parameters and computation, and improving detection accuracy and speed in the UAV crack detection task is achieved. This enables the UAV to complete the crack detection task more efficiently and accurately.

[0016] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0018] Figure 1 It is a flow chart of a crack detection method applied to a drone according to an embodiment of the specification.

[0019] Figure 2 It is a structural schematic diagram of a crack detection algorithm network model shown in an embodiment of the specification.

[0020] Figure 3 It is a structural schematic diagram of a multi-scale convolution module according to an embodiment of the specification.

[0021] Figure 4 It is a schematic diagram of a spatial weight allocation according to an embodiment of the specification.

[0022] Figure 5 It is a structural schematic diagram of a neck network module according to an embodiment of the specification.

[0023] Figure 6 It is a schematic diagram of the structure of a double convolution module according to an embodiment of the specification.

[0024] Figure 7 This is an image after annotating a crack image shown in an embodiment of the specification.

[0025] Figure 8 It is a schematic diagram of dynamically allocating sample labels of crack images of training samples in a training set according to an embodiment of the specification.

[0026] Fig. 9It is a schematic diagram of dynamically allocating sample labels of crack images of training samples in a training set according to an embodiment of the specification.

[0027] Fig.10 These are comparison images of recognition results before and after the model is improved according to the embodiments of the specification.

[0028] Fig.11 It is a block diagram of a crack detection device applied to a drone according to an embodiment of the specification. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0031] Before introducing the crack detection method for drones provided by the present disclosure, the technology in the relevant scenarios is first introduced. For example, using a drone equipped with an image recognition device to automatically detect cracks can not only improve the efficiency of detection, but also improve the accuracy of detection. However, the load capacity of drones is limited, and the computing power of the image equipment carried is very limited. In order to further improve the accuracy of detection and the computing power of drones, crack detection based on YOLO is proposed. However, crack detection based on YOLO has deficiencies in efficiency, accuracy and real-time performance. Therefore, in order to solve the deficiencies of the YOLO detection method in terms of efficiency, accuracy and real-time performance, there are many research schemes.

[0032] The Adaptive Attention Enhanced YOLO model (AAEY) is applied to building wall crack detection. By combining the Transformer's self-attention mechanism, the detection accuracy and stability can be improved. In the experiment, the high robustness and generalization ability of the AAEY model in complex environments have been verified. Further combining traditional image processing technology with the YOLO model can achieve efficient detection of wall cracks, especially in resource-constrained environments.

[0033] Furthermore, the improved YOLOv3 algorithm solves the problems of poor real-time performance and low precision of traditional detection methods. The YOLOv3 algorithm annotates crack images and trains models based on annotated images to improve detection accuracy and speed. Experiments show that its detection accuracy reaches 88%, and its speed is better than other detection algorithms. Similarly, the YOLOv2 framework is used to improve detection efficiency and consistency through transfer learning and data increment, although there is still room for improvement in some crack types.

[0034] Furthermore, by combining the generative adversarial network (PCGAN) with the improved version of YOLOv3 (YOLO-MF), the problem of insufficient data in pavement crack detection is solved, and efficient detection and tracking are achieved. The experimental results show that the system has high performance in terms of accuracy and real-time performance. Based on the MOD-YOLO algorithm, the accuracy and robustness of crack detection in civil infrastructure are improved by maintaining the original dimension of feature information and enhancing the global receptive field. The experimental results prove the good generalization ability of MOD-YOLO on different datasets and demonstrate its application value in real-time detection. Aiming at the application environment of edge computing devices, the EMG-YOLO algorithm is proposed. By introducing the efficiency decoupling head and GCC3 module, the problems of high computational burden and poor data quality are effectively solved, and the detection accuracy and efficiency are significantly improved.

[0035] In terms of crack detection, the YOLO algorithm is improved by introducing anchor boxes and convolutional layer optimization to better detect small and overlapping targets. Experimental results show that the detection accuracy and speed are improved, especially in real-time applications. Combining YOLOv6 3.0 and YOLOv7, the RSG-YOLO model is proposed. Through improved feature fusion and loss function, the accuracy and recall rate are improved, especially in complex backgrounds. The improved YOLOv3 is combined with MobileNets and convolutional block attention module (CBAM) to achieve improved detection speed and accuracy. The experiment verifies the effectiveness of this method in bridge deck crack detection and demonstrates its application potential on UAV platforms. The YOLO algorithm is integrated into the UAV, and its effectiveness under different environmental conditions is verified through experiments, which emphasizes the application potential of the YOLO UAV platform.

[0036] In summary, through different improvements to the YOLO model, the performance of crack detection has been effectively improved, showing its wide application potential in various fields. However, the performance of different models in specific application scenarios still needs to be improved. The most prominent problem is that they are generally based on old YOLO versions, and the obtained models are not lightweight enough, and their potential for application on UAV platforms is still limited.

[0037] In view of this, the present disclosure provides a crack detection method applied to a drone. Figure 1 The following is a flow chart of a crack detection method applied to a drone according to an embodiment. Specifically, the method includes:

[0038] In step S11, in response to the crack detection operation of the drone starting, an image to be detected is acquired through a camera configured on the drone;

[0039] Among them, an unmanned aerial vehicle (UAV) can be an aircraft that can fly automatically or be remotely controlled without the need for an onboard pilot, and can be, for example, a four-winged unmanned aerial vehicle equipped with one or more ordinary cameras or high-definition cameras.

[0040] Crack Detection is the process of using the drone’s camera to capture images, use machine vision and other technical means to automatically identify and mark cracks on the surface of objects (such as roads, building walls, etc.).

[0041] In the disclosed embodiment, the drone flies according to a preset route or manual operation through a built-in flight control system to ensure that the camera can capture images of the target area. The camera, as a sensing device of the drone, is responsible for capturing images of the target area. The working principle of the camera is based on the principle of optical imaging. The lens focuses light on a photosensitive element (such as CMOS or CCD), thereby converting light signals into electrical signals to form digital images. The image acquisition system on the drone is responsible for transmitting the image data captured by the camera to the drone's storage system in real time.

[0042] In step S12, the image to be detected acquired by the camera configured by the UAV is input into the crack detection algorithm network model to obtain the crack detection result output by the crack detection algorithm network model.

[0043] In the disclosed embodiment, the acquired image to be detected is used as input and passed to a pre-trained crack detection algorithm network model. The crack detection algorithm network model can be a convolutional neural network (CNN) structure in deep learning, which can automatically extract hierarchical feature information from the input image. The network model gradually extracts local features (such as edges, textures, etc.) in the image through multi-layer convolution and pooling operations, and then forms a global feature representation. Based on the extracted feature information, the network model classifies each pixel or area in the image through a fully connected layer or classifier to determine whether it belongs to a crack.

[0044] Wherein, the backbone network module of the crack detection algorithm network model includes a plurality of linearly connected single convolution submodules, two-time and multi-scale convolution submodules and spatial pyramid pooling fast modules, wherein the single convolution submodules and the two-time and multi-scale convolution submodules are alternately configured, and a first number of the two-time and multi-scale convolution submodules are respectively connected to a second number of feature fusion submodules of the neck network module of the crack detection algorithm network model, and the first number is less than the second number;

[0045] Among them, the double and multi-scale convolution submodule includes a double convolution module and a multi-scale convolution module, the double convolution module uses the output of the single convolution submodule as input to perform gradient shunt convolution to obtain a gradient shunt convolution result, the multi-scale convolution module uses the gradient shunt convolution result output by the double convolution module as input to perform multi-scale convolution to obtain double convolution and multi-scale convolution results corresponding to the double and multi-scale convolution submodules, and the spatial pyramid pooling fast module aggregates features of different receptive fields through multiple sequential maximum pooling operations;

[0046] The feature fusion submodule in the neck network module is constructed to not include the convolution operation after upsampling, and only includes bottom-up and top-down network structures.

[0047] Among them, see Figure 2 The network structure shown in the figure, Backbone Network Module: In deep learning models, especially convolutional neural networks (CNNs), the basic network structure responsible for extracting image features usually contains multiple convolutional layers, pooling layers, etc., which are used to generate high-level feature representations of images.

[0048] Single Convolution Submodule: A module that performs a single convolution operation, usually containing a convolution layer, used to extract local features of the image.

[0049] Double and Multi-scale ConvolutionSubmodule: A module that contains double convolution operations and multi-scale convolution operations, which aims to further extract image features and enhance the model's ability to capture features of different scales through multi-scale convolution.

[0050] Gradient splitting convolution uses a specific convolution kernel design or weight distribution strategy to enable the convolution result to reflect the gradient changes of the input image in different directions or scales, thereby enhancing the model's capture of image detail features.

[0051] The Spatial Pyramid Pooling Fast Module (SPPF) performs maximum pooling operations at different scales to capture features in different receptive fields and enhance the model's ability to integrate global and local information of the image.

[0052] Neck Network Module: In tasks such as target detection, the network structure located between the backbone network module and the detection head is usually used for feature fusion, feature enhancement, etc. to improve the model's detection performance for target objects.

[0053] Feature Fusion Submodule: In the neck network module, it is used to fuse feature information from different levels or scales to improve the model's ability to handle complex scenes.

[0054] In the disclosed embodiment, the single convolution submodule Conv serves as a basic building block of the backbone network module, and extracts local features of the image through a single convolution operation.

[0055] The double and multi-scale convolution submodule C2f_MSB is based on the single convolution submodule. It further extracts features through double convolution operations and enhances the model's ability to capture features of different scales through multi-scale convolution. This alternating configuration helps the model gradually deepen its understanding of image features.

[0056] In the disclosed embodiment, the output of the single convolution submodule is received as input, and the gradient information of the image in different directions or scales is extracted through the gradient shunting convolution operation, thereby enhancing the model's ability to capture image detail features. The output of the double convolution module is used as input to perform multi-scale convolution operations to capture feature information at different scales. This design helps the model to focus on both global and local information when processing complex scenes.

[0057] In the disclosed embodiment, the spatial pyramid pooling fast module captures feature information under different receptive fields by performing maximum pooling operations at different scales. This design helps the model maintain consistent performance when processing input images of different sizes. Pooling results at different scales are integrated together to form a richer feature representation, providing strong support for subsequent detection tasks.

[0058] In the disclosed embodiment, the feature fusion submodule in the neck network module adopts a bottom-up and top-down network structure, and improves the model's ability to handle complex scenes by fusing feature information at different levels. This design helps the model to use global and local information at the same time when detecting target objects. In order to maintain the original information of the features and reduce the amount of calculation, the feature fusion submodule does not use the convolution operation after upsampling during the fusion process. This design helps the model maintain high detection accuracy and efficiency when processing high-resolution images.

[0059] The above technical solution can extract richer multi-scale features and improve the model's ability to identify cracks by alternately configuring single convolution submodules and double and multi-scale convolution submodules, combined with a spatial pyramid pooling fast module. The design of the multi-level feature fusion extraction module helps to reduce model parameters and computational complexity while ensuring feature extraction capabilities. The AFPN structure further reduces model parameters and computational complexity, making the model more lightweight. In this way, by optimizing the backbone network module, neck network module, and detection head of the algorithm network model, the effect of enhancing the network feature extraction capability, reducing model parameters and computational complexity, and improving detection accuracy and speed in the UAV crack detection task is achieved. This enables the UAV to complete crack detection tasks more efficiently and accurately.

[0060] In a possible implementation, the multi-scale convolution module uses the gradient split convolution result output by the two convolution modules as input to perform multi-scale convolution, and obtains two convolution and multi-scale convolution results corresponding to the two and multi-scale convolution submodules, including:

[0061] In the multi-scale convolution module, according to the number of convolution branches in the multi-scale convolution module, the gradient split convolution result is divided into a plurality of feature subsets whose number of sets is the same as the number of convolution branches;

[0062] In the disclosed embodiment, in the multi-scale convolution module, first, according to the number of convolution branches, the gradient split convolution result is divided into multiple feature subsets equal to the number of convolution branches. The purpose of this step is to disperse the input data into different convolution branches for processing so as to capture features of different scales.

[0063] The data in each feature subset is assigned one by one to one of the convolution branches for convolution processing, wherein, inside the convolution branch, the input data is convolved for the first time through a convolution kernel of a first size, and the result of the first convolution is convolved through a convolution kernel of a second size for depth convolution, and then the result obtained by the depth convolution is convolved through a convolution kernel of a third size for the second time to obtain the convolution processing result of the convolution branch, and between the convolution branches, the convolution processing result corresponding to the previous convolution branch and the data in the feature subset assigned to the next convolution branch together constitute the input data of the next convolution branch;

[0064] In the disclosed embodiment, each feature subset is assigned to a convolution branch for processing. Within the convolution branch, the processing process generally includes three convolution operations:

[0065] First convolution: Convolve the input data using a convolution kernel of the first size to extract basic feature information.

[0066] Depth convolution: Take the result of the first convolution as input and perform depth convolution using the convolution kernel of the second size to further extract deep features. Depth convolution usually means that the number of channels of the convolution kernel is the same as the number of channels of the input data, allowing convolution operations to be performed independently within each channel.

[0067] Second convolution: Take the result of the depthwise convolution as input and perform a second convolution using a convolution kernel of the third size to further fuse and extract features.

[0068] Between convolution branches, the convolution processing result of the previous convolution branch and the data in the feature subset assigned to the next convolution branch together constitute the input data of the next convolution branch. This design helps to transfer information between convolution branches and enhance the feature extraction ability of the model.

[0069] The processing results of each convolution branch in the multi-scale convolution module are spliced ​​to obtain the double convolution and multi-scale convolution results corresponding to the double and multi-scale convolution sub-modules.

[0070] In the disclosed embodiment, the processing results of all convolution branches in the multi-scale convolution module are spliced ​​to obtain the double convolution and multi-scale convolution results corresponding to the double and multi-scale convolution submodules. The purpose of this step is to fuse the features extracted by different convolution branches so that the subsequent network layers can use these fused features for higher-level processing (such as classification, regression, etc.).

[0071] In this way, the multi-scale convolution module can effectively capture the features of the input data at different scales and improve the performance of the model.

[0072] Among them, the most important function of the multi-scale convolution module MSBlock is to improve the fusion capability of multi-scale features. MSBlock uses a hierarchical feature fusion strategy and a heterogeneous convolution kernel selection protocol to process feature fusion of different scales at different levels of the network, so that the model can better identify and process targets of different sizes. For example, see Figure 3 As shown in the figure, MSBlock improves the performance of the model in processing multi-scale information by improving the size and structure of the convolution kernel and optimizing the feature fusion method, thereby improving the overall target detection accuracy and efficiency.

[0073] In the MSBlock module, the input is split into multiple branches, each processing a different subset of features. Inside the branch, feature transformation is performed through 1x1 convolution, and then a kxk depth convolution is applied, followed by a 1x1 convolution to enhance the features and reduce the number of parameters. Finally, all branches are fused through a 1x1 convolution to integrate the features of each branch.

[0074] In order to further improve the feature fusion capability and utilize the results of parameter tuning of C2f modules at each level of YOLOv8, it is also necessary to reduce the number of parameters and computing power.

[0075] In a possible implementation, the feature fusion submodule in the neck network module performs feature fusion in the following manner:

[0076] Selecting the feature graphs {C3, C4, C5} output by the backbone network module in the last three layers as input features of the feature pyramid network formed by the feature fusion submodule in the neck network module;

[0077] In the disclosed embodiment, the output feature maps are selected from the last three layers (usually referred to as C3, C4, and C5) of the backbone network module as the input of the feature fusion submodule. These feature maps represent feature information of different scales, where C3 contains finer-grained detail information and C5 contains coarser-grained semantic information.

[0078] Perform a convolution with a convolution kernel of the fourth size and upsample the input features using bilinear interpolation to align the dimensions of each level;

[0079] In the disclosed embodiment, in order to enable the fusion of feature maps at different levels, their dimensions need to be aligned first. This is usually achieved through the following two steps:

[0080] Convolution: Perform convolution operation on each input feature map using a convolution kernel of the fourth size to adjust its number of channels. The purpose of this step is to ensure that the number of channels of all input feature maps is the same, which facilitates subsequent operations.

[0081] Upsampling: Bilinear interpolation is used to upsample the lower resolution feature maps to match the resolution of the highest resolution feature map. Bilinear interpolation is a commonly used image scaling method that can effectively increase the resolution of an image while maintaining image smoothness.

[0082] For the sampling features obtained by upsampling, downsampling is performed a first preset number of times using a convolution kernel of a fifth size with a first step length and downsampling is performed a second preset number of times using a convolution kernel of a sixth size with a second step length to adapt to different downsampling rates;

[0083] In the disclosed embodiment, in order to construct a feature pyramid network, it is necessary to perform downsampling of different scales on the upsampled feature map. Use the convolution kernel of the fifth size with the first step length to perform downsampling for the first preset number of times to obtain a first set of sampling features; then use the convolution kernel of the sixth size with the second step length to perform downsampling for the second preset number of times to obtain a second set of sampling features. Different downsampling rates and step sizes can capture feature information of different scales.

[0084] For example, 1×1 convolution and bilinear interpolation are used to upsample the features so that the dimensions of each level are aligned; 2×2 convolution with a stride of 2 is used to achieve 2 times of downsampling, and 4×4 convolution with a stride of 4 is used to achieve 4 times of downsampling to accommodate the different downsampling rates.

[0085] See also Figure 4 As shown, for the sampling features obtained by downsampling, spatial weight allocation is performed and then adaptive feature fusion is performed, wherein, if Represents the feature vector at position (i, j) from the nth layer to the lth layer, l = 1, 2, 3, corresponding to the C3, C4, C5 feature layers respectively, represents the resulting eigenvector, then It is obtained by the adaptive spatial fusion feature method of multi-level features, which is a linear combination of feature vectors and Multiply by the weight parameters and Then sum up to get, where and

[0086] In the embodiment of the present disclosure, spatial weight allocation is to allocate spatial weights to the sampling features obtained by downsampling. The purpose of this step is to give different weights according to the importance of the features. and The spatial weights of the features of the three levels at level l are obtained by performing 1×1 convolution on the three adjusted feature layers and performing a Softmax operation.

[0087] In the disclosed embodiment, the purpose of performing adaptive feature fusion is to effectively fuse sampling features of different scales to generate a feature map with stronger representation capability. Adaptive feature fusion generally includes the following steps:

[0088] Feature splicing: Sampling features of different scales are spliced ​​to form a feature set containing information of multiple scales.

[0089] Attention mechanism: The attention mechanism is introduced to dynamically adjust the importance of different features. The attention mechanism can automatically select the features that are most helpful for the task based on the contextual information of the input data.

[0090] Fusion operation: The features adjusted by the attention mechanism are fused through weighted summation, maximum pooling or other fusion strategies to obtain the final fused feature map.

[0091] in, The spatial weights between feature layers enhance the importance of key layers and mitigate the impact of conflicting information from different targets.

[0092] It can be explained that compared with YOLOv7, the Neck part of YOLOv8n removes the convolution operation after upsampling, maintaining the lightweight structure of the network. In order to further improve the accuracy of the model in crack identification, reduce the degradation of deep feature semantic information, reduce the number of parameters, and improve the degree of fusion of semantic information.

[0093] AFPN is a network that gradually integrates low-level, high-level and top-level features in the bottom-up feature extraction process of the Backbone network. Figure 5 As shown in the figure, specifically, AFPN initially fuses the Low-Level features, then fuses the High-Level features, and finally fuses the top-level features, which are the most abstract features. The semantic gap between non-adjacent level features is greater than the semantic gap between adjacent level features, especially the bottom and top features. This directly leads to the poor fusion effect of non-adjacent level features. Therefore, it is unreasonable to directly use Backbone's C3, C4, and C5 for feature fusion. Since the architecture of AFPN is progressive, this will make the semantic information of different Level features closer in the progressive fusion process, thereby alleviating this problem.

[0094] In a possible implementation, the loss function L of the crack detection algorithm network model is ECIoU as follows:

[0095]

[0096]

[0097]

[0098] Wherein, IoU is the intersection over union ratio of the crack bounding box and the real bounding box in the image to be detected, ρ is the Euclidean distance between the center points of the crack bounding box and the real bounding box, c is the diagonal length of the minimum circumscribed rectangle surrounding the predicted rectangular box and the real bounding box, b is the coordinate of the crack bounding box, w is the width of the crack rectangular box, h is the height of the crack rectangular box, and b is the coordinate of the crack bounding box. gt is the coordinate of the true bounding box, w gt is the width of the true bounding box, h gt is the height of the ground-truth bounding box.

[0099] It can be explained that the regression loss calculation of YOLOv8 is divided into two parts: CIou_Loss+Distribution FocalLoss. CIou_Loss is used to calculate the IoU between the predicted box and the target box. CIou is currently the best performing boundary regression loss function. It measures the overlapping area between the target and the real box through IoU, Euclidean distance, and the corresponding aspect ratio and angle. However, the last term of the CIoU loss function uses the inverse tangent function to form a penalty term for the aspect ratio of the rectangle. There are the following two problems, which affect the convergence speed and robustness of CIoU.

[0100] 1. The robustness is weak, it is sensitive to outliers, and is affected by outliers, resulting in large fluctuations in the loss function value, affecting the performance of the loss function.

[0101] 2. The value range of the inverse tangent function is (0, π / 2), which cannot directly meet the normalization requirements of the loss function. In order to achieve the numerical normalization of the penalty term, a new coefficient is introduced, which increases the computational complexity.

[0102] In order to solve this problem, a more efficient and direct regression loss function ECIoU based on CIoU is introduced. The ECIoU loss function further considers the difference in the aspect ratio distribution of the two rectangular bounding boxes, effectively reducing the total degree of freedom of the loss and greatly improving the speed and efficiency of model convergence. The last term in the ECIoU loss function takes the rectangular aspect ratio as the domain of the Sigmoid (y = 1 / (1 + exp(-x))) function, takes the difference in the aspect ratio between the predicted box and the true box as the value range of the function, and uses the function idea to optimize the penalty term of the loss function. Compared with the penalty term of the CIoU loss function, it is more robust and smoother, and its function output value is (0, 0.25), which can directly meet the normalization requirements and simplify the complexity of the original loss function, so that the regression loss function ECIoU can obtain faster convergence speed, better positioning results and model performance.

[0103] In a possible implementation, the double convolution module uses the output of the single convolution submodule as input to perform gradient split convolution to obtain a gradient split convolution result, including:

[0104] See also Figure 6 As shown, the double convolution module takes the output of the single convolution submodule as input and performs convolution and then divides it into blocks;

[0105] In the disclosed embodiment, the double convolution module receives the output of the single convolution submodule as input, which is usually a feature map that contains some feature representation of the input image or data.

[0106] Next, the input feature map is convolved. The purpose of this step is to extract higher-level features. The convolution operation can use convolution kernels of any size, depending on the network design and task requirements.

[0107] After convolution, the obtained feature map is divided into blocks. The block operation usually divides the feature map into multiple small blocks according to certain rules (such as the number of channels, spatial position, etc.).

[0108] The first bottleneck module in the two convolution modules performs two convolutions on the output after the blocks, and then performs feature addition on the output after the blocks;

[0109] In the disclosed embodiment, the block-wise output is sent to the first bottleneck module. The bottleneck module is a special convolutional network structure, which is usually used to reduce the amount of calculation and prevent overfitting.

[0110] In the first bottleneck module, the block output is first convolved twice. These two convolutions usually use smaller convolution kernels and fewer channels to reduce computational complexity.

[0111] Then, the result of the second convolution is added to the original output after the block. The purpose of this step is to fuse the original features with the features after convolution to retain more information.

[0112] The second bottleneck module in the double convolution module performs two convolutions on the output of the first bottleneck module and then performs feature addition with the output of the first bottleneck module, wherein the number of channels of the second bottleneck module is half of the number of channels of the first bottleneck module;

[0113] In the disclosed embodiment, the output of the first bottleneck module is fed into the second bottleneck module. Unlike the first bottleneck module, the number of channels of the second bottleneck module is usually half of that of the first bottleneck module. This design helps to further reduce the amount of calculation and the number of parameters.

[0114] In the second bottleneck module, two convolution operations are also performed on the input. The convolution kernel size and number of channels of these two convolutions may be adjusted according to the network design.

[0115] Then, the result of the second convolution is added to the output of the first bottleneck module. The purpose of this step is to further fuse features at different levels to enhance the representation ability of features.

[0116] The number of channels of each BottleNeck input tensor in C2f is only 0.5 times that of the previous level, so the amount of calculation is significantly reduced. On the other hand, the increase in gradient flow can also significantly improve the convergence speed and effect.

[0117] The splicing module in the two convolution modules splices the output of the second bottleneck module with the output after blocking, and performs convolution after the splicing is completed to obtain the gradient split convolution result.

[0118] In the disclosed embodiment, the output of the second bottleneck module is sent to the splicing module, and the task of the splicing module is to splice the output of the second bottleneck module with the original output after the blocks.

[0119] The concatenation operation is usually performed on the channel dimension of the feature map, that is, the channels of two feature maps are merged into a larger feature map. In this way, the concatenated feature map contains feature information from different levels and modules.

[0120] After the splicing is completed, the convolution operation is performed on the spliced ​​feature map. The purpose of this step is to further fuse and extract the spliced ​​features to obtain the final gradient split convolution result.

[0121] In this way, the output of a single convolution submodule can be effectively convolved and feature fused multiple times to generate a feature map with stronger representation capabilities.

[0122] In one possible implementation, the sample data set for training the crack detection algorithm network model is constructed in the following manner:

[0123] Obtain crack images of different gap categories of different exterior wall types to construct an initial data set;

[0124] In the disclosed embodiment, crack images of different types of exterior walls are collected. The images should cover all possible types of cracks, such as horizontal cracks, longitudinal cracks, cross cracks, etc., as well as cracks on different wall materials (such as brick walls, concrete walls, paint walls, etc.).

[0125] To ensure the diversity of the dataset, crack images from different geographical locations, different weather conditions, and different time periods can be collected. This helps the model learn more features during training and improves the generalization ability of the model.

[0126] Performing enhancement operations on the crack images in the initial data set by random brightness transformation, rotation, vertical flipping, horizontal flipping, and adding at least one of black and white dots as noise to obtain an enhanced data set;

[0127] Random brightness transformation: Randomly adjust the brightness of the crack images in the initial dataset to simulate the crack performance under different lighting conditions. This helps the model to accurately detect cracks in different lighting environments.

[0128] Rotation: Rotate the image at random angles to simulate crack images from different perspectives. This helps the model learn the directional characteristics of the cracks and improve detection accuracy.

[0129] Vertical flip and horizontal flip: Flip the image vertically or horizontally to increase the diversity of the dataset. This operation helps the model learn the spatial characteristics of the cracks and is not affected by the image orientation.

[0130] Add black and white noise: Randomly add black and white dots as noise in the image to simulate the interference in the image acquisition process. This helps the model learn more robust features and improves the detection ability in noisy environments.

[0131] The crack images in the enhanced data set are annotated to obtain a target data set, wherein the annotations satisfy the following conditions: all cracks are annotated as transverse cracks, connected cracks are annotated as one object, multiple unconnected cracks included in any crack image are annotated separately, and after any crack is annotated, other cracks covered within the selected range are no longer annotated separately;

[0132] Among them, see Figure 7 As shown in the figure, the images are manually labeled using the LabelImg annotation tool. In this paper, all cracks are labeled as a category called HorizontalCrack. Connected cracks are labeled as one object. If there are multiple cracks in the image that are not connected together, they are labeled separately. If there are long cracks, small cracks are also selected within the range and no longer labeled.

[0133] The crack images in the target data set are divided into a training set, a validation set and a test set in the sample data set according to a preset ratio to complete the construction of the sample data set.

[0134] In order to meet the task of detecting wall cracks in complex environments, this paper uses a variety of data enhancement strategies to enhance the training data through random brightness transformation, rotation, vertical flipping, horizontal flipping, adding black and white dots (strong and weak) as noise, etc., and finally obtains a data set of 5200 pictures, each with a resolution of 640*640. According to the ratio of 10:1:1, the data set is divided into three categories: training set, validation set and test set.

[0135] In this way, a high-quality and diverse crack detection algorithm network model sample dataset is constructed. This dataset not only contains rich crack images and annotation information, but also improves the generalization ability and robustness of the model through data enhancement operations.

[0136] In one possible implementation, see Figure 8 As shown, the sample labels of the training sample crack images in the training set are dynamically assigned in the following manner:

[0137] Inputting the training sample crack image in the training set into the initial network model to obtain a predicted boundary box of the crack in the training sample crack image by the initial network model;

[0138] In the embodiment of the present disclosure, inputting training samples: inputting the training sample crack images in the training set into the initial network model. The initial network model can be any deep learning model for target detection, such as YOLO, SSD, Faster R-CNN, etc.

[0139] Predicted bounding boxes: The initial network model processes the input training sample crack images and outputs predicted bounding boxes. These predicted bounding boxes represent the model's prediction of the crack locations in the image.

[0140] Determining a predicted intersection-and-union ratio of the crack according to a predicted bounding box of the crack in the training sample crack image, wherein the predicted intersection-and-union ratio is used to represent an intersection-and-union ratio between the predicted bounding box and a true bounding box;

[0141] In the disclosed embodiment, the intersection over union (IoU) is calculated: for each training sample crack image, the intersection over union (IoU) between its predicted bounding box and the true bounding box is calculated. IoU is a commonly used evaluation index used to measure the degree of overlap between the predicted bounding box and the true bounding box. The calculated predicted IoU is recorded.

[0142] Determine, according to a standard bounding box of a preset anchor point crack in the training sample crack image, an anchor point intersection-and-union ratio of a preset anchor point crack in the training sample crack image, wherein the anchor point intersection-and-union ratio is used to represent an intersection-and-union ratio between a predefined anchor box and the real bounding box;

[0143] In the disclosed embodiment, anchor cracks are preset: in the training sample crack image, a series of standard bounding boxes of anchor cracks are preset. These anchor cracks can be representative cracks selected based on experience or prior knowledge. Anchor IoU is calculated: for each preset anchor crack, the IoU between its standard bounding box and the true bounding box is calculated, that is, the anchor intersection-over-union ratio.

[0144] Determine a joint intersection-and-union ratio according to the anchor point intersection-and-union ratio of the preset anchor point cracks in the training sample crack image and the predicted intersection-and-union ratio of the cracks in the training sample crack image;

[0145] In the disclosed embodiment, the predicted IoU and the anchor point IoU are combined to calculate the joint intersection-over-union ratio. This can be achieved by weighted averaging the predicted IoU and the anchor point IoU or other suitable combinations.

[0146] Determine a joint mean value according to the predicted intersection-and-union ratio mean of the training sample crack images in the training set and the anchor point intersection-and-union ratio mean of the preset anchor point cracks;

[0147] Determine a joint standard deviation according to the predicted intersection-and-union ratio standard deviation of the training sample crack image in the training set and the anchor point intersection-and-union ratio standard deviation of the preset anchor point crack;

[0148] In the disclosed embodiment, the mean and standard deviation are calculated: the predicted IoU mean and standard deviation of the crack images of all training samples in the training set, and the anchor point IoU mean and standard deviation of the preset anchor point cracks are calculated respectively.

[0149] Joint mean and joint standard deviation: The predicted IoU mean and the anchor point IoU mean are added to get the joint mean, and the predicted IoU standard deviation and the anchor point IoU standard deviation are added to get the joint standard deviation.

[0150] The sum of the joint mean and the joint standard deviation is used as an intersection-over-union ratio threshold;

[0151] When the joint IoU is greater than or equal to the IoU threshold, the sample label of the training sample crack image is determined to be a positive sample label; otherwise, the sample label of the training sample crack image is determined to be a negative sample label.

[0152] It can be explained that the samples are dynamically allocated. It considers anchor allocation and weighting at the same time, and uses the weighted product of classification quality and regression quality to describe the prediction quality of the anchor, as shown in the following formula. Through training, t can guide the network to dynamically focus on high-quality positive samples.

[0153] t=s α ×u β

[0154] Use the above formula to calculate the Anchor-level alignment for each GT box instance: s and u are the classification score and IoU value respectively, and α and β are weight hyperparameters. The strategy considers the preset anchor and ignores the predicted bbox generated during the training process, resulting in some high-quality prediction results being classified as negative samples, which has a greater impact on performance. However, directly adding the prediction results to the adaptive strategy may affect the training because the prediction results in the early stage of training are not credible enough. To this end, the DynamicATSS dynamic label assignment method combines the preset anchor and predicted bbox to solve this problem. The overall network structure is as follows: Figure 7 As shown, the preset anchor point and the prediction result constitute the combined IoU (intersection over union, a concept used in target detection. It calculates the overlap rate of the "predicted bounding box" and the "real bounding box", that is, the ratio of their intersection and union) for label assignment. In the early stage of training, due to inaccurate prediction results, the anchor is used as the main criterion for label definition. As the training progresses, the prediction results gradually control the combined IoU and guide the label assignment in the training stage. See the specific operation method of DynamicATSS shown in the following formula.

[0155] CIoUs=PIoUs+AIoUs

[0156] mean(CIoUs)=mean(PIoUs)+mean(AIoUs)

[0157] std(CIoUs)=std(PIoUs)+std(AIoUs)

[0158] thresnold(CIoUs)=mean(CIoUs)+std(CIoUs)

[0159] PIoU is the predicted IoU between the predicted box and the true box, and AIoU represents the anchors IoU between the predefined anchor box and the true box. CIoU represents the combined IoU, which is the sum of the predicted IoU and the anchors IoU. When calculating the mean and standard deviation of CIoU, PIoU and AIoU are calculated separately and added together. Finally, the threshold is calculated by combining the mean and standard deviation of CIoU to define positive and negative samples. Since both PIoU and AIoU are IoUs against the true box, they can be combined together by simply summing them without designing any complicated formulas or reducing the number of positive samples during training.

[0160] In a possible implementation, the type of the exterior wall includes at least one of a cement wall, a red brick wall, a gray brick wall, and a painted wall;

[0161] See also Fig. 9 The cracks shown are of multiple types, and the crack types include at least one of a first through-type crack whose crack size exceeds a preset threshold, radial cracks, non-through-type cracks, a second through-type crack whose crack size does not exceed a preset threshold, local cracks, local detachment cracks, and hidden cracks.

[0162] The crack detection algorithm network model provided by the present invention is evaluated, and the precision rate (P), recall rate (R), mean average precision (mAP@0.5), FPS (frames per second), and the number of parameters (Params) and the amount of calculation (GFLOPs) of the model are selected as evaluation indicators; the specific calculation formulas of the precision rate, recall rate, and average precision are respectively expressed by the following formulas.

[0163] P=P T / (P T +P F )

[0164] R=R T / (R T +N F )

[0165]

[0166]

[0167] Where: PT is the number of samples correctly predicted as cracks, PF is the number of samples incorrectly predicted as cracks, NF is the number of samples not predicted as cracks; PA is the mean precision; PmA is mAP, mAP@0.5 is the mean precision with the IoU threshold set to 0.5; N is the total number of categories. Because in this article, there is only one category of objects to be identified, so PA is equal to PmA.

[0168] The experimental environment is as follows: the training and testing of the crack detection model are run in the same environment. The operating system used in the experiment is Ubuntu 22.04, the CPU model is Intel(R) Xeon(R) Platinum8352V CPU@2.10GHzCPU@2.10GHz, the GPU model is Nvidia Geforce RTX 4090, the video memory is 24GB, the deep learning framework used is Pytorch 2.3.0, the Python version is 3.12, and the GPU acceleration library is CUDA 12.1. The training parameters are set as follows: the image size is set to 640×640, the batch size is 96, the epochs is 300 rounds, and the learning rate is 0.0001. All processes are enabled with half precision.

[0169] In order to verify the effectiveness of the improved parts of the model, an ablation test was designed. The test plan is as follows: add the C2f_MSB module to YOLOv8n, named YOLOv8n-C2f_MSB, change the neck network of YOLOv8n to AFPN, named YOLOv8n-AFPN, change the loss function of YOLOv8n to ECIoU, named YOLOv8n-ECIoU, change the positive and negative sample allocation strategy of YOLOv8n to DynamicATSS, named YOLOv8n-DynamicATSS, and compare it with the YOLOv8n-CMAED model with all improvements added at the same time.

[0170] Detailed parameters are shown in Table 1.

[0171]

[0172] Table 1

[0173] As can be seen from the table, after replacing the C2f module in the Backbone of YOLOv8n with C2f_MSB, the mAP increased by 0.9%, and the amount of calculation decreased by 6%, but the frame rate decreased significantly; after improving the neck network of YOLOv8n to AFPN, the Precision increased by 2%, the Recall increased by 1.8%, the mAP increased by 0.6%, and the amount of calculation decreased by 18.3%, and the frame rate increased significantly by 50.2%; after improving the loss function of YOLOv8n to ECIoU, the performance of each item did not degrade, but the frame rate increased significantly by 30%; after improving the positive and negative sample allocation strategy of YOLOv8n to DynamicATSS, the performance of each item did not degrade, but the frame rate also increased significantly by 30%; YOLOv8n-CMAED, which combines all the above improvements, has balanced the remaining performance while achieving the ultimate reduction of the desired computing power requirements. The computing power decreased by 28% to 5.9GFLOPS, the number of parameters decreased by 33% to 2.0M, the frame rate increased by 30% to 400FPS, the Precision increased to 0.927, and the mAP increased to 0.931.

[0174] The YOLOv8n and YOLOv8n-CMAED models were used to verify the target detection on the test set. Fig.10 As shown, it can be seen that the YOLOv8n-CMAED model has better detection performance. In the first and second groups of comparison pictures, the improved model did not have repeated recognition. In the third group of comparison pictures, it can be found that the improved model has a more accurate target recognition ability. The recognition areas are all real crack areas, and there are large non-crack areas within the range of YOLOv8n recognition. In the fourth control group, the above two situations occurred at the same time. The improved model did not have repeated recognition, and the recognition range was more accurate.

[0175] In order to further verify the effectiveness and model performance of the improved YOLOv8n-CMAED algorithm for crack detection, the mainstream target detection models YOLOv5nu, YOLOv5su, YOLOv6n, YOLOv6s, YOLOv8n, YOLOv9t and YOLOv10n are selected to compare the detection effects of the improved YOLOv8n-CMAED algorithm in this paper on the crack dataset. The six indicators of Precision, Recall, mAP, Params, GFLOPs and FPS are used for evaluation. The experimental results are shown in Table 2.

[0176]

[0177] Table 2

[0178] From the comparison in Table 2, we can see that compared with other algorithms, YOLOv8n-CMAED has reached the highest in mAP, FLOPs, and Params, which are 93.1%, 5.9GFLOPs, and 2.0M respectively. While achieving the above advantages, the FPS reaches 400, which is only lower than YOLOv6n and YOLOv10n, but mAP, FLOPs, and Params have significant advantages. The above data fully demonstrate that in terms of detection accuracy, the algorithm in this paper is generally superior to other algorithms. In terms of parameter size, computational complexity, and detection speed, compared with other algorithms, the algorithm in this paper is the best. In addition, compared with the YOLOv8n model, the FPS of YOLOv8n-CMAED is increased to 400, the processing time of a single frame image is shorter, the model parameters are reduced by 1MB, which is only 66.7% of the original, and the computational complexity is reduced by 2.3GFLOPs, which is only 72.0% of the original. These improvements show that the improvements in this paper have achieved the preset goals of computing power requirements, high FPS, small number of parameters, and high mAP, which is conducive to the use of the model on drones.

[0179] Through various improvements, in the Backbone network, a C2f_MSB multi-level feature fusion extraction module is proposed to replace the existing C2f module, which enhances the network feature extraction capability and reduces the number of parameters and calculations; in the neck network part, the AFPN structure is used to replace the PAN structure, which greatly reduces the model parameters and the amount of calculation; in the detection head part, DynamicATSS is used for dynamic label allocation, and the ECIOU loss function is used to improve the FPS. The algorithm was fully tested on the external wall crack dataset, and the results show the effectiveness and advancement of the algorithm in this paper. On the basis of greatly reducing the number of parameters and calculations, the highest mAP was achieved, which is 0.8% higher than the mAP of YOLOv8n. Even compared with the latest YOLOv9t and YOLOv10n, it still has obvious advantages.

[0180] In summary, by optimizing the model structure, optimizing the feature extraction module, optimizing the loss function, and adding a dynamic label allocation mechanism based on YOLOv8n, the crack detection algorithm network model is finally obtained. In order to further enhance the feature extraction capability of the original Backbone network of YOLOv8, the feature extraction capability of each layer of Backbone is enhanced by improving the C2f module. In view of the problem that the original feature fusion network PAN of YOLOv8 has insufficient feature information fusion capability, the AFPN feature fusion method is introduced to enhance the network's fusion capability of multi-scale features and adapt to the detection needs of targets of different scales. Finally, the DynamicATSS dynamic label allocation method is adopted, and the ECIoU loss function replaces the original loss function to better identify crack targets and improve the performance of the algorithm.

[0181] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in any one of the aforementioned embodiments are implemented.

[0182] The present disclosure also provides an electronic device, including:

[0183] a memory having a computer program stored thereon;

[0184] A processor is used to execute the computer program in the memory to implement the steps of the method in any one of the aforementioned embodiments.

[0185] Fig.11 The crack detection device 100 applied to a drone shown includes: a processor 1001 and a memory 1003. The processor 1001 and the memory 1003 are connected, such as through a bus 1002. Optionally, the crack detection device 100 applied to a drone may also include a communication component 1004, which may be used for data interaction between the device 100 and other devices, such as data transmission and / or data reception. It should be noted that in actual scheduling, the communication component 1004 is not limited to one, and the structure of the crack detection device 100 applied to a drone does not constitute a limitation on the embodiments of the present application.

[0186] Processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0187] The bus 1002 may include a path to transmit information between the above components. The bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0188] The memory 1003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium, other magnetic storage devices, or any other medium that can be used to carry or store program code and can be read by a computer, without limitation herein.

[0189] The memory 1003 is used to store program codes for executing the embodiments of the present disclosure, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the program codes stored in the memory 1003 to implement the steps shown in the above crack detection method embodiment applied to a drone.

[0190] The embodiment of the present disclosure also provides a computer-readable storage medium having program code stored thereon. When the program code is executed by a processor, the steps and corresponding contents of the aforementioned crack detection method embodiment applied to a drone can be implemented.

[0191] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments; within the technical concept of the present disclosure, various changes, modifications, substitutions and variations may be made to these embodiments, and these changes, modifications, substitutions and variations all fall within the protection scope of the present disclosure.

[0192] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction, and they should also be regarded as the contents disclosed in this disclosure. In order to avoid unnecessary repetition, this disclosure will not further describe various possible combinations. The technical scope of this application is not limited to the contents in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A crack detection method applied to an unmanned aerial vehicle, characterized in that: The method comprises: In response to the crack detection operation of the drone starting, an image to be detected is acquired through a camera configured for the drone; Input the image to be detected acquired by the camera configured by the unmanned aerial vehicle into the crack detection algorithm network model to obtain the crack detection result output by the crack detection algorithm network model, wherein the backbone network module of the crack detection algorithm network model includes a plurality of linearly connected single convolution submodules, two-time and multi-scale convolution submodules and spatial pyramid pooling fast modules, wherein the single convolution submodules and the two-time and multi-scale convolution submodules are alternately configured, and a first number of the two-time and multi-scale convolution submodules are respectively connected to a second number of feature fusion submodules of the neck network module of the crack detection algorithm network model, and the first number is less than the second number; Among them, the double and multi-scale convolution submodule includes a double convolution module and a multi-scale convolution module, the double convolution module uses the output of the single convolution submodule as input to perform gradient shunt convolution to obtain a gradient shunt convolution result, the multi-scale convolution module uses the gradient shunt convolution result output by the double convolution module as input to perform multi-scale convolution to obtain double convolution and multi-scale convolution results corresponding to the double and multi-scale convolution submodules, and the spatial pyramid pooling fast module aggregates features of different receptive fields through multiple sequential maximum pooling operations; The feature fusion submodule in the neck network module is constructed to not include the convolution operation after upsampling, and only includes bottom-up and top-down network structures.

2. The crack detection method applied to an unmanned aerial vehicle according to claim 1, characterized in that: The multi-scale convolution module uses the gradient split convolution result output by the two convolution modules as input to perform multi-scale convolution, and obtains the two convolution and multi-scale convolution results corresponding to the two and multi-scale convolution submodules, including: In the multi-scale convolution module, according to the number of convolution branches in the multi-scale convolution module, the gradient split convolution result is divided into a plurality of feature subsets whose number of sets is the same as the number of convolution branches; The data in each feature subset is assigned one by one to one of the convolution branches for convolution processing, wherein, within the convolution branch, the input data is convolved for the first time through a convolution kernel of a first size, and the result of the first convolution is convolved through a convolution kernel of a second size for depth convolution, and then the result obtained by the depth convolution is convolved through a convolution kernel of a third size for the second time to obtain the convolution processing result of the convolution branch, and between the convolution branches, the convolution processing result corresponding to the previous convolution branch and the data in the feature subset assigned to the next convolution branch together constitute the input data of the next convolution branch; The processing results of each convolution branch in the multi-scale convolution module are spliced ​​to obtain the double convolution and multi-scale convolution results corresponding to the double and multi-scale convolution sub-modules.

3. The crack detection method applied to an unmanned aerial vehicle according to claim 1, characterized in that: The feature fusion submodule in the neck network module performs feature fusion in the following manner: Selecting the feature graphs {C3, C4, C5} output by the backbone network module in the last three layers as input features of the feature pyramid network formed by the feature fusion submodule in the neck network module; Perform a convolution with a convolution kernel of the fourth size and upsample the input features using bilinear interpolation to align the dimensions of each level; For the sampling features obtained by upsampling, downsampling is performed a first preset number of times using a convolution kernel of a fifth size with a first step length and downsampling is performed a second preset number of times using a convolution kernel of a sixth size with a second step length to adapt to different downsampling rates; For the sampling features obtained by downsampling, spatial weight allocation is performed and then adaptive feature fusion is performed, wherein, if Represents the feature vector at position (i, j) from the nth layer to the lth layer, l = 1, 2, 3, corresponding to the C3, C4, C5 feature layers respectively, represents the resulting eigenvector, then It is obtained by the adaptive spatial fusion feature method of multi-level features, which is a linear combination of feature vectors and Multiply by the weight parameters and Then sum up to get, where and 4. The crack detection method applied to an unmanned aerial vehicle according to claim 1, characterized in that: The loss function L of the crack detection algorithm network model is ECIoU as follows: Wherein, IoU is the intersection over union ratio of the crack bounding box and the real bounding box in the image to be detected, ρ is the Euclidean distance between the center points of the crack bounding box and the real bounding box, c is the diagonal length of the minimum circumscribed rectangle surrounding the predicted rectangular box and the real bounding box, b is the coordinate of the crack bounding box, w is the width of the crack rectangular box, h is the height of the crack rectangular box, and b is the coordinate of the crack bounding box. gt is the coordinate of the real bounding box, w gt is the width of the ground-truth bounding box, h gt is the height of the ground-truth bounding box.

5. The crack detection method applied to an unmanned aerial vehicle according to claim 1, characterized in that: The double convolution module uses the output of the single convolution submodule as input to perform gradient split convolution to obtain a gradient split convolution result, including: The double convolution module takes the output of the single convolution submodule as input, performs convolution and then divides it into blocks; The first bottleneck module in the two convolution modules performs two convolutions on the output after the blocks, and then performs feature addition on the output after the blocks; The second bottleneck module in the double convolution module performs two convolutions on the output of the first bottleneck module and then performs feature addition with the output of the first bottleneck module, wherein the number of channels of the second bottleneck module is half of the number of channels of the first bottleneck module; The splicing module in the two convolution modules splices the output of the second bottleneck module with the output after blocking, and performs convolution after the splicing is completed to obtain the gradient split convolution result.

6. The crack detection method applied to an unmanned aerial vehicle according to any one of claims 1 to 5, characterized in that: The sample data set for training the crack detection algorithm network model is constructed in the following way: Obtain crack images of different gap categories of different exterior wall types to construct an initial data set; Performing enhancement operations on the crack images in the initial data set by random brightness transformation, rotation, vertical flipping, horizontal flipping, and adding at least one of black and white dots as noise to obtain an enhanced data set; The crack images in the enhanced data set are annotated to obtain a target data set, wherein the annotations satisfy the following conditions: all cracks are annotated as transverse cracks, connected cracks are annotated as one object, multiple unconnected cracks included in any crack image are annotated separately, and after any crack is annotated, other cracks covered within the selected range are no longer annotated separately; The crack images in the target data set are divided into a training set, a validation set and a test set in the sample data set according to a preset ratio to complete the construction of the sample data set.

7. The crack detection method applied to an unmanned aerial vehicle according to claim 6, characterized in that: The sample labels of the training sample crack images in the training set are dynamically assigned in the following manner: Inputting the training sample crack image in the training set into the initial network model to obtain a predicted boundary box of the crack in the training sample crack image by the initial network model; Determining a predicted intersection-and-union ratio of the crack according to a predicted bounding box of the crack in the training sample crack image, wherein the predicted intersection-and-union ratio is used to represent an intersection-and-union ratio between the predicted bounding box and a true bounding box; Determine, according to a standard bounding box of a preset anchor point crack in the training sample crack image, an anchor point intersection-and-union ratio of a preset anchor point crack in the training sample crack image, wherein the anchor point intersection-and-union ratio is used to represent an intersection-and-union ratio between a predefined anchor box and the real bounding box; Determine a joint intersection-and-union ratio according to the anchor point intersection-and-union ratio of the preset anchor point cracks in the training sample crack image and the predicted intersection-and-union ratio of the cracks in the training sample crack image; Determine a joint mean value according to the predicted intersection-and-union ratio mean of the training sample crack images in the training set and the anchor point intersection-and-union ratio mean of the preset anchor point cracks; Determine a joint standard deviation according to the predicted intersection-and-union ratio standard deviation of the training sample crack image in the training set and the anchor point intersection-and-union ratio standard deviation of the preset anchor point crack; The sum of the joint mean and the joint standard deviation is used as an intersection-over-union ratio threshold; When the joint IoU is greater than or equal to the IoU threshold, the sample label of the training sample crack image is determined to be a positive sample label; otherwise, the sample label of the training sample crack image is determined to be a negative sample label.

8. The crack detection method applied to an unmanned aerial vehicle according to claim 7, characterized in that: The type of the exterior wall includes at least one of a cement wall, a red brick wall, a gray brick wall, and a painted wall; The crack categories include at least one of a first through-type crack whose crack size exceeds a preset threshold, radial cracks, non-through-type cracks, a second through-type crack whose crack size does not exceed a preset threshold, local cracks, local detachment cracks, and hidden cracks.

9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

Images