A method and system for detecting cracks of a UAV based on multi-scale self-adaptation and edge enhancement YOLO

Abstract

This invention discloses a method and system for crack detection in unmanned aerial vehicles (UAVs) based on multi-scale adaptive and edge-enhanced YOLO, belonging to the field of machine vision target detection technology. The method includes: introducing a local enhancement guidance module at the end of the backbone network to collaboratively enhance global context and local detail features using a parallel dual-stream structure; replacing the original C2f module with a multi-scale adaptive hole module in the neck network to fuse receptive fields of different scales to capture multi-scale crack features, and employing a dynamic upsampling module to improve feature upsampling accuracy; and introducing an edge-enhanced attention mechanism in front of the detection head to explicitly strengthen crack boundary features. This invention maintains extremely low parameter count and computational complexity while accurately capturing minute cracks, effectively suppressing background noise, and significantly improving detection accuracy and recall, providing an efficient and accurate solution for UAV crack detection.

Description

Technical Field

[0001] This invention relates to the field of machine vision target detection technology, and in particular to a method and system for detecting cracks in unmanned aerial vehicles based on multi-scale adaptive and edge-enhancing YOLO. Background Technology

[0002] Unmanned aerial vehicles (UAVs) have become an effective tool for infrastructure inspection, enabling rapid and flexible data collection in large or hard-to-access areas. Cracks are among the most common and critical defects in infrastructure, and timely detection is crucial for ensuring structural safety. However, in real-world UAV inspection scenarios, existing mainstream deep learning object detection models face significant limitations. First, cracks typically exhibit subtle, low-contrast morphology, easily confused with background textures (such as stains, seams, and shadows). Second, due to varying UAV flight altitudes, target cracks show significant scale variations. Furthermore, existing high-precision models often involve a large number of parameters and computational overhead, making direct deployment on resource-constrained UAV edge computing devices difficult. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for detecting cracks in unmanned aerial vehicles (UAVs) based on multi-scale adaptive and edge-enhanced YOLO, aiming to solve or improve at least one of the above-mentioned technical problems.

[0004] To achieve the above objectives, the present invention provides the following solution: A UAV crack detection method based on multi-scale adaptive and edge-enhanced YOLO includes: By using an image acquisition device mounted on a drone, images of the target to be tested are acquired, and a crack image dataset is constructed. An improved YOLO-MSEE model is constructed based on the YOLOv8 model. Specifically, a local enhancement guidance module is introduced at the end of the backbone network; the original C2f module in the neck network is replaced with a multi-scale adaptive dilatation module, and the original upsampling module in the upsampling path is replaced with a dynamic upsampling module; an edge enhancement attention mechanism is introduced before the detection head; and the feature map output from the end of the backbone network is directly input into the multi-scale adaptive dilatation module in the neck network's upsampling path. The YOLO-MSEE model was trained using a crack image dataset to obtain the final YOLO-MSEE model. Using the final YOLO-MSEE model, input the image to be tested and output the crack detection results.

[0005] Furthermore, the locally enhanced boot module includes: The input features are processed through channel concatenation, convolution, and normalization to unify the feature distribution and generate basic feature representations. Based on the basic feature representation, a two-stream structure is adopted to extract global and local features separately, and then they interact to generate intermediate features; The intermediate features are projected into a linear layer, then added element-wise with the convolutional local features, and finally connected via a multilayer perceptron to generate the final enhanced features.

[0006] Furthermore, based on the fundamental feature representation, a two-stream structure is adopted to extract global and local features separately, and these features are interacted to generate intermediate features, including: The global context branch takes the basic feature representation as input to the window-based multi-head attention mechanism submodule and generates it through a linear layer. The spatial correlation between pixels is calculated and a relative position bias is introduced. After Softmax normalization, the global features are output. Local branching involves projecting the basic feature representations into the linear layer to generate local features. The global features and local features are multiplied element-wise, and then residual connections are made with the base feature representations and local features after convolution to generate intermediate features.

[0007] Furthermore, the multi-scale adaptive hole module replaces the original Bottleneck structure with the MADBlock module based on the original C2f module. After the Concat module, batch normalized BN and nonlinear activation function ReLU are introduced, and finally the features are output through an adaptive gated fusion module.

[0008] Furthermore, the MADBlock module includes depthwise separable convolutions and dilated convolutions, followed by batch normalized normalization (BN) and ReLU activation functions, expressed as: In the formula, For the first The output feature map tensors of the MADBlock module in each branch; After equalization, input to the first... Sub-feature maps of each branch; It is a depthwise separable convolution function; For the first The kernel size used in each branch; For the first The void ratio used in each branch; For the first Learnable network parameters of depth-separable convolutional layers in each branch; For batch normalization function; It is a non-linear activation function.

[0009] Furthermore, the adaptive gating fusion module generates channel attention and spatial attention through global average pooling and 7×7 convolution, respectively, and performs adaptive balanced fusion using dynamic gating coefficients G, expressed as: In the formula, The output feature map after adaptive gating fusion; This is a multi-scale fused feature map; To fuse feature maps across multiple scales using global average pooling and fully connected layers Extracted channel attention weight map; To achieve mean and max pooling along the channel dimension and Convolution pairs fuse feature maps at multiple scales Extracted spatial attention weight map; For the feature map through Dynamic gating coefficients learned by convolution and the sigmoid activation function; This represents the complementary gating weights assigned to channel attention; This refers to element-wise multiplication between tensors.

[0010] Furthermore, edge-enhanced attention mechanisms include: Input features Figure 1 This is achieved through depthwise convolution. Pointwise convolution, batch normalization, and ReLU activation function are used to generate edge feature maps; Global average pooling is performed on the edge feature map, dimensionality reduction and expansion are achieved through a fully connected layer, and activation is performed through the Sigmoid function to generate channel weight vectors; the channel weight vectors are then multiplied element-wise with the edge feature map to generate enhanced features; Channel-level average pooling and max pooling are performed on the enhanced features. The resulting two two-dimensional feature maps are then concatenated and processed... The convolutional layer, combined with batch normalization and the sigmoid function, generates a spatial attention weight map; the spatial attention weight map is then multiplied element-wise with the augmented features to generate spatial augmented features. The spatial augmentation features are used to perform residual connections with the input features, and the channel dimensions are adjusted to generate the final output features.

[0011] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention discloses a method and system for crack detection in unmanned aerial vehicles (UAVs) based on multi-scale adaptive and edge-enhanced YOLO. The method, by fusing receptive fields of different scales and explicitly enhancing boundary perception features, can accurately capture minute cracks with varying scales and irregular geometries, effectively suppressing background noise interference. While maintaining extremely low parameter count and computational complexity, it significantly improves detection accuracy and recall. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the YOLO-MSEE model structure in this embodiment; Figure 3 This is a schematic diagram of the local enhancement guidance module structure at the end of the backbone network in this embodiment; Figure 4 This is a comparative diagram of the C2f module and the multi-scale adaptive void module in this embodiment; Figure 5 This is a schematic diagram of the multi-scale adaptive void module structure in this embodiment; Figure 6 This is a schematic diagram of the dynamic upsampling module structure of the neck network in this embodiment; Figure 7 This is a schematic diagram of the edge enhancement attention mechanism structure in this embodiment; Figure 8 This is a schematic diagram of the crack detection system in this embodiment; In the figure, 101 is the surface of the crack being inspected; 102 is the drone equipped with a camera; 103 is the data processing system; and 104 is the display terminal. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] The purpose of this invention is to provide a method and system for detecting cracks in unmanned aerial vehicles (UAVs) based on multi-scale adaptive and edge-enhanced YOLO, aiming to solve or improve at least one of the above-mentioned technical problems.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] like Figure 1 As shown, this invention provides a UAV crack detection method based on multi-scale adaptive and edge-enhanced YOLO, including: Step 1: Use an image acquisition device mounted on a drone to acquire images of the target under test and construct a crack image dataset; The expression for the image dataset is: In the formula, For image datasets; The total number of elements; The image size is , For image channels, Image height, This represents the image width.

[0018] Image annotation was performed using the open-source tool labelImg, with the annotation content being cracks. The annotated information file is in YOLO format.

[0019] like Figure 2 As shown, in step 2, the YOLO-MSEE model is constructed by improving the YOLOv8 model. In this model, a local enhancement guidance module is introduced at the end of the backbone network; the original C2f module in the neck network is replaced with a multi-scale adaptive dilatation module, and the original upsampling module in the upsampling path is replaced with a dynamic upsampling module; an edge enhancement attention mechanism is introduced in front of the detection head; and the feature map output from the end of the backbone network is directly input into the multi-scale adaptive dilatation module in the upsampling path of the neck network.

[0020] like Figure 3 As shown, the local enhancement guidance module achieves collaborative enhancement of global context and local details through a parallel dual-stream structure. The processing flow includes: The input features are processed through channel concatenation, convolution, and normalization to unify the feature distribution and generate a basic feature representation, expressed as: In the formula, Basic feature representation; For splicing operations; This is a convolution operation; This is a normalization operation; Based on the fundamental feature representation, a two-stream structure is adopted to extract global and local features separately, and these features are interacted to generate intermediate features, including: The global context branch takes the basic feature representation as input to the window-based multi-head attention (WATT) submodule and generates it through a linear layer. The spatial correlation between pixels is calculated and a relative position bias is introduced. After Softmax normalization, the global features are output. The local branch projects the basic feature representations into the linear layer to generate local features; The global and local features are multiplied element-wise, and then residually concatenated with the base feature representations and local features obtained after convolution to generate intermediate features, expressed as: In the formula, As an intermediate feature; For global features; Local features; After projecting the intermediate features into a linear layer, they are element-wise added to the convolutional local features, and then residual connections are made through a multilayer perceptron to generate the final enhanced features, expressed as: In the formula, To enhance the feature map; This is a linear projection operation; It is a multilayer perceptron.

[0021] like Figure 4 and Figure 5 As shown, the multi-scale adaptive hole module replaces the original Bottleneck structure with a MADBlock module based on the original C2f module. After the Concat module, batch normalization (BN) and the nonlinear activation function ReLU are introduced. Finally, features are output through an adaptive gated fusion module. The processing flow includes: The input feature map is subjected to channel dimensionality reduction through a 1×1 convolution to generate an initial feature map, expressed as: In the formula, The input feature map tensor; The number of channels in the input feature map; The spatial height of the feature map; The spatial width of the feature map; The kernel size is The convolution function; for Learnable weights and bias parameters in convolution operations; This refers to the number of intermediate channels allocated to each subsequent parallel multi-scale feature extraction branch after dimensionality reduction; This represents the total number of branches for subsequent parallel feature extraction. This is the initial feature map obtained after dimensionality reduction.

[0022] The initial feature map is divided into n sub-feature maps in the channel dimension, and each sub-feature map is fed into the MADBlock module with multiple parallel branches. The MADBlock module includes depthwise separable convolution and dilated convolution, followed by batch normalized normalization (BN) and ReLU activation functions. In this embodiment, the depthwise separable convolution kernel size is K={3,5}, the dilation rate is D={1,2,3}, and a total of n=6 branches are generated. The MADBlock module has the following expression: In the formula, For the first The output feature map tensors of the MADBlock module in each branch; After equalization, input to the first... Sub-feature maps of each branch; It is a depthwise separable convolution function; For the first The kernel size used in each branch; For the first The void ratio used in each branch; For the first Learnable network parameters of depth-separable convolutional layers in each branch; For batch normalization function; To correct the nonlinear activation function of the linear unit.

[0023] After concatenating the multi-scale features output from all branches, normalization and dimensionality reduction are performed to generate a multi-scale fused feature map, expressed as: In the formula, This is the multi-scale fused feature map after dimensionality reduction; This is a tensor splicing function performed along the channel dimension; They represent Feature maps output by each branch; This is an intermediate feature map; for Convolution function; These are the learnable weight parameters for the convolution operation; The target number of channels for the final output; For batch normalization function; To correct the nonlinear activation function of the linear unit.

[0024] like Figure 5 The multi-scale fused feature map is input into the adaptive gated fusion module, where channel attention and spatial attention are generated through global average pooling and 7×7 convolution, respectively. Adaptive balanced fusion is then performed using dynamic gating coefficients G, expressed as follows: In the formula, The output feature map after adaptive gating fusion; This is a multi-scale fused feature map; To fuse feature maps across multiple scales using global average pooling and fully connected layers Extracted channel attention weight map; To achieve mean and max pooling along the channel dimension and Convolution pairs fuse feature maps at multiple scales Extracted spatial attention weight map; For the feature map through Dynamic gating coefficients learned by convolution and the sigmoid activation function; This represents the complementary gating weights assigned to channel attention; This refers to element-wise multiplication between tensors or multiplication operations performed using a broadcast mechanism.

[0025] like Figure 6 As shown, the dynamic upsampling module is used to reduce information loss during feature reconstruction of irregular cracks, including: Let the input feature map be X, the upsampling factor be s, and the number of sampling groups be g; The sampling point generator predicts the original offset and the modulation scalar, and calculates the normalized offset, expressed as: In the formula, This is the normalized offset; This is the original offset; For modulation scalar; For the Sigmoid function; The hyperbolic tangent activation function is used to limit the original offset to the interval [-1, 1] to prevent the sampling point from deviating too far from the effective receptive field. The offset is superimposed onto the initial grid coordinates to generate the final sample set, expressed as: In the formula, This is the final sample set; These are the initial grid coordinates; The input features are sampled using the final sample set to obtain the spatially reconstructed upsampled feature map, expressed as: In the formula, This is an upsampled feature map; For grid-based resampling, bilinear interpolation is used in this embodiment, based on the input coordinate grid. Pixel values ​​are extracted from the original feature map X to generate high-resolution features.

[0026] like Figure 7 As shown, the edge enhancement attention mechanism is used to suppress complex background interference and highlight crack edges. The process includes: Input features Figure 1 This is achieved through depthwise convolution. Pointwise convolution, batch normalization, and ReLU activation function are used to generate edge feature maps, expressed as follows: In the formula, For edge feature maps; To correct the nonlinear activation function of the linear unit; This is a batch normalization function used to stabilize the feature distribution during training. The kernel size is The pointwise convolution function is used to achieve cross-channel feature fusion and information interaction; The number of groups is equal to the number of input channels. Depth convolution functions are used to efficiently capture the local response of crack edges in space; This is the input feature map.

[0027] Global average pooling is applied to the edge feature map, followed by dimensionality reduction and expansion through a fully connected layer, and activation using the Sigmoid function to generate channel weight vectors. The channel weight vectors are then element-wise multiplied with the edge feature map to generate enhanced features, expressed as: In the formula, To enhance features; Channel weight; Channel-level average pooling and max pooling are performed on the enhanced features. The resulting two two-dimensional feature maps are then concatenated and processed... The convolutional layer, combined with batch normalization and the sigmoid function, generates a spatial attention weight map. This spatial attention weight map is then multiplied element-wise with the augmented features to generate the spatial augmented features, expressed as: In the formula, For spatial enhancement features; This is an element-wise multiplication operation; Spatial attention weight map; This is a max pooling operation; This is an average pooling operation; The spatial augmentation features are coupled with the input features using residual connections, and the channel dimensions are adjusted to generate the final output features, expressed as follows: In the formula, For output features.

[0028] The residual structure described above effectively preserves the integrity of the original feature information and prevents the gradient vanishing problem that occurs during deep network training. The final output feature map... The input category classification is combined with the bounding box regression branch.

[0029] Step 3: Maintain consistent configuration of each part of the network, and train the YOLO-MSEE model using a dataset of crack images with real annotations to obtain the final YOLO-MSEE model. In this embodiment, the network training parameters are set as follows: the optimizer uses SGD, the momentum is set to M=0.937, the initial learning rate is set to lr=0.01, the batch size is set to N=16, and the training epoch is set to E=350 rounds.

[0030] Step 4: Using the final YOLO-MSEE model, input the image to be tested and output the crack detection results, including: The system takes an input image as input, performs feature extraction and detection head decoding, and outputs multi-scale target prediction bounding boxes and their confidence scores. The output bounding boxes are sorted in descending order of confidence score, and redundant prediction boxes with an Intersection over Union (IoU) ratio greater than a set threshold are removed using the Non-Maximum Suppression (NMS) algorithm. Finally, the coordinates of the prediction boxes are obtained and plotted on the test image. If a valid detection box is output, a crack is determined to exist in the target area, and the detection results are automatically uploaded to the cloud or ground station to provide data support for infrastructure condition assessment.

[0031] like Figure 8 As shown, in one embodiment, a crack detection system applying the above method is provided, including: a crack surface 101 to be inspected, a drone 102 equipped with a camera, a data processing system 103, and a display terminal 104; The UAV 102 establishes a communication connection with the data processing system 103 and outputs the detected and analyzed images to the display terminal 104; The data processing system 103 is equipped with a YOLO-MSEE model to acquire images of the surface 101 of the crack being inspected and to perform crack detection.

[0032] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0033] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for crack detection in unmanned aerial vehicles based on multi-scale adaptive and edge-enhanced YOLO, characterized in that, include: By using an image acquisition device mounted on a drone, images of the target to be tested are acquired, and a crack image dataset is constructed. Based on the YOLOv8 model, an improved YOLO-MSEE model is constructed; in this model, a local enhancement guidance module is introduced at the end of the backbone network. The original C2f module in the neck network is replaced with a multi-scale adaptive dilatation module, and the original upsampling module in the upsampling path is replaced with a dynamic upsampling module; an edge enhancement attention mechanism is introduced in front of the detection head; and the feature map output from the end of the backbone network is directly input into the multi-scale adaptive dilatation module in the upsampling path of the neck network. The YOLO-MSEE model was trained using a crack image dataset to obtain the final YOLO-MSEE model. Using the final YOLO-MSEE model, input the image to be tested and output the crack detection results.

2. The UAV crack detection method based on multi-scale adaptive and edge-enhanced YOLO according to claim 1, characterized in that, The local enhancement guidance module includes: The input features are processed through channel concatenation, convolution, and normalization to unify the feature distribution and generate basic feature representations. Based on the basic feature representation, a two-stream structure is adopted to extract global and local features separately, and then they interact to generate intermediate features; The intermediate features are projected into a linear layer, then added element-wise with the convolutional local features, and finally connected via a multilayer perceptron to generate the final enhanced features.

3. The UAV crack detection method based on multi-scale adaptive and edge-enhanced YOLO according to claim 2, characterized in that, The method of using a two-stream structure based on basic feature representation to extract global and local features separately, and then interacting to generate intermediate features includes: The global context branch takes the basic feature representation as input to the window-based multi-head attention mechanism submodule and generates it through a linear layer. The spatial correlation between pixels is calculated and a relative position bias is introduced. After Softmax normalization, the global features are output. Local branching involves projecting the basic feature representations into the linear layer to generate local features. The global features and local features are multiplied element-wise, and then residual connections are made with the base feature representations and local features after convolution to generate intermediate features.

4. The UAV crack detection method based on multi-scale adaptive and edge-enhanced YOLO according to claim 1, characterized in that, The multi-scale adaptive hole module replaces the original Bottleneck structure with the MADBlock module based on the original C2f module. After the Concat module, batch normalization (BN) and nonlinear activation function ReLU are introduced, and finally the features are output through an adaptive gated fusion module.

5. The UAV crack detection method based on multi-scale adaptive and edge-enhanced YOLO according to claim 4, characterized in that, The MADBlock module includes depthwise separable convolution and dilated convolution, followed by batch normalization (BN) and ReLU activation functions, expressed as follows: In the formula, For the first The output feature map tensors of the MADBlock module in each branch; After equalization, input to the first... Sub-feature maps of each branch; It is a depthwise separable convolution function; For the first The kernel size used in each branch; For the first The void ratio used in each branch; For the first Learnable network parameters of depth-separable convolutional layers in each branch; For batch normalization function; It is a non-linear activation function.

6. The UAV crack detection method based on multi-scale adaptive and edge-enhanced YOLO according to claim 4, characterized in that, The adaptive gating fusion module generates channel attention and spatial attention through global average pooling and 7×7 convolution, respectively, and performs adaptive balanced fusion using dynamic gating coefficients G, as expressed in the following expression: In the formula, The output feature map after adaptive gating fusion; This is a multi-scale fused feature map; To fuse feature maps across multiple scales using global average pooling and fully connected layers Extracted channel attention weight map; To achieve mean and max pooling along the channel dimension and Convolution pairs fuse feature maps at multiple scales Extracted spatial attention weight map; For the feature map through Dynamic gating coefficients learned by convolution and the sigmoid activation function; This represents the complementary gating weights assigned to channel attention; This refers to element-wise multiplication between tensors.

7. The UAV crack detection method based on multi-scale adaptive and edge-enhanced YOLO according to claim 1, characterized in that, The edge-enhanced attention mechanism includes: The input feature map is passed through a depthwise convolution in one pass. Pointwise convolution, batch normalization, and ReLU activation function are used to generate edge feature maps; Global average pooling is performed on the edge feature map, dimensionality reduction and expansion are achieved through a fully connected layer, and activation is performed through the Sigmoid function to generate channel weight vectors; the channel weight vectors are then multiplied element-wise with the edge feature map to generate enhanced features; Channel-level average pooling and max pooling are performed on the enhanced features. The resulting two two-dimensional feature maps are then concatenated and processed... The convolutional layer, combined with batch normalization and the sigmoid function, generates a spatial attention weight map; the spatial attention weight map is then multiplied element-wise with the augmented features to generate spatial augmented features. The spatial augmentation features are used to perform residual connections with the input features, and the channel dimensions are adjusted to generate the final output features.

8. A crack detection system applying the method according to claims 1-7, characterized in that, include: The surface of the crack being inspected (101), the drone equipped with a camera (102), the data processing system (103), and the display terminal (104). The UAV (102) establishes a communication connection with the data processing system (103) and outputs the detected and analyzed images to the display terminal (104); The data processing system (103) is equipped with a YOLO-MSEE model to acquire images of the surface (101) of the crack being inspected and to perform crack detection.

Images