A dam defect recognition method based on attention feature fusion enhanced network

Abstract

A dam defect recognition method based on an attention feature fusion enhanced network is proposed. The method comprises the following steps: setting a model training set, a model verification set and a model test set; improving the trunk, neck and head parts of the YOLOv8 target detection algorithm model; training, verifying and testing the improved YOLOv8 target detection algorithm model through the model training set, the model verification set and the model test set, and obtaining the tested YOLOv8 target detection algorithm model; detecting the dam body through the tested YOLOv8 target detection algorithm model, and judging whether there are crack defects; using the improved YOLOv8 dam defect detection model for target detection, an improvement of about 1.5% on the underwater robot professional competition data set is achieved without a significant increase in the number of parameters and computational complexity, which effectively improves the recognition precision and accuracy in underwater fuzzy environments, and provides an efficient network model for dam defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater visual recognition, and in particular to a dam defect recognition method based on an attention feature fusion enhanced network. Background Art

[0002] With the popularization of underwater robots in all walks of life, visual recognition technology plays an important role in underwater robots; however, affected by suspended matter in the reservoir water and attachments on the dam surface, the imaging of dam surface defects is blurred during operation, and accurate detection and identification encounter substantial difficulties. In addition, affected by underwater light, water quality, etc., the quality of underwater image collection is poor. These factors make the identification of dam defects extremely difficult. Publication No. CN118552830A is a lightweight method for detecting dangerous cracks in dam bodies for edge computing. It reduces the amount of calculation parameters, improves recognition efficiency, and has a good effect in some application scenarios. In dam defect detection, due to the influence of factors such as suspended matter in the water and water scattering effects, the image becomes blurred, which significantly increases the difficulty of dam defect detection. Summary of the invention

[0003] In view of the problems in the prior art, the present invention provides a dam defect recognition method based on an attention feature fusion enhanced network, aiming to improve the detection capability of fuzzy targets in complex environments and improve the accuracy of detecting dam defects.

[0004] A dam defect recognition method based on an attention feature fusion enhanced network includes the following steps:

[0005] Step 1: Set up model training set, model validation set and model test set;

[0006] Step 2: Improve the trunk, neck, and head of the YOLOv8 target detection algorithm model. The improved modules and network structure are as follows:

[0007] Step 2.1: Integrate the multi-channel feature enhancement module MCEF into the C2f module of its backbone to form a C2f+MCFF module;

[0008] Step 2.2: Add a spatial attention fusion and channel attention fusion collaborative module SCAFM to the data output end of the neck part, and connect a branch of the C2f+MCFF module to the output of the trunk part and send it to the spatial attention fusion and channel attention fusion collaborative module SCAFM in two ways;

[0009] Step 2.3: In the head part, Shape-IoU is introduced as the bounding box loss function;

[0010] Step 3: Train, verify and test the improved YOLOv8 target detection algorithm model through the model training set, model verification set and model test set, and obtain the tested YOLOv8 target detection algorithm model;

[0011] Step 4: Use the tested YOLOv8 target detection algorithm model to detect the dam body and determine whether there are crack defects. If there are crack defects, output the detection result map.

[0012] Further, in step 2, the Bottlenck module in the C2f module is replaced by the fused multi-channel feature enhancement module MCEF to obtain the C2F-MCEF module.

[0013] Furthermore: an efficient channel attention mechanism ECA and a global response normalization mechanism GRN are introduced in the fused multi-channel feature enhancement module MCEF. The efficient channel attention mechanism ECA is used to weight the importance of each channel, and the global response normalization mechanism GRN is used to optimize the contrast and selectivity of features through global feature aggregation and normalization.

[0014] Further: after deep convolution operation on the feature map, it enters the efficient channel attention mechanism ECA, and then global response normalization and batch normalization are performed. The normalized feature map is passed into deep convolution, and then global response normalization and batch normalization operations are performed.

[0015] Further: The global response normalization mechanism GRN includes the following steps: first, through the global feature aggregation operation, the spatial features are condensed into a global vector to extract the overall information of the image; then, these overall information are normalized; finally, the normalized score is used to calibrate the input response.

[0016] Further: The fused multi-channel feature enhancement module MCEF also introduces residual connections,

[0017]

[0018]

[0019]

[0020]

[0021] Conv1(·) and Conv2(·) represent depthwise separable convolutions with kernel sizes of C×3×3 and C×1×1, respectively. It is the input of the fused multi-channel feature enhancement module MCEF. The output of the multi-channel feature enhancement module MCEF is integrated during the training process. In the reasoning process, the output of the multi-channel feature enhancement module MCEF is integrated. is the final training output of the MCEF module, It is the final reasoning output of the fused multi-channel feature enhancement module MCEF.

[0022] Further: In the spatial attention fusion and channel attention fusion collaborative module SCAFM, first, the feature map X output by the backbone network is input into the spatial attention fusion module; in the spatial attention fusion module, the feature map X first passes through the ECA attention, enters the Conv2d convolution with a convolution kernel size of 1×1, passes through batch normalization and activation function RELU processing, and then passes through the convolution kernel size 1×1Conv2d and batch normalization, and finally passes through the sigmoid function to generate weight information, and the generated weight information is multiplied with the feature map input Y processed by the neck network, and then passed out of the spatial attention fusion module; secondly, the feature map input Y processed by the neck network is passed into the channel attention fusion module. In the channel attention fusion module, the input passes through the average pooling operation, passes through the Conv1d convolution with a convolution kernel size of 1×1, and finally generates weight information through the sigmoid function. The generated weight information is multiplied by the feature map X output by the backbone network, and then transmitted to the channel attention fusion module; the feature maps output by the channel attention fusion module and the spatial attention fusion module are added, and then passed through the Conv2d convolution with a convolution kernel size of 1×1 to output the spatial attention fusion and channel attention fusion collaborative module SCAFM.

[0023] Further: The output of the spatial attention fusion module ,

[0024]

[0025]

[0026] Where Conv1(·) represents a 1D convolution with a convolution kernel of size C×1×1;

[0027] Output of the channel attention module ,

[0028]

[0029]

[0030] Among them, Conv2(·) represents a C×3×3 2D convolution;

[0031] The output Z of the spatial attention fusion and channel attention fusion collaborative module SCAFM is:

[0032]

[0033] in, is the input of the spatial attention fusion module, is the input of the channel attention module, input and input Have the same shape and number of channels; BN is batch normalization and Avgpool is average pooling.

[0034] Further, the CIoU loss function in the head part is replaced by the Shape-IoU loss function.

[0035] The beneficial effects of the present invention are as follows: the improved YOLOv8 dam defect detection model is used for target detection, which achieves an increase of about 1.5% and 1.3% in mAP (50:95) on the underwater robot professional competition dataset and the underwater object detection dataset, without a significant increase in the number of parameters and computational complexity, effectively improving the precision and accuracy in underwater fuzzy environments; providing a more efficient recognition algorithm model for underwater dam defect identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a structural diagram of the attention feature fusion enhancement network in the present invention. DETAILED DESCRIPTION

[0037] The present invention is described in detail below in conjunction with the accompanying drawings. The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention. The directional terms such as left, middle, right, top, and bottom in the embodiments of the present invention are only relative concepts or are based on the normal use state of the product, and should not be considered as restrictive.

[0038] A dam defect recognition method based on attention feature fusion enhanced network, such as Figure 1 The method comprises the following steps:

[0039] Step 1: The present invention uses two public data sets, namely the underwater robot professional competition data set and the underwater object detection data set, as evaluation benchmarks, and divides the data sets to set a model training set, a model verification set, and a model test set; then, the model training set, the model verification set, and the model test set are letterboxed to pre-process the image size to make the image size uniform, and then the image is normalized to scale the pixel value of the image to between 0 and 1, or the mean and standard deviation are normalized according to the statistical data during model training;

[0040] Step 2: Improve the trunk, neck, and head of the YOLOv8 target detection algorithm model. The improved modules and network structure are as follows:

[0041] Step 2.1: In the C2f module of its main part, the multi-channel feature enhancement module MCEF is integrated to form the C2f-MCFF module; the C2F-MCEF module can optimize the feature extraction effect of the model, enhance the contrast and selectivity of the features, and perform better when processing complex image tasks, successfully balancing the accuracy, speed and resource consumption, and significantly improving the overall performance of the model and the dam defect recognition ability; specifically: the Bottlenck module in the C2f module is replaced by the fused multi-channel feature enhancement module MCEF to obtain the C2F-MCEF module; and the efficient channel attention mechanism ECA and the global response normalization mechanism GRN are introduced into the fused multi-channel feature enhancement module MCEF to enhance the feature extraction ability of the model. The efficient channel attention mechanism ECA is used to weight the importance of each channel and optimize the model's attention to key features. responsiveness, and improve the accuracy of feature expression; the global response normalization mechanism GRN is used to optimize the contrast and selectivity of features through global feature aggregation and normalization; wherein, after the deep convolution operation on the feature map, it enters the efficient channel attention mechanism ECA, and then performs global response normalization and batch normalization, and the normalized feature map is passed into the deep convolution, and then global response normalization and batch normalization operations are performed; the global response normalization mechanism GRN includes the following steps: first, through global feature aggregation operations, such as global average pooling operations, the spatial features are condensed into a global vector to extract the overall information of the image; then, these overall information are normalized to calculate the importance of each channel relative to other channels, and enhance the responsiveness to key channels; finally, the normalized score is used to calibrate the input response, thereby improving the channel contrast and selectivity of the model;

[0042] The fused multi-channel feature enhancement module MCEF also introduces residual connections.

[0043]

[0044]

[0045]

[0046]

[0047]

[0048] Conv1(·) and Conv2(·) represent depthwise separable convolutions with kernel sizes of C×3×3 and C×1×1, respectively. It is the input of the fused multi-channel feature enhancement module MCEF. The output of the multi-channel feature enhancement module MCEF is integrated during the training process. In the reasoning process, the output of the multi-channel feature enhancement module MCEF is integrated. The final training output of the fused multi-channel feature enhancement module MCEF, It is the final reasoning output of the fused multi-channel feature enhancement module MCEF; the fused multi-channel feature enhancement module MCEF adopts the reparameterization technology derived from the RepVGG network, and introduces a multi-branch topology structure in the training stage to improve performance; the multi-branch structure can capture more features in the training stage and enhance the expressiveness of the model; in the reasoning stage, these multi-branch structures will be integrated into a single branch, thereby effectively reducing the consumption of computing and memory; this design utilizes the performance improvement brought by the multi-branch structure during training, and avoids additional resource overhead during reasoning, taking into account both performance and efficiency;

[0049] Step 2.2: Add a spatial attention fusion and channel attention fusion collaborative module SCAFM to the data output end of the neck part, and make a branch of the C2f-MCFF module connected to the output of the trunk part and send it to the spatial attention fusion and channel attention fusion collaborative module SCAFM in two ways; by introducing the spatial attention fusion and channel attention fusion collaborative module SCAFM, the present invention can more effectively fuse context information and enrich detail features; the present invention can enhance the fusion of features at different levels, improve the feature expression ability of the network, and thus improve the target detection performance in complex environments;

[0050] In the spatial attention fusion and channel attention fusion collaborative module SCAFM, first, the feature map X (size is H×W×C) output by the backbone network is input into the spatial attention fusion module; in the spatial attention fusion module, the feature map X first passes through the ECA attention, enters the Conv2d convolution with a convolution kernel size of 1×1 (the feature map size becomes H×W×C / 4), passes through batch normalization and activation function RELU processing, then passes through the convolution kernel size 1×1Conv2d (the feature map size becomes H×W×C) and batch normalization, and finally passes through the sigmoid function to generate weight information, which is multiplied by the feature map input Y processed by the neck network, and then passes out of the spatial attention fusion module. Secondly, the feature map input Y (size is H×W×C) processed by the neck network is passed into the channel attention fusion module. In the channel attention fusion module, the input passes through the average pooling operation (the feature map size becomes 1×1×C), passes through the Conv1d convolution with a convolution kernel size of 1×1 (the feature map size is 1×1×C), and finally passes through the sigmoid function to generate weight information. The generated weight information is multiplied by the feature map X output by the backbone network, and then transmitted to the channel attention fusion module; the feature maps output by the channel attention fusion module and the spatial attention fusion module are added, and then passed through the Conv2d convolution with a convolution kernel size of 1×1 to the spatial attention fusion and channel attention fusion collaborative module SCAFM;

[0051] The spatial attention fusion module is used to further enhance the target recognition. By aggregating the channel context features of each spatial position, the spatial details in the low-level features are used to enrich the high-level features, thereby improving the detection effect. In addition, although the deep features provide accurate semantic information, the deep features alone are still insufficient to meet the detection needs of complex targets. Therefore, the channel attention fusion module is used to solve this problem. By adjusting the importance weights of each channel, the shallow and deep features are effectively integrated to enhance the feature expression ability. The output of the spatial attention fusion module ,

[0052]

[0053]

[0054] Where Conv1(·) represents a 1D convolution with a convolution kernel of size C×1×1;

[0055] Output of the channel attention module ,

[0056]

[0057]

[0058] Among them, Conv2(·) represents a C×3×3 2D convolution;

[0059] The channel attention module first extracts the global feature representation of each channel through global average pooling, and learns the importance weights of the channels through a convolutional network; then, these weights are normalized and combined with shallow features to embed deep semantic information and optimize feature expression, thereby improving the model's detection effect on targets in complex scenes; the output Z of the spatial attention fusion and channel attention fusion collaborative module SCAFM is:

[0060]

[0061] in, is the input of the spatial attention fusion module, is the input of the channel attention module, input and input have the same shape and number of channels; , Explain its meaning); the spatial attention fusion and channel attention fusion collaborative module SCAFM expresses the details of deep features and enhances the semantic understanding of low-level features, thereby improving the detection ability of blurred targets in complex environments;

[0062] In step 2.1 and step 2.2, after image preprocessing, the processed image will be input into the backbone network for feature extraction; in the backbone network, convolution operation is first performed through the Conv module, and then feature extraction is performed through the C2F-MCEF module. The Conv module and the C2F-MCEF module are serially stacked 4 times for feature extraction. In the 4 times of serial stacking of the Conv module and the C2F-MCEF module, three feature maps of different scales are extracted from the second layer P3, the third layer P4, and the fourth layer P5. The output of the backbone part is divided into two parts, one part is transmitted to the neck part through the second layer P3, the third layer P4, and the fourth layer P5, and the other part continues to be transmitted to the spatial pyramid pooling module SPPF. The feature maps are pooled at the same scale to extract richer feature information. The features processed by SPPF will also be fused and output to the neck supplement. In addition, the multi-scale feature maps extracted from the trunk generate features of different sizes through the second layer P3, the third layer P4, and the fourth layer P5, and these feature maps are passed to the neck part for further convolution operations and upsampling, downsampling and other processing. In the neck part, the feature maps after upsampling and feature fusion will be further processed through multiple convolution layers and C2f modules to ensure that cross-level features are optimized and enhanced. The feature fusion operation of each layer is compressed and refined through the convolution and C2f modules. After passing through the neck part, the data continues to be transmitted to the head part.

[0063] Step 2.3: In the head part, Shape-IoU is introduced as the bounding box loss function; specifically, the CIoU loss function in the head part is replaced by the Shape-IoU loss function; compared with the traditional IoU loss function, the Shape-IoU loss function not only focuses on the overlapping area, but also combines the shape and size factors of the bounding box to calculate the loss. This method makes the bounding box regression more accurate and improves the detection performance of the model in complex environments; the Shape-IoU loss function carefully describes the difference between the predicted box and the real box by considering factors such as the width, height, and position of the bounding box; in complex underwater scenes, the Shape-IoU loss function can better cope with complex situations and improve the accuracy and robustness of the predicted box. Its design integrates the overlap, shape, and size of the bounding box, so it performs well on diverse data sets. Overall, the Shape-IoU loss function provides a more accurate loss calculation method for target detection, especially for complex application scenarios such as underwater target detection;

[0064] Step 3: Train, verify and test the improved YOLOv8 target detection algorithm model through the model training set, model verification set and model test set, and obtain the tested YOLOv8 target detection algorithm model;

[0065] Step 4: Use the tested YOLOv8 target detection algorithm model to detect the dam body and determine whether there are crack defects. If there are crack defects, output the detection result map, including target category, location, size and other information.

[0066] As shown in Table 1, the improved YOLOv8 dam defect detection model is used for target detection, which achieves an increase of about 1.5% and 1.3% in mAP (50:95) on the underwater robot professional competition dataset and underwater object detection dataset, effectively improving the precision and accuracy in underwater fuzzy environments, and providing an efficient network model for dam defect detection.

[0067] Table 1.

[0068]

[0069] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A dam defect recognition method based on attention feature fusion enhanced network, characterized in that: The following steps are involved: Step 1: Set up model training set, model validation set and model test set; Step 2: Improve the trunk, neck, and head of the YOLOv8 target detection algorithm model. The improved modules and network structure are as follows: Step 2.1: Integrate the multi-channel feature enhancement module MCEF into the C2f module in the backbone part to form a C2f-MCFF module; In the fused multi-channel feature enhancement module MCEF, an efficient channel attention mechanism ECA and a global response normalization mechanism GRN are introduced. The efficient channel attention mechanism ECA is used to weight the importance of each channel, and the global response normalization mechanism GRN is used to optimize the contrast and selectivity of features through global feature aggregation and normalization. The MCEF module also introduces residual connections. Conv1(·) and Conv2(·) represent depthwise separable convolutions with kernel sizes of C×3×3 and C×1×1, respectively. N1 is the input of the fused multi-channel feature enhancement module MCEF, Z1 is the output of the fused multi-channel feature enhancement module MCEF during training, Z2 is the output of the fused multi-channel feature enhancement module MCEF during reasoning, W1 is the final training output of the fused multi-channel feature enhancement module MCEF, and W2 is the final reasoning output of the fused multi-channel feature enhancement module MCEF. Step 2.2: Add a spatial attention fusion and channel attention fusion collaborative module SCAFM to the data output end of the neck part, and connect a branch of the C2f-MCFF module to the output of the trunk part and send it to the spatial attention fusion and channel attention fusion collaborative module SCAFM in two ways; Step 2.3: In the head part, Shape-IoU is introduced as the bounding box loss function; Step 3: Train, verify and test the improved YOLOv8 target detection algorithm model through the model training set, model verification set and model test set, and obtain the tested YOLOv8 target detection algorithm model; Step 4: Use the tested YOLOv8 target detection algorithm model to detect the dam body and determine whether there are crack defects. If there are crack defects, output the detection result map.

2. The dam defect recognition method based on the attention feature fusion enhanced network according to claim 1 is characterized in that: In step 2, the Bottlenck module in the C2f module is replaced by the fused multi-channel feature enhancement module MCEF to obtain the C2F-MCEF module.

3. The dam defect recognition method based on attention feature fusion enhanced network according to claim 1 is characterized in that: After deep convolution operation on the feature map, it enters the efficient channel attention mechanism ECA, and then global response normalization and batch normalization are performed. The normalized feature map is passed into deep convolution, and then global response normalization and batch normalization operations are performed.

4. The dam defect recognition method based on attention feature fusion enhanced network according to claim 1 is characterized in that: The global response normalization mechanism GRN includes the following steps: first, through the global feature aggregation operation, the spatial features are condensed into a global vector to extract the overall information of the image; then, this overall information is normalized; finally, the normalized score is used to calibrate the input response.

5. The dam defect recognition method based on attention feature fusion enhanced network according to claim 1 is characterized in that: In the spatial attention fusion and channel attention fusion collaborative module SCAFM, first, the feature map X output by the backbone network is input into the spatial attention fusion module; In the spatial attention fusion module, the feature map X first passes through the efficient channel attention mechanism ECA, enters the Conv2d convolution with a convolution kernel size of 1×1, passes through batch normalization and activation function RELU processing, then passes through the convolution kernel size 1×1Conv2d and batch normalization, and finally passes through the sigmoid function to generate weight information. The generated weight information is multiplied with the feature map input Y processed by the neck network, and then transmitted out of the spatial attention fusion module; secondly, the feature map input Y processed by the neck network is passed into the channel attention fusion module; in the channel attention fusion module, the input passes through the average pooling operation, passes through the Conv1d convolution with a convolution kernel size of 1×1, and finally passes through the sigmoid function to generate weight information. The generated weight information is multiplied with the feature map X output by the backbone network, and then transmitted out of the channel attention fusion module; the feature maps output by the channel attention fusion module and the spatial attention fusion module are added, and then passed through the Conv2d convolution with a convolution kernel size of 1×1 to output the spatial attention fusion and channel attention fusion collaborative module SCAFM.

6. The dam defect recognition method based on attention feature fusion enhanced network according to claim 5 is characterized in that: The output D1 of the spatial attention fusion module, D1=Sigmoid(BN(Conv1(ReLU(BN(Conv1(ECA(X1))))))) Where Conv1(·) represents a 1D convolution with a convolution kernel of size C×1×1; The output D2 of the channel attention module, D2=Sigmoid(Conv2(Avgpool(Y1))) Among them, Conv2(·) represents a C×3×3 2D convolution; The output Z of the spatial attention fusion and channel attention fusion collaborative module SCAFM is: Among them, X1 is the input of the spatial attention fusion module, X2 is the input of the channel attention module, input X1 and input Y1 have the same shape and number of channels; BN is batch normalization, and avgpool is average pooling.

7. The dam defect recognition method based on attention feature fusion enhanced network according to claim 1 is characterized in that: Replace the CIoU loss function in the head part with the Shape-IoU loss function.

Images