Bridge apparent disease detection method based on wavelet multi-band channel attention

Abstract

The invention relates to the technical field of image processing and computer vision, and discloses a bridge apparent disease detection method based on wavelet multi-band channel attention, and the method comprises the steps: obtaining a to-be-detected bridge apparent disease image, and inputting the image into a pre-trained bridge apparent disease detection model, obtaining a disease positioning detection result of the to-be-detected bridge apparent disease image; the bridge apparent disease detection model comprises a backbone network, a neck network and a detection head network; the bridge apparent disease image serves as input of the backbone network, multi-level semantic features are extracted, the extracted multi-level semantic features are fused through the neck network, multi-scale fusion features are obtained and input to the detection head network, and finally a disease positioning detection result of the bridge apparent disease image is obtained. According to the bridge apparent disease detection model constructed by the invention, the positioning precision of bridge apparent disease detection is effectively improved, the calculation complexity is reduced, and network parameters are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and computer vision, and in particular to a method for detecting apparent bridge defects based on wavelet multi-band channel attention. Background Art

[0002] As of the end of 2022, China had a total of 1.0332 million highway bridges, totaling 85.7649 million meters. Over time, concrete bridges can develop defects such as concrete spalling, exposed rebar, corrosion, and weathering due to vehicle traffic, climate change, and material aging. These problems can pose safety risks and even lead to collapse. Therefore, timely detection and maintenance of concrete bridge defects are crucial.

[0003] Currently, traditional bridge defect detection relies primarily on manual inspection, with engineers and technicians performing regular inspections through visual inspection and instrumental measurements. However, this method suffers from high subjectivity, high rates of missed and false detections, and difficulty inspecting long-span bridges and bridges in mountainous areas, making it difficult to provide comprehensive and accurate inspection results. Most importantly, manual inspections cannot provide comprehensive and continuous monitoring of bridge structures, making it difficult to promptly detect potential hazards. To address these challenges, digital image processing technology has been introduced to bridge defect detection. By utilizing high-resolution digital cameras and advanced image processing algorithms, it enables automated, real-time monitoring of bridge structures, significantly improving detection accuracy. Early efforts to detect structural defects primarily focused on cracks in concrete structures. Using image processing algorithms, digital images of defects were processed through grayscale processing, binarization, sharpening, image enhancement, and edge detection to extract and isolate crack features, enabling crack identification. However, digital image processing technology still faces challenges when dealing with complex scenarios and specific defects. For example, image processing algorithms may not be able to accurately identify diseases in cases of uneven lighting or a large number of shadows. In addition, the sensitivity of digital image processing technology needs to be further improved for some minor diseases.

[0004] To further improve the accuracy and efficiency of defect detection, deep learning-based object detection methods have gradually replaced traditional manual inspections in recent years and have become a hot topic in bridge defect detection. Deep learning-based object detection methods not only improve detection accuracy but also expand the detection range, leading to their increasing application in bridge surface defect detection. However, deep learning-based object detection methods still face challenges, including difficulty detecting in complex backgrounds, severe noise interference, and significant loss of feature information. These issues limit the performance of concrete bridge surface defect detection systems, necessitating a more effective and precise solution. Summary of the Invention

[0005] In response to the above-mentioned shortcomings of the existing technology, the present invention provides a bridge apparent disease detection method based on wavelet multi-band channel attention. By designing an improved image processing module, including a multi-band channel attention module based on wavelet transform and a C2f_BNS lightweight feature extraction module based on channel rearrangement, the positioning accuracy of bridge apparent disease detection is effectively improved, the computational complexity is reduced, and the network parameters are reduced.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A bridge apparent defect detection method based on wavelet multi-band channel attention obtains a bridge apparent defect image to be detected, inputs it into a pre-trained bridge apparent defect detection model, and obtains the defect location detection result of the bridge apparent defect image to be detected;

[0008] The bridge apparent defect detection model includes a backbone network, a neck network and a detection head network. The bridge apparent defect image is used as the input of the backbone network to extract multi-level semantic features to capture defect information from low-level texture to high-level semantics. The extracted multi-level semantic features are then fused through the neck network to obtain multi-scale fusion features and input into the detection head network. Finally, the defect location detection result of the bridge apparent defect image is obtained as the output of the bridge apparent defect detection model.

[0009] As a preferred solution, the backbone network includes a CBS module, three wavelet multi-frequency lightweight fusion units and a spatial pyramid pooling module connected in sequence; the wavelet multi-frequency lightweight fusion unit includes a cascaded CBS module and a C2f_BNS lightweight feature extraction module, wherein a wavelet multi-band channel attention module is inserted into each of the wavelet multi-frequency lightweight fusion units; the CBS module includes a cascaded convolutional layer, a BN layer and a SiLU activation function.

[0010] As a preferred solution, the processing process of the wavelet multi-band channel attention module is shown in the following formula:

[0011] WMA(X)=X*sigmoid(fc(F sq (Z g )));

[0012] F sq (Z g )=concat(Z LL ,Z LH ,Z HL ,Z HH ,Z MIX );

[0013]

[0014] Where WMA(X) represents the output of the wavelet multi-band channel attention module, X represents the input feature map, * represents channel-by-channel multiplication, sigmoid(·) represents the activation function, fc(·) represents the fully connected layer, and F sq (·) represents the feature vector after multi-band compression, concat(·) represents concatenation along the channel dimension, Z LL ,Z LH ,Z HL ,Z HH They represent the compression features of the low-frequency, horizontal high-frequency, vertical high-frequency and diagonal high-frequency sub-bands after the input feature map X is decomposed by wavelet, and Z MIX Represents the compression characteristics of the mixed frequency band, Z g represents the global channel statistics after the compression of each frequency band feature, Compress(·) represents the global spatial compression, and Y g It represents the frequency band feature map reconstructed after inverse wavelet transform, and H and W represent the height and width of the feature map respectively.

[0015] As a preferred solution, the C2f_BNS lightweight feature extraction module is constructed based on channel rearrangement and the BNS module using depthwise separable convolution and pointwise convolution. The specific processing process includes the following steps:

[0016] The input feature map is subjected to feature extraction by the first CBS module, and the extracted feature map is split into two groups of parallel branches along the channel dimension through the Split operation. The split group of branch feature maps enters N BNS modules in turn through depth-wise separable convolution and point-by-point convolution for feature extraction, and the output of the previous BNS module is used as the input of the next BNS module. The feature map extracted by each BNS module is fused with the other group of branch features after splitting in the connection layer. The fused feature map passes through the channel rearrangement algorithm and then passes through the second CBS module for output.

[0017] As a preferred solution, the processing process of the BNS module includes the following steps:

[0018] The input feature map is split into two groups of parallel branches along the channel dimension through the Split operation. The split group of branches first undergoes channel dimension transformation through a 1×1 point-by-point convolution module, and then undergoes spatial feature extraction through a 3×3 depth-wise convolution module. The channel dimension is then adjusted again through a 1×1 point-by-point convolution module. The adjusted feature map is concatenated with the other group of split branches in the connection layer and used as the input of the channel rearrangement algorithm to re-interleave the output channels of different branches.

[0019] As a preferred solution, the neck network includes two upsampling lightweight feature fusion units and two convolutional feature recombination units connected in sequence; the upsampling lightweight feature fusion unit includes a cascaded upsampling module, a connection layer and a C2f_BNS lightweight feature extraction module, and the convolutional feature recombination unit includes a cascaded CBS module, a connection layer and a C2f_BNS lightweight feature extraction module;

[0020] Among them, the input of the bridge apparent disease detection model serves as the input of the backbone network; the output of the second wavelet multi-frequency lightweight fusion unit and the third wavelet multi-frequency lightweight fusion unit in the backbone network are also used as the input of the connection layer in the first upsampling lightweight feature fusion unit and the connection layer in the second upsampling lightweight feature fusion unit in the neck network respectively; the output of the spatial pyramid pooling module in the backbone network is also used as the input of the connection layer in the second convolutional feature recombination unit in the neck network; the output of the first upsampling lightweight feature fusion unit in the neck network is also used as the input of the connection layer in the first convolutional feature recombination unit.

[0021] As a preferred solution, the detection head network includes three detection units connected in sequence, and the three detection units are connected in parallel, wherein each detection unit includes a bounding box prediction module and a category prediction module connected in parallel, the bounding box prediction module includes a cascaded CBS module, a two-dimensional convolutional layer and a bounding box loss function, and the category prediction module includes a cascaded CBS module, a two-dimensional convolutional layer and a category loss function;

[0022] Among them, the outputs of the second upsampling lightweight feature fusion unit and the first convolutional feature reorganization unit in the neck network are respectively used as the inputs of the first detection unit and the second detection unit in the detection head network, and the output of the second convolutional feature reorganization unit in the neck network is used as the input of the third detection unit in the detection head network.

[0023] As a preferred solution, the training and testing process of the bridge apparent disease detection model includes:

[0024] The learning rate and hyperparameters of the bridge apparent disease detection model are set, and the bridge apparent disease detection model with pre-completed bridge area segmentation and disease labeling is used as training samples to form a training sample set. The bridge apparent disease detection model is input, and the number of iterations is set to train the bridge apparent disease detection model. The CIoU loss function is used to calculate the training loss of the plant disease detection model. The parameters of the bridge apparent disease detection model are updated with the goal of minimizing the CIoU loss function to obtain the trained bridge apparent disease detection model. Finally, the bridge apparent disease detection model is tested based on the training sample set.

[0025] As a preferred solution, the training sample set is constructed using a sample set of a public data set and an acquired bridge apparent disease data set.

[0026] As a preferred solution, the CIoU loss function is expressed as:

[0027]

[0028] Among them, L CIoU represents CIoU loss; ρ(b,b gt ) represents the center point b of the model predicted disease area positioning frame and the center point b of the real disease area positioning frame gt c represents the diagonal length of the minimum bounding rectangle covering the predicted diseased area positioning frame and the true diseased area positioning frame; α is the preset weight coefficient; v is the consistency parameter used to measure the difference in aspect ratio between the predicted diseased area positioning frame and the true diseased area positioning frame, and w and h are the width and height of the predicted disease area positioning box, respectively. gt 、h gt are the width and height of the predicted disease area positioning frame respectively; IoU represents the ratio of the intersection area to the union area of ​​the predicted disease area positioning frame and the true disease area positioning frame, that is:

[0029]

[0030] Among them, B, B gt They represent the predicted disease area positioning frame and the real disease area positioning frame respectively, and Area(·) represents area operation.

[0031] Compared with the prior art, the present invention has the following technical effects:

[0032] (1) The wavelet multi-band channel attention module (WMA) of the present invention introduces the multi-band information and channel attention mechanism of wavelet transform, obtains the multi-band channel attention mechanism with different attention levels to each channel from different frequency bands, generalizes the existing channel attention in the wavelet band, solves the information loss problem of different frequency bands, and improves the target detection network's ability to extract feature information of different bridge surface defects; specifically, the module uses the Haar wavelet basis to decompose the input feature map to obtain four feature sub-bands of LL (low-frequency component), LH (horizontal high-frequency component), HL (vertical high-frequency component) and HH (diagonal high-frequency component), and reconstructs the features of each sub-band through inverse wavelet transform, thereby completely retaining the defect feature information of different directions and scales; in the design of the attention mechanism, WMA performs global spatial compression on the features of each frequency band to generate channel descriptors, dynamically learns the correlation between frequency bands through the fully connected layer, and uses sigmoid activation to realize soft attention weight distribution, so that the model can adaptively focus on important features of different frequency bands.

[0033] (2) The C2f_BNS lightweight feature extraction module of the present invention first combines the dual advantages of point-by-point convolution and depth-wise separable convolution. Point-by-point convolution is responsible for cross-channel feature information fusion and realizes efficient channel dimension transformation through 1×1 convolution kernel; while depth-wise separable convolution focuses on spatial feature extraction within a single channel, using 3×3 convolution kernel to capture local features while reducing computational complexity; secondly, it integrates the channel rearrangement algorithm, which breaks the information isolation problem caused by traditional grouped convolution through periodic channel permutation operations, enhances the feature extraction capability, and thus improves the lightweight and reliability of the detection system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to make the purpose, technical solutions and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0035] Figure 1 This is a schematic diagram of the structure of the bridge apparent disease detection model constructed by the multi-band channel attention module based on wavelet transform and the C2f_BNS lightweight feature extraction module based on channel rearrangement in the present invention;

[0036] Figure 2 This is a sample diagram of apparent damage to a bridge according to an embodiment of the present invention;

[0037] Figure 3 This is a distribution diagram of the number of various apparent disease samples in the self-made dataset (CCBAD) of the embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of the structure of a multi-band channel attention module based on wavelet transform according to an embodiment of the present invention;

[0039] Figure 5 Schematic diagram of the channel reordering algorithm process according to an embodiment of the present invention;

[0040] Figure 6 Schematic diagram of a C2f_BNS lightweight feature extraction module based on channel rearrangement according to an embodiment of the present invention;

[0041] Figure 7 Schematic diagram of the structure of the BNS module according to an embodiment of the present invention;

[0042] Figure 8 This is a graph showing the increase in training accuracy of each model in an embodiment of the present invention;

[0043] Figure 9 The following are the detection results of each model in the comparative experiment of the embodiment of the present invention on the self-made dataset (CCBAD). DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0045] The present invention will be described in further detail below with reference to the accompanying drawings.

[0046] Figure 1 The present invention discloses a bridge apparent defect detection method based on wavelet multi-band channel attention. The method inputs the acquired bridge apparent defect image to be detected into a trained bridge apparent defect detection model, identifies and locates the detected bridge apparent defect image, and obtains the defect location detection result of the bridge apparent defect image to be detected.

[0047] The present invention aims to solve the problem of insufficient feature extraction of defects under complex backgrounds. In the bridge apparent defect detection model used in the present invention, a multi-band channel attention module based on wavelet transform and a C2f_BNS lightweight feature extraction module based on channel rearrangement are constructed. A multi-band channel attention module (WMA) based on wavelet transform is proposed. This technology decomposes the features into four feature sub-bands: LL low-frequency approximation, LH horizontal details, HL vertical details and HH diagonal details through Haar wavelet transform, and designs a mixed frequency band feature Y MIX Cross-band fusion is achieved, and the frequency-domain sensitive channel attention mechanism is combined to solve the problem of information loss in different frequency bands, thereby improving the target detection network's ability to extract feature information of different bridge apparent defects; secondly, in response to the problem of high model calculation complexity, the bridge apparent defect detection model used in the present invention proposes a C2f_BNS lightweight feature extraction module, which effectively reduces the number of network parameters through the innovative combination of depthwise separable convolution and point-by-point convolution. At the same time, the channel rearrangement algorithm is integrated to effectively reduce the loss of channel information and enhance the feature extraction capability, thereby improving the lightweight and reliability of the detection system.

[0048] The bridge apparent defect detection method based on wavelet multi-band channel attention of the present invention is described in more detail below.

[0049] 1. Constructing a bridge apparent damage dataset

[0050] A small number of images of bridge apparent defects that meet the needs of actual bridge engineering maintenance and images of common and serious apparent defects of concrete bridges collected in this embodiment, totaling 1,450, were screened from the CODEBRIM dataset and the dacl10k dataset. In order to avoid the inability to obtain the optimal model due to the defects of the dataset, they were screened after data enhancement, and all images were annotated more carefully to obtain categories with more than 7,500 instances. Finally, 7,243 images were obtained, including four common and serious concrete bridge apparent defects: rust, exposed reinforcement, salt corrosion, and spalling, with a total of 40,065 labels. In this embodiment, it is named Chongqing Concrete Bridge Apparent Defect Dataset (CCBAD).

[0051] A single image contains more than one concrete bridge apparent disease. In order to better present the concrete bridge apparent disease, the image is cropped appropriately, such as Figure 2 As shown, Figure 2 The first row of sample images primarily shows exposed rebar, a surface defect of concrete bridges, accompanied by corrosion and spalling. The second row shows corrosion, a surface defect of concrete bridges. The third row shows spalling, and the fourth row shows salt corrosion. Figure 3 This is the distribution diagram of the number of samples of various apparent diseases in the self-made data set.

[0052] 2. Bridge apparent disease detection model

[0053] Aiming at the pain points of low efficiency of existing manual detection, prominent safety hazards and insufficient adaptability of existing algorithms, the bridge surface disease detection model used in the solution of the present invention is as follows: Figure 1 As shown in the figure, the model combines wavelet frequency domain analysis and spatial domain feature extraction technology. Through the multimodal feature collaborative processing mechanism, it significantly improves the accuracy and robustness of disease identification in complex environments. Its lightweight architecture design enables the model to be efficiently deployed on mobile detection equipment to meet the real-time processing needs in actual engineering scenarios. Specifically, Figure 1 As shown, the bridge apparent disease detection model used in the scheme of the present invention includes a backbone network, a neck network and a detection head network; the bridge apparent disease image input to the bridge apparent disease detection model is used as the input of the backbone network, and multi-level semantic features are extracted to capture disease information from low-level texture to high-level semantics. The extracted multi-level semantic features are then fused through the neck network to obtain multi-scale fusion features and input into the detection head network. Finally, the disease location detection result of the bridge apparent disease image is obtained as the output of the bridge apparent disease detection model.

[0054] Specifically, the backbone network includes a CBS module, three wavelet multi-frequency lightweight fusion units and a spatial pyramid pooling module connected in sequence; the wavelet multi-frequency lightweight fusion unit includes a cascaded CBS module and a C2f_BNS lightweight feature extraction module, wherein a wavelet multi-band channel attention module is inserted into each of the wavelet multi-frequency lightweight fusion units; the CBS module includes a cascaded convolutional layer, a BN layer and a SiLU activation function.

[0055] The neck network includes two sequentially connected upsampling lightweight feature fusion units and two convolutional feature reorganization units; the upsampling lightweight feature fusion unit includes a cascaded upsampling module, a connection layer and a C2f_BNS lightweight feature extraction module, and the convolutional feature reorganization unit includes a cascaded CBS module, a connection layer and a C2f_BNS lightweight feature extraction module;

[0056] The detection head network includes three detection units connected in sequence, and the three detection units are connected in parallel, wherein each detection unit includes a bounding box prediction module and a category prediction module connected in parallel, the bounding box prediction module includes a cascaded CBS module, a two-dimensional convolutional layer and a bounding box loss function, and the category prediction module includes a cascaded CBS module, a two-dimensional convolutional layer and a category loss function;

[0057] Among them, the input of the bridge apparent disease detection model serves as the input of the backbone network; the output of the second wavelet multi-frequency lightweight fusion unit and the third wavelet multi-frequency lightweight fusion unit in the backbone network are also used as the input of the connection layer in the first upsampling lightweight feature fusion unit and the connection layer in the second upsampling lightweight feature fusion unit in the neck network respectively; the output of the spatial pyramid pooling module in the backbone network is also used as the input of the connection layer in the second convolutional feature reorganization unit in the neck network; the output of the first upsampling lightweight feature fusion unit in the neck network is also used as the input of the connection layer in the first convolutional feature reorganization unit; the output of the second upsampling lightweight feature fusion unit and the first convolutional feature reorganization unit in the neck network are respectively used as the input of the first detection unit and the second detection unit in the detection head network; the output of the second convolutional feature reorganization unit in the neck network serves as the input of the third detection unit in the detection head network.

[0058] As can be seen, the bridge surface defect detection model of this embodiment introduces a multi-band channel attention module based on wavelet transform in the backbone network, which can extract rich and diverse frequency domain information and compress each frequency domain information. The compressed frequency domain information enables the network to focus on more important channels. By introducing a channel rearrangement algorithm and designing a new Bottleneck (BNS) module, a channel rearrangement-based C2f_BNS lightweight feature extraction module is obtained. This allows information between the stacked C2f channels in the original YOLOv8 to be exchanged and achieves better capacity-efficiency, resulting in richer feature information and increasing the model's understanding and representation capabilities of the input data.

[0059] The following is a detailed introduction to these two modules.

[0060] 2.1 Multi-band channel attention module based on wavelet transform

[0061] like Figure 4 The figure shows the framework of the multi-band channel attention module based on wavelet transform in this embodiment. This embodiment uses the Haar wavelet basis due to its computational efficiency and simplicity. Input X is defined as the Haar wavelet basis G = [GLL, GLH, GHL, GHH]. X is decomposed into four sub-band images using the Haar wavelet basis.

[0062] G is expressed by the following formula:

[0063]

[0064] Where G LL ,G LH ,G HL ,G HH They represent low-frequency filter, horizontal high-frequency filter, vertical high-frequency filter and diagonal high-frequency filter respectively.

[0065] The four sub-band images are defined as LL, LH, HL, and HH, which represent the low-frequency component, the horizontal high-frequency component, the vertical high-frequency component, and the diagonal high-frequency component, respectively. The resolution of each channel in each spatial dimension is half of X. Therefore, the DWT operation can be defined as the formula:

[0066]

[0067] Where, Represents the convolution operation, which is to convolve the input X using different wavelet filters to obtain the corresponding frequency domain components;

[0068] Secondly, after the image is converted to the wavelet domain, its high-frequency sub-band and low-frequency sub-band not only have the characteristics of the wavelet domain, but also retain the properties of the spatial domain. In order to further strengthen the wavelet domain information and retain and enrich the information of each frequency domain to ensure that more channel information is obtained when the channel attention is compressed, the present invention uses the inverse wavelet operation after recombining each frequency domain component to obtain a feature map rich in frequency domain information. The feature map containing different frequency domain information is defined as:

[0069] Y LL =IWT(G,concat(LL,LL,LL,LL);

[0070] Y LH =IWT(G,concat(LJ,LJ,LH,LH);

[0071] Y HL =IWT(G,concat(HL,HL,HL,HL);

[0072] Y HH =IWT(G,concat(HH,HH,HH,HH);

[0073] Y MIX =IWT(G,concat(LL,LH,HL,HH);

[0074] Where Y LL ,Y LH ,Y HL ,Y HH Respectively represent low-frequency reconstruction features, horizontal high-frequency reconstruction features, vertical high-frequency reconstruction features and diagonal high-frequency reconstruction features, Y MIX represents the reconstructed features of mixed frequency bands, IWT represents the inverse wavelet operation, G represents the wavelet basis convolution kernel required for the inverse wavelet, and concat(·) represents concatenation along the channel dimension;

[0075] The sub-band images of different frequency domains generated by DWT are specifically spliced ​​together to enhance the information of the corresponding characteristic frequency domain. The spliced ​​sub-band images are used as input, and the original signal is restored through inverse wavelet operation to obtain a feature map containing different frequencies.

[0076] For the frequency domain feature information generated by the wavelet frequency domain component of each part, these different frequency domain feature information are used as the compression result of the channel attention weight, and its formula is expressed as follows:

[0077]

[0078] Where Z grepresents the global channel statistics after the compression of each frequency band feature, Compress(·) represents the global spatial compression, and Y g represents the frequency band feature map reconstructed after inverse wavelet transform, H and W represent the height and width of the feature map respectively;

[0079] Each feature map obtained in the wavelet frequency domain is compressed and then spliced ​​together to obtain the compressed feature information vector, which is expressed as follows:

[0080] F sq (Z g )=cancat(Z LL ,Z LH ,Z LH ,Z HH ,Z MIX );

[0081] Where, F sq (·) represents the feature vector after multi-band compression, concat(·) represents concatenation along the channel dimension, Z LL ,Z LH ,Z HL ,Z HH Respectively represent the compression features of the low-frequency, horizontal high-frequency, vertical high-frequency and diagonal high-frequency sub-bands after the input feature map X is decomposed by wavelet;

[0082] Therefore, the formula of the entire wavelet transform-based multi-band channel attention module is expressed as:

[0083] WMA(X)=X*sigmoid(fc(F sq (Z g )));

[0084] Where WMA(X) represents the output of the wavelet multi-band channel attention module, X represents the input feature map, * represents channel-by-channel multiplication, sigmoid(·) represents the activation function, and fc(·) represents the fully connected layer.

[0085] The wavelet multi-band channel attention module (WMA) of this embodiment introduces the multi-band information and channel attention mechanism of wavelet transform, obtains a multi-band channel attention mechanism with different degrees of attention to each channel from different frequency bands, generalizes the existing channel attention in the wavelet band, solves the problem of information loss in different frequency bands, and improves the target detection network's ability to extract feature information of different bridge surface defects. Specifically, the module uses the Haar wavelet basis to decompose the input feature map to obtain four feature sub-bands: LL (low-frequency component), LH (horizontal high-frequency component), HL (vertical high-frequency component), and HH (diagonal high-frequency component), and reconstructs the features of each sub-band through inverse wavelet transform, thereby fully retaining the defect feature information of different directions and scales. In terms of attention mechanism design, WMA performs global spatial compression on the features of each frequency band to generate channel descriptors, dynamically learns the correlation between frequency bands through the fully connected layer, and uses sigmoid activation to realize soft attention weight allocation, so that the model can adaptively focus on important features of different frequency bands.

[0086] 2.2. C2f_BNS lightweight feature extraction module based on channel rearrangement

[0087] The C2f_BNS lightweight feature extraction module in this embodiment is constructed based on channel rearrangement and a BNS module using depthwise separable convolution and pointwise convolution.

[0088] In this embodiment, the Channel_shuffle algorithm is introduced into the C2f structure to enable isolated channels to communicate information. The Channel_shuffle algorithm is an algorithm for channel information communication, which aims to increase the information flow between channels, thereby enhancing the expression ability of the neural network. It can enable the network model to learn the important features and details of the input data well, thereby making the network perform well in practical applications. Figure 5 As shown in the figure, the principle of the Channel_shuffle algorithm is to divide the channels of the feature map after splicing the C2f module into several subgroups, and then feed different subgroups into each group in the next layer. If a convolution layer has g groups, its output has g×n channels; first reshape the output channel dimension to (g,n), transpose it and then flatten it as the input of the next layer. In order to meet the needs of actual engineering and improve the extraction of important features by C2f, depthwise separable convolution and point-by-point convolution are used to improve Bottleneck and lightweight it. The improved C2f is the C2f_BNS lightweight feature extraction module in this embodiment. Figure 6 As shown, the specific processing process includes the following steps:

[0089] The input feature map is subjected to feature extraction by the first CBS module, and the extracted feature map is split into two groups of parallel branches along the channel dimension through the Split operation. The split group of branch feature maps enters N BNS modules in turn through depth-wise separable convolution and point-by-point convolution for feature extraction, and the output of the previous BNS module is used as the input of the next BNS module. The feature map extracted by each BNS module is fused with the other group of branch features after splitting in the connection layer. The fused feature map passes through the channel rearrangement algorithm and then passes through the second CBS module for output.

[0090] YOLOv8 uses the same Bottleneck block in all stages, but deep stages and large models may show more redundancy. Therefore, simply applying the same Bottleneck module to all stages is not the best capacity-efficiency trade-off strategy. To solve this problem, following the lightweight principle proposed in ShuffleNet V2, a new Bottleneck (BNS) design scheme is proposed, such as Figure 7 As shown in the figure, this solution splits the channel into two parts. One part uses cheap depthwise convolution for spatial mixing and cost-effective point-by-point convolution for channel mixing. The two branches are then spliced ​​and the channels are rearranged so that the BNS module can obtain more complete feature information when connected in series. While ensuring the effect of extracting important features, the number of parameters of the entire network is reduced by half. The specific processing process of the BNS module includes the following steps:

[0091] The input feature map is split into two groups of parallel branches along the channel dimension through the Split operation. The split group of branches first undergoes channel dimension transformation through a 1×1 point-by-point convolution module, and then undergoes spatial feature extraction through a 3×3 depth-wise convolution module. The channel dimension is then adjusted again through a 1×1 point-by-point convolution module. The adjusted feature map is concatenated with the other group of split branches in the connection layer and used as the input of the channel rearrangement algorithm to re-interleave the output channels of different branches.

[0092] 3. Training of bridge apparent defect detection model

[0093] In the present invention, the bridge apparent disease detection model is trained in the following way:

[0094] The learning rate and hyperparameters of the bridge apparent disease detection model are set, and the bridge apparent disease detection model with pre-completed bridge area segmentation and disease labeling is used as training samples to form a training sample set. The bridge apparent disease detection model is input, and the number of iterations is set to train the bridge apparent disease detection model. The CIoU loss function is used to calculate the training loss of the plant disease detection model. The parameters of the bridge apparent disease detection model are updated with the goal of minimizing the CIoU loss function to obtain the trained bridge apparent disease detection model. Finally, the bridge apparent disease detection model is tested based on the training sample set.

[0095] In specific applications, the training sample set can be constructed using a combination of publicly available datasets and samples from a dataset of bridge surface defects to ensure the comprehensiveness and usability of the training sample set. Furthermore, the training sample set can be divided into training and test sets for training the bridge surface defect detection model and for post-training testing and verification.

[0096] Specifically, the CIoU loss function used in training the bridge apparent defect detection model can be expressed as:

[0097]

[0098] Among them, L CIoU represents CIoU loss; ρ(b,b gt ) represents the center point b of the model predicted disease area positioning frame and the center point b of the real disease area positioning frame gt c represents the diagonal length of the minimum bounding rectangle covering the predicted diseased area positioning frame and the true diseased area positioning frame; α is the preset weight coefficient; v is the consistency parameter used to measure the difference in aspect ratio between the predicted diseased area positioning frame and the true diseased area positioning frame, and w and h are the width and height of the predicted disease area positioning box, respectively. gt 、h gt are the width and height of the predicted disease area positioning frame respectively; IoU represents the ratio of the intersection area to the union area of ​​the predicted disease area positioning frame and the true disease area positioning frame, that is:

[0099]

[0100] Among them, B, B gt They represent the predicted disease area positioning frame and the real disease area positioning frame respectively, and Area(·) represents area operation.

[0101] The CIoU loss function is used to evaluate the model's disease location detection results, which more comprehensively considers multiple dimensions such as overlapping area, center point distance, and aspect ratio consistency. It solves the gradient vanishing problem of the traditional IoU loss function in non-overlapping or inconsistent proportions. By setting the weight coefficient α and the aspect ratio consistency parameter v, it also helps to accelerate the training convergence speed of the model and improve the recognition effect and positioning accuracy of small disease target areas.

[0102] 4. Experimental testing and verification

[0103] In the specific application of this implementation, the concrete bridge apparent disease detection model was built using the pytorch tool and carried out under the operating system Windows 11. The GPU was NVIDIA RTX 3060, the memory was 12GB, the deep learning framework was Pytorch1.11.0, and the software programming environment was Python3.7.7, Cuda11.2, and Conda4.8.3.

[0104] The training parameters were set as follows: 200 epochs, 16 batch sizes, 640×640 input images, SGD optimizer, 0.001 initial learning rate, and 0.8 momentum. The dataset was divided into training and test sets. The weight file with the lowest loss on the test set was saved and used to predict the apparent defects of concrete bridges.

[0105] The trained bridge surface defect detection model was used to detect surface defects on concrete bridges. The weighted file with the lowest loss was used to test the surface defects of actual concrete bridges. The concrete bridge surface defect image was first divided into multiple 640×640 images, and predictions were performed on each image before being combined into a single image.

[0106] By designing a wavelet-based multi-band channel attention module (WMA), a multi-band channel attention mechanism is developed that focuses on different channels at varying levels across different frequency bands. This generalizes existing channel attention to the wavelet band, addressing information loss across frequency bands and improving the object detection network's ability to extract feature information from various bridge surface defects. Furthermore, the channel-reordering-based C2f_BNS lightweight feature extraction module combines the advantages of point-by-point convolution and depthwise separable convolution with a channel-reordering algorithm, effectively minimizing channel information loss and enhancing feature extraction capabilities. Furthermore, it significantly reduces the number of network parameters.

[0107] In order for those skilled in the art to have a more intuitive understanding of the effect of this method, the following verification experiment is described.

[0108] 4.1 Dataset

[0109] CODEBRIM: The CODEBRIM dataset provides 1,590 images of bridge surface defects, including five categories: cracks, rust, exposed reinforcement, salt corrosion, and spalling. However, some surface defects have a small number of instances, are unevenly distributed, and are sometimes mislabeled. Therefore, 1,320 suitable images were selected and relabeled.

[0110] dacl10k: The dacl0k dataset provides 9,920 images of concrete bridge surface defects. The dataset contains 19 categories of concrete bridge surface defects and includes annotations for segmentation. 3,230 suitable images were selected and fine-grained re-annotation was performed.

[0111] I myself created 1,450 images of common and serious apparent defects on concrete bridges. Most of them had complex backgrounds and belonged to the same category as the images selected from the first two datasets.

[0112] CCBAD: To avoid the inability to obtain the optimal model due to dataset defects, we screened the dataset after data augmentation and annotated all images in a more detailed manner, obtaining categories with more than 7,500 instances. Ultimately, we obtained 7,243 images, covering four common and serious concrete bridge apparent defects: rust, exposed rebar, salt corrosion, and spalling. A total of 40,065 labels were accumulated and named the Chongqing Concrete Bridge Apparent Defect Dataset (CCBAD).

[0113] 4.2 Comparison Methods and Evaluation Indicators

[0114] This example is compared with six publicly available algorithm models: YOLOv3, YOLOv5, YOLOv7, YOLOv8, YOLOv9, YOLOv10, and YOLOv11. The comparison methods for all networks are performed under the same experimental conditions, with a training set batch size of 16, a test set batch size of 16, and an input image size of 640×640.

[0115] In order to objectively evaluate the performance of the improved network model, the accuracy P (precision), recall R (recall) and average accuracy mAP are used. 50In order to evaluate the lightweight effect of the proposed C2f_BNS, the number of parameters is used as an additional evaluation indicator, and the performance of the model is evaluated under different values ​​of the grouping number g.

[0116] The wavelet-based multi-band channel attention module (WMA) in this embodiment was added to the backbone network and the performance of six comparison models was evaluated. Taking YOLOv8 as an example, the performance of the WMA and C2f_BNS modules was compared with other mainstream detection models. In other types of comparative experiments, YOLOv8 was used as the base model for verification and comparison. The specific experimental results are as follows.

[0117] The effect of WMA under different frequency band components

[0118] The embodiment compares the disease detection performance differences between the original YOLOv8 network and the improved YOLOv8 network (YOLOv8+WMA) with the wavelet multi-frequency channel attention mechanism that selects different frequency band components. The results of the embodiment are shown in Table 1. LL ), the performance of the network for disease detection has not been greatly improved, and the network's mAP50 has only increased by 0.7%; the feature map generated by the combination of high-frequency components (Y LH , Y HL , Y HH ), the performance of the network has been improved to a certain extent, and the network's mAP50 has increased by 1.9%. When the feature map generated by combining all the generated frequency domain components (Y LL , Y LH , Y HL , Y HH , Y MIX ), it has excellent performance in disease detection, and the mAP50 of the overall network is improved by 2.4%.

[0119] Table 1 Comparison of the detection performance of the original YOLOv8 network and the bridge apparent disease detection model of this embodiment

[0120]

[0121] Effects of WMA on different detection models

[0122] In order to objectively evaluate the practicality of the wavelet multi-band channel attention mechanism on the current mainstream target detection network, this example will use YOLOv3, YOLOv5, YOLOv7, YOLOv8, YOLOv9 and YOLOv10 to detect bridge surface defects on the self-made CCBAD dataset, and add the wavelet multi-band channel attention mechanism to the backbone network of the above network model. The experimental results are compared in Table 2.

[0123] Table 2 Bridge surface defect detection using six public algorithm models on the self-made CCBAD dataset

[0124]

[0125] Effect of C2f_BNS under different number of groups

[0126] The embodiment is based on the original YOLOv8 network and introduces C2f_BNS modules with different numbers of groups, where g represents different numbers of groups. The most suitable number of channel groupings is determined by comparing the differences in the detection performance of apparent defects of concrete bridges. The results are shown in Table 3. When g = 2, the mAP50 of the network model is improved by 0.8%, and the detection accuracy of each defect is improved to a certain extent. When g = 4, the mAP50 of the network model has the greatest improvement, which is an increase of 1.4%. For the above analysis, comparing the improved accuracy of defect detection, the number of groups g = 4 is selected, which achieves better performance in the detection of apparent defects of concrete bridges than the original YOLOv8 network, with an increase of 1.4% in the mAP50 value.

[0127] Table 3 Effect of C2f_BNS under different number of groups

[0128]

[0129] 4.3 Ablation Experiment

[0130] The embodiment introduces the designed multi-band channel attention module WMA based on wavelet transform into the original YOLOv8 network in the backbone network part, selects the feature map (Y LL , Y LH , Y HL , Y HH ) to enhance the overall network's extraction of important feature information in the frequency domain; an improved C2f module (C2f_BNS), based on the proposed BNS module, was incorporated, and experiments were conducted using g = 4. The results are shown in Table 4. Ablation experiments show that using either module improves the localization accuracy of concrete bridge surface defect detection, and using the C2f_BNS module significantly reduces the number of parameters in the entire detection network.

[0131] Table 4 Comparison of ablation test results of different module combinations

[0132]

[0133] 4.4 Comparative Experiment

[0134] In order to verify the performance of the proposed network model, various mainstream network models are compared on the self-made CCBAD dataset. The results are shown in Table 5. It can be seen from the table that the proposed network model has a higher mAP 50 .in Figure 8 The curve chart of the accuracy increase after 200 rounds of training for each model can more intuitively see the mAP of the proposed model 50 It is significantly higher than other models. Since the apparent defects of concrete bridges have a relatively complex background, various apparent defects are mixed and overlapped, and it is difficult to extract important feature information when using the YOLOv8 network for feature extraction. Therefore, the wavelet multi-frequency domain channel attention mechanism is added to make the network model pay attention to the channels containing important features from the frequency domain perspective, thereby extracting rich feature information. In addition, the C2f_BNS module is incorporated to enable each layer of the network to work together effectively, enhancing the model's ability to understand and represent input data. At the same time, the detection results of various mainstream network models on the self-made CCBAD dataset are displayed through intuitive visualization results. The test results are as follows Figure 9 As shown. Figure 8 The visualization results mainly include the detection box position, confidence value and different colors.

[0135] Table 5 Performance comparison of mainstream detection models on CCBAD dataset

[0136]

[0137] 5. Overview

[0138] The present invention proposes a bridge apparent defect detection model used in a bridge apparent defect detection method based on wavelet multi-band channel attention. The proposed wavelet multi-band channel attention module (WMA) introduces multi-band information and a channel attention mechanism of wavelet transform, obtains a multi-band channel attention mechanism with different degrees of attention to each channel from different frequency bands, generalizes the existing channel attention in the wavelet band, solves the problem of information loss in different frequency bands, and improves the target detection network's ability to extract feature information of different bridge apparent defects. Specifically, the module uses the Haar wavelet basis to decompose the input feature map to obtain four feature sub-bands: LL (low-frequency component), LH (horizontal high-frequency component), HL (vertical high-frequency component), and HH (diagonal high-frequency component), and reconstructs the features of each sub-band through inverse wavelet transform, thereby completely retaining the feature information of defects in different directions and scales. In terms of attention mechanism design, the WMA performs global spatial compression on the features of each frequency band to generate a channel descriptor, dynamically learns the correlation between frequency bands through the fully connected layer, and uses sigmoid activation to realize soft attention weight distribution, so that the model can adaptively focus on important features of different frequency bands.

[0139] In addition, the C2f_BNS lightweight feature extraction module proposed in the bridge apparent defect detection model of the present invention first combines the dual advantages of point-by-point convolution and depth-wise separable convolution. Point-by-point convolution is responsible for the fusion of feature information across channels, and realizes efficient channel dimension transformation through 1×1 convolution kernel; while depth-wise separable convolution focuses on spatial feature extraction within a single channel, using 3×3 convolution kernel to capture local features while reducing computational complexity; secondly, it integrates the channel rearrangement algorithm, which breaks the information isolation problem caused by traditional grouped convolution through periodic channel permutation operations, enhances the feature extraction capability, and thus improves the lightweight and reliability of the detection system.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described with reference to the preferred embodiments of the present invention, it should be understood by those skilled in the art that various changes can be made in form and details without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A bridge apparent disease detection method based on wavelet multi-band channel attention, characterized in that: Obtaining an apparent defect image of a bridge to be detected, inputting the image into a pre-trained apparent defect detection model for the bridge, and obtaining a defect location detection result of the apparent defect image of the bridge to be detected; The bridge apparent defect detection model includes a backbone network, a neck network and a detection head network. The bridge apparent defect image is used as the input of the backbone network to extract multi-level semantic features to capture defect information from low-level texture to high-level semantics. The extracted multi-level semantic features are then fused through the neck network to obtain multi-scale fusion features and input into the detection head network. Finally, the defect location detection result of the bridge apparent defect image is obtained as the output of the bridge apparent defect detection model.

2. The bridge apparent defect detection method based on wavelet multi-band channel attention according to claim 1 is characterized in that: The backbone network includes a CBS module, three wavelet multi-frequency lightweight fusion units and a spatial pyramid pooling module connected in sequence; the wavelet multi-frequency lightweight fusion unit includes a cascaded CBS module and a C2f_BNS lightweight feature extraction module, wherein a wavelet multi-band channel attention module is inserted into each of the wavelet multi-frequency lightweight fusion units; the CBS module includes a cascaded convolutional layer, a BN layer and a SiLU activation function.

3. The bridge apparent defect detection method based on wavelet multi-band channel attention according to claim 2 is characterized in that: The processing process of the wavelet multi-band channel attention module is shown in the following formula: WMA(X)=X*sigmoid(fc(F sg (Z g ))); F sq (From g )=concat(Z LL ,Z LH ,Z HL ,Z HH ,Z MIX ); Where WMAX(X) represents the output of the wavelet multi-band channel attention module, X represents the input feature map, * represents channel-by-channel multiplication, sigmoid(·) represents the activation function, fc(·) represents the fully connected layer, and F sq (·) represents the feature vector after multi-band compression, concat(·) represents concatenation along the channel dimension, Z LL ,Z LH ,Z HL ,Z HH They represent the compression features of the low-frequency, horizontal high-frequency, vertical high-frequency and diagonal high-frequency sub-bands after the input feature map X is decomposed by wavelet, and Z MIX Represents the compression characteristics of the mixed frequency band, Z g represents the global channel statistics after the compression of each frequency band feature, Compress(·) represents the global spatial compression, and Y g It represents the frequency band feature map reconstructed after inverse wavelet transform, and H and W represent the height and width of the feature map respectively.

4. The bridge apparent defect detection method based on wavelet multi-band channel attention according to claim 2 is characterized in that: The C2f_BNS lightweight feature extraction module is constructed based on channel rearrangement and the BNS module using depthwise separable convolution and pointwise convolution. The specific processing process includes the following steps: The input feature map is subjected to feature extraction by the first CBS module, and the extracted feature map is split into two groups of parallel branches along the channel dimension through the Split operation. The split group of branch feature maps enters N BNS modules in turn through depth-wise separable convolution and point-by-point convolution for feature extraction, and the output of the previous BNS module is used as the input of the next BNS module. The feature map extracted by each BNS module is fused with the other group of branch features after splitting in the connection layer. The fused feature map passes through the channel rearrangement algorithm and then passes through the second CBS module for output.

5. The bridge apparent defect detection method based on wavelet multi-band channel attention according to claim 4 is characterized in that: The processing process of the BNS module includes the following steps: The input feature map is split into two groups of parallel branches along the channel dimension through the Split operation. The split group of branches first undergoes channel dimension transformation through a 1×1 point-by-point convolution module, and then undergoes spatial feature extraction through a 3×3 depth-wise convolution module. The channel dimension is then adjusted again through a 1×1 point-by-point convolution module. The adjusted feature map is concatenated with the other group of split branches in the connection layer and used as the input of the channel rearrangement algorithm to re-interleave the output channels of different branches.

6. The bridge apparent defect detection method based on wavelet multi-band channel attention according to claim 2 is characterized in that: The neck network includes two upsampling lightweight feature fusion units and two convolutional feature reorganization units connected in sequence; the upsampling lightweight feature fusion unit includes a cascaded upsampling module, a connection layer and a C2f_BNS lightweight feature extraction module, and the convolutional feature reorganization unit includes a cascaded CBS module, a connection layer and a C2f_BNS lightweight feature extraction module; Among them, the input of the bridge apparent disease detection model serves as the input of the backbone network; the output of the second wavelet multi-frequency lightweight fusion unit and the third wavelet multi-frequency lightweight fusion unit in the backbone network are also used as the input of the connection layer in the first upsampling lightweight feature fusion unit and the connection layer in the second upsampling lightweight feature fusion unit in the neck network respectively; the output of the spatial pyramid pooling module in the backbone network is also used as the input of the connection layer in the second convolutional feature recombination unit in the neck network; the output of the first upsampling lightweight feature fusion unit in the neck network is also used as the input of the connection layer in the first convolutional feature recombination unit.

7. The bridge apparent defect detection method based on wavelet multi-band channel attention according to claim 2 is characterized in that: The detection head network includes three detection units connected in sequence, and the three detection units are connected in parallel, wherein each detection unit includes a bounding box prediction module and a category prediction module connected in parallel, the bounding box prediction module includes a cascaded CBS module, a two-dimensional convolutional layer and a bounding box loss function, and the category prediction module includes a cascaded CBS module, a two-dimensional convolutional layer and a category loss function; Among them, the outputs of the second upsampling lightweight feature fusion unit and the first convolutional feature reorganization unit in the neck network are respectively used as the inputs of the first detection unit and the second detection unit in the detection head network, and the output of the second convolutional feature reorganization unit in the neck network is used as the input of the third detection unit in the detection head network.

8. The bridge apparent defect detection method based on wavelet multi-band channel attention according to claim 1 is characterized in that: The training process of the bridge apparent defect detection model includes: The learning rate and hyperparameters of the bridge apparent disease detection model are set, and the bridge apparent disease detection model with pre-completed bridge area segmentation and disease labeling is used as training samples to form a training sample set. The bridge apparent disease detection model is input, and the number of iterations is set to train the bridge apparent disease detection model. The CIoU loss function is used to calculate the training loss of the plant disease detection model. The parameters of the bridge apparent disease detection model are updated with the goal of minimizing the CIoU loss function to obtain the trained bridge apparent disease detection model. Finally, the bridge apparent disease detection model is tested based on the training sample set.

9. The bridge apparent defect detection method based on wavelet multi-band channel attention according to claim 8 is characterized in that: The training sample set is constructed by using a sample set of a public data set and an acquired bridge apparent disease data set.

10. The bridge apparent defect detection method based on wavelet multi-band channel attention according to claim 8 is characterized in that: The CIoU loss function is expressed as: Among them, L CIoU represents CIoU loss; ρ(b,b gt ) represents the center point b of the diseased area positioning frame predicted by the model and the center point b of the real diseased area positioning frame gt c represents the diagonal length of the minimum bounding rectangle covering the predicted diseased area positioning frame and the true diseased area positioning frame; α is the preset weight coefficient; v is the consistency parameter used to measure the difference in aspect ratio between the predicted diseased area positioning frame and the true diseased area positioning frame, and w and h are the width and height of the predicted disease area positioning box, respectively. gt 、h gt are the width and height of the predicted disease area positioning frame respectively; IoU represents the ratio of the intersection area to the union area of ​​the predicted disease area positioning frame and the true disease area positioning frame, that is: Among them, B, B gt They represent the predicted disease area positioning frame and the real disease area positioning frame respectively, and Area(·) represents area operation.

Images