Bridge apparent subtle damage identification method based on multi-scale feature single sequence fusion algorithm

Abstract

The present invention discloses a method for identifying subtle bridge damage based on a multi-scale feature single-sequence fusion algorithm. The method comprises a multi-scale feature fusion model, CITFPN, RPN, RoI Align, and a network head architecture that outputs prediction results. The CITFPN includes a feature extraction network that incorporates an attention module and a single-sequence feature fusion module, IFPN, that incorporates transposed convolution. The IFPN is used to perform a one-time single-sequence superposition fusion of the high-semantic feature map generated by the last convolution stage of the feature extraction network with the feature maps generated by the remaining convolution stages (excluding the first) of the feature extraction network. The fused feature map is then further convolved. Simultaneously, the high-semantic feature map is downsampled and further fused with the other feature maps after further convolution processing to serve as the output of the CITFPN. The present invention enables automated and refined segmentation of bridge damage instances.

Description

Technical Field

[0001] The present invention relates to the field of bridge structure appearance information detection, and in particular to a method for identifying bridge appearance subtle damage based on a multi-scale feature single sequence fusion algorithm. Background Art

[0002] In recent years, many researchers at home and abroad have conducted research on automatic surface damage identification methods based on deep learning, and have achieved good results in the detection of various defects such as cracks, spalling, and holes. However, existing bridge inspection technologies still have many problems to be solved when it comes to identifying and quantifying bridge defects in complex real-world scenarios:

[0003] (1) In actual engineering applications, due to the special features of bridge structures and the safety requirements of shooting equipment, the number of pixels occupied by the defect targets in drone images is often small, and there is a significant imbalance between the number of pixels occupied by the defect targets and the background. In addition, most images have complex backgrounds and contain a lot of noise. Currently, instance segmentation models for bridge defects are mostly trained based on crack datasets collected in laboratory environments or magnified defect images. These datasets have a large proportion of defect pixels and good image quality, which is significantly different from the actual situation. As a result, existing models are difficult to directly apply to small target recognition tasks in actual engineering, and their application is limited.

[0004] (2) The accuracy of identifying apparent minor damage needs to be improved.

[0005] (3) Existing identification methods involve a single type of disease, most of which focus on typical diseases such as cracks, and rarely identify multiple diseases simultaneously. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, this paper proposes a bridge apparent subtle damage identification method based on a multi-scale feature single sequence fusion algorithm. The specific technical solution is as follows:

[0007] A bridge apparent subtle damage identification method based on a multi-scale feature single sequence fusion algorithm includes the following steps:

[0008] S1: Establish a bridge apparent subtle damage dataset;

[0009] S2: Build and train a bridge apparent subtle damage detection model; the bridge apparent subtle damage detection model includes a multi-scale feature fusion model CITFPN, RPN, RoI Align, and a network head architecture that outputs prediction results; the multi-scale feature fusion model includes a feature extraction network that introduces an attention module and a single-sequence feature fusion module IFPN that introduces transposed convolution;

[0010] The multi-scale feature fusion model is used to perform a one-time single-sequence superposition fusion of the high-semantic feature map generated by the last convolution stage of the feature extraction network and the feature maps generated by the remaining convolution stages except the first convolution stage output by the feature extraction network, and then further convolution processing is performed on the fused fusion feature map; at the same time, the high-semantic feature map is downsampled; then, all the feature maps after the further convolution processing and the downsampled high-semantic feature map are fused again as the output of the multi-scale feature fusion model; the output of the multi-scale feature fusion model is used as the input of the RPN, and a fixed-size feature region is generated through the RPN and RoI Align, and the corresponding mask, bounding box and category label are output through the network head architecture that outputs the prediction result;

[0011] S3: Use the trained bridge apparent subtle damage detection model to automatically identify bridge apparent subtle damage.

[0012] Furthermore, the feature extraction network is Resnet101, which includes five convolution stages C1, C2, C3, C4, and C5. The single-sequence feature fusion module IFPN upsamples the feature map p5 generated in the C5 stage, and performs a one-time single-sequence superposition fusion with the feature maps generated by C2, C3, and C4 to obtain p2, p3, and p4, and then performs 3×3 convolution processing on p2, p3, p4, and p5 to obtain P2, P3, P4, and P5; finally, P5 is downsampled to obtain P6, and P2, P3, P4, P5, and P6 are fused again as the output of the multi-scale feature fusion model.

[0013] Furthermore, an attention module is built into the identity block and convolution block in the feature extraction network Resnet101. The attention module includes a channel attention submodule and a spatial attention submodule, which compress and weight features in the channel dimension and spatial dimension respectively, so as to enhance detail features and suppress noise.

[0014] Furthermore, the specific steps of the attention module to compress and weight features in the channel dimension and spatial dimension are as follows:

[0015] (1) After the input feature F1 is compressed by the average pooling layer and the maximum pooling layer, it passes through two layers of perceptrons to obtain two independent channel features. The two channel features are further added element by element, and the channel attention weight Mc is obtained through the convolution layer with Sigmoid activation function;

[0016] (2) Multiplying the channel attention weight Mc by the input feature F1 to obtain the channel attention feature F2, inputting the channel attention feature F2 into the average pooling layer and the maximum pooling layer in series, and then passing through the convolution layer with the Sigmoid activation function to obtain the spatial attention weight Ms;

[0017] (3) Multiply the spatial attention weight Ms by the channel attention feature F2 to obtain the final attention enhancement feature F3.

[0018] Furthermore, the single-sequence feature fusion module IFPN upsamples the feature map p5 generated in the C5 stage using transposed convolution.

[0019] Furthermore, when training the bridge apparent subtle damage detection model, the transfer learning method is first used to transfer the training weights of the COCO dataset to initialize the model; then, a staged training method is adopted, that is, first, while freezing the other layers, the parameters of the model head layer are adjusted through training, and then all layers are trained while unfreezing the other layers.

[0020] A bridge apparent subtle damage identification system based on a multi-scale feature single sequence fusion algorithm includes one or more processors for implementing a bridge apparent subtle damage identification method based on a multi-scale feature single sequence fusion algorithm.

[0021] An electronic device, comprising:

[0022] one or more processors;

[0023] A storage device is used to store one or more programs. When the one or more programs are executed by the electronic device, the electronic device implements a method for identifying apparent subtle damage to bridges based on a multi-scale feature single-sequence fusion algorithm.

[0024] A computer-readable storage medium stores a program, which, when executed by a processor, implements a bridge apparent subtle damage identification method based on a multi-scale feature single-sequence fusion algorithm.

[0025] The present invention has the following beneficial effects:

[0026] (1) The bridge apparent subtle damage identification method based on the multi-scale feature single sequence fusion algorithm of the present invention adopts a single fusion feature strategy to reduce the network complexity while enhancing its processing ability of the disease detail information, thereby improving the continuity and accuracy of the pixel-level identification of bridge diseases, and is more suitable for the identification of minor defects in field bridge inspections.

[0027] (2) In the bridge apparent subtle damage detection model of the present invention, a parameter-learnable transposed convolution is introduced into the multi-scale feature fusion model as an upsampling method for expanding the feature map size, so that the upsampling process is continuously optimized, the texture and edges of the image are effectively reconstructed, and a clearer and more continuous image is obtained, thereby providing better accuracy and continuity for the bridge disease segmentation task.

[0028] (3) In order to reduce the impact of background noise on the accuracy of small target recognition, the multi-scale feature fusion model introduces a multiple attention mechanism to suppress useless information such as cluttered background and noise, enhance the local feature processing capability, and select a strategy with higher accuracy and lower model complexity in the selection of connection strategy. It has better performance in the segmentation of apparent subtle damage of bridges in complex environments.

[0029] (4) The bridge apparent subtle damage identification method based on the multi-scale feature single sequence fusion algorithm of the present invention can perform image segmentation on images with large scale, unbalanced categories and large background noise taken during on-site bridge inspections. It has high robustness and overcomes the problems of large defect size and small background noise in traditional data set images. It is convenient for high-precision and high-efficiency identification to be carried out by equipment such as drones.

[0030] (5) The bridge apparent subtle damage identification method based on the multi-scale feature single sequence fusion algorithm of the present invention realizes multi-category pixel-level identification of bridge apparent subtle damage for small targets and complex backgrounds in field bridge inspections, effectively meets the subtle damage segmentation requirements of small targets in complex backgrounds, obtains the contour, position, area and other information of subtle damage, contributes to the automated and refined detection of bridge diseases, and provides a more reliable, objective and comprehensive basis for bridge health monitoring and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of a method for identifying apparent subtle damage to bridges based on a multi-scale feature single-sequence fusion algorithm according to an embodiment of the present invention.

[0032] Figure 2 This is the network structure diagram of CITNET, a bridge apparent subtle damage detection model.

[0033] Figure 3 This is the MS Defect dataset provided by the embodiment of the present invention.

[0034] Figure 4 An example diagram of typical defects identified by an embodiment of the present invention.

[0035] Figure 5 Comparison chart of recognition results for different CBAM connection strategies. DETAILED DESCRIPTION

[0036] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become more apparent. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] Technical term explanation

[0038] ResNet, Residual Neural Network, residual network;

[0039] FPN, Feature Pyramid Network, feature pyramid network;

[0040] CBAM, Convolutional Block Attention Module, convolutional block attention module;

[0041] RPN, Region Proposal Network, region proposal network;

[0042] RoI Align, Region of Interest Align;

[0043] CA, Channel Attention, channel attention;

[0044] SA, Spatial Attention, spatial attention;

[0045] MLP, Multi-Layer Perceptron, multi-layer perceptron;

[0046] Identity Block, identity block;

[0047] Conv Block, Convolutional Block, convolution block;

[0048] FCN, Fully Convolutional Network, full convolutional network;

[0049] Dense Layers, Dense Layers, fully connected layers;

[0050] Adam, Adaptive Moment Estimation, adaptive moment estimation;

[0051] SGD, Stochastic Gradient Descent, stochastic gradient descent;

[0052] IoU, the Intersection over Union, intersection over union ratio;

[0053] P, Precision, accuracy;

[0054] R, Recall, recall rate;

[0055] F1 Score, F1 Score, F1 score;

[0056] mAP, mean Average Precision, average precision;

[0057] like Figure 1 As shown, the bridge apparent subtle damage identification method based on the multi-scale feature single sequence fusion algorithm of this embodiment includes the following steps:

[0058] S1: Establish a bridge apparent subtle damage dataset.

[0059] Images of subtle bridge surface damage are collected using drone-mounted cameras and other imaging equipment. The MS Defect dataset is constructed based on raw data from photos taken during on-site bridge inspections. Each image contains at least one subtle defect, and the entire bridge surface subtle damage dataset should include a variety of subtle defects, including slender tubular defects like cracks and exposed rebar, as well as other common defects like pockmarks and holes. Each subtle defect category should contain a sufficient number of images to meet model training requirements, for example, at least 100 images.

[0060] We then performed data augmentation on the images in the MS Defect dataset, including rotation, translation, shearing, scaling, and horizontal flipping. The dataset was divided into non-overlapping training and validation datasets. Each dataset used the original image as input and the annotated mask image as the prediction comparison value for testing and training network performance.

[0061] S2: Build and train a bridge surface subtle damage detection model.

[0062] like Figure 2As shown in Figure 1, the CITNet model for bridge surface subtle damage detection comprises a multi-scale feature fusion model, CITFPN, RPN, RoI Align, and a network head architecture that outputs prediction results. The multi-scale feature fusion model includes a feature extraction network with an attention module and a single-sequence feature fusion module, IFPN, which incorporates transposed convolution. The multi-attention mechanism and feature fusion effectively alleviate the complex background and class imbalance issues in bridge defect identification. Introducing learnable transposed convolution in the single-sequence feature fusion module as an upsampling method to increase the size of the feature map, the upsampling process is continuously optimized, resulting in clearer and more continuous images, thus improving the accuracy and continuity of bridge defect segmentation.

[0063] (1) Feature extraction network introducing attention module

[0064] Taking the feature extraction network Resnet101 as an example, it contains five convolution stages C1, C2, C3, C4, and C5. In the process of disease identification and segmentation, the bridge surface image data is first input into the feature extraction network. Resnet is a residual network structure that optimizes image processing performance through a deep network architecture while ensuring that the performance is at least equal to that of a shallow network. In order to solve the problem of performance degradation of deep networks, a shortcut connection of identity mapping between blocks is established. Specifically, each block accepts an input feature x, and the feature objective function H(x)=F(x)+x that originally needs to be learned by the layer, F(x) represents the residual part that the network block needs to learn, and x is the input feature directly added to the output through a shortcut connection. Then:

[0065] No. Layer output: , for Layer input;

[0066] No. +1 layer output: ;

[0067] Output of layer L: ;

[0068] Use the chain rule to calculate the gradient back propagation:

[0069]

[0070] The gradient of each layer is transmitted not only through the residual function F(x) but also directly through shortcut connections of the identity mapping. This structural design ensures that the overall gradient does not vanish in deep networks, avoiding the problem of gradient loss of shallow parameters caused by the chain rule. Furthermore, when adding network layers fails to significantly improve performance, the residual F(x) tends to 0. At this point, the output feature y is the same as the input feature x, equivalent to not adding new layers to the network, thus ensuring that network performance does not degrade.

[0071] The feature maps generated at each level in the residual network (ResNet101) often contain rich, even redundant, information. While this ensures a comprehensive understanding of the input data, it can also generate excessive parameters, leading to redundant feature information and wasted resources. To ensure focus on key features while enhancing detailed features and suppressing background noise, the present invention introduces a CBAM module within the ResNet101 CITFPN module to improve sensitivity to specific features. CBAM consists of a channel attention module (CA) and a spatial attention module (SA), which compress and weight features in the channel and spatial dimensions, respectively. The input feature F1 is compressed through average and max pooling layers and then passed through a two-layer perceptron (MLP) to obtain two independent channel feature representations. These two channel features are then element-wise added and passed through a convolutional layer with a sigmoid activation function to obtain the channel attention weights Mc. Mc helps the model assess the importance of different channels, emphasizing channels that are helpful for the task and suppressing irrelevant channels. The weights Mc are multiplied by the initial input feature F1 to produce the channel attention features F2. The channel-wise attention feature F2 is input into a series of average and max pooling layers, followed by a convolutional layer with a sigmoid activation function to obtain spatial attention weights Ms. Ms helps highlight important image regions and reduce the influence of unimportant areas such as background noise. The weights Ms are multiplied by the channel-wise attention feature F2 to obtain the final attention-enhanced feature F3. To improve sensitivity to important features at different stages of feature map generation, CBAM is incorporated into the identity block and conv block of ResNet. In the identity block, CBAM applies more attention to the output of the feature map after skip connections, thereby emphasizing or suppressing certain features. In the conv block, CBAM adjusts attention during depth transformations of the feature map, resulting in a more focused feature learned by the model. With the addition of CBAM, the network allocates limited computational resources to more important features during training, enhancing sensitivity to useful features and suppressing sensitivity to unimportant ones, thereby improving model efficiency and accuracy.

[0072] In addition, since the newly designed feature map connection and fusion method can realize the direct fusion of deep and shallow feature information, it avoids the problem of shallow feature information not being significant in the traditional multi-level fusion method, and thus improves the recognition accuracy of small disease details. Taking C2 and C5 as examples, C2 is the second convolution stage that the image passes through. The resulting feature map is closer to the original image, has a higher resolution, contains more detail information, and less semantic information; C5 is the fifth convolution stage that the image passes through. Due to multiple convolution and pooling operations, the resolution is lower and the semantic information is more enriched, but the detail information may be lost. Since the size and number of channels of the feature map at each stage are different, all feature maps must undergo a 1×1,256 channel convolution to obtain the same size and channel initial feature map p i (i=2~5). The traditional FPN uses a layer-by-layer fusion method to perform the convolution on each convolution stage C. i The generated feature maps are integrated. Specifically, before the feature map of p2 is fused with p3, p3 itself has already integrated the high-level feature information from p4 and p5. In this process, the low-level features are gradually dominated by the high-level features, resulting in the suppression of the original detailed information. To solve this problem, IFPN changes the fusion strategy based on the FPN network architecture to:

[0073] like Figure 2 As shown in the figure, Resnet101 uses continuous convolution and pooling to extract multi-level features from the input image, obtaining a series of multi-scale feature maps with decreasing resolution and increasing semantic information. The single-sequence feature fusion module (IFPN) upsamples the feature map p5 generated in stage C5 and then performs a one-time, single-sequence superposition fusion with the feature maps generated in stages C2, C3, and C4 to generate p2, p3, and p4. This single-shot fusion method effectively reduces the number of repeated fusions between low-level and high-level features, thereby reducing the risk of low-level details being overwhelmed by high-level features during the fusion process. By increasing the proportion of shallow-level features in the final fused feature map, IFPN can more accurately capture small objects and details in the image, helping to improve the performance of fine-grained object detection. Furthermore, this one-shot fusion strategy helps simplify the network structure and reduce the computational burden, enabling IFPN to achieve higher efficiency in practical applications while maintaining the advantages of feature fusion. p2, p3, p4, and p5 are then subjected to a 3×3 convolution to obtain P2, P3, P4, and P5, mitigating the aliasing effects of upsampling. Finally, after downsampling P5, a reference feature map P6 is generated, which is more conducive to generating prediction results. P2, P3, P4, P5, and P6 are fused again as the output of the multi-scale feature fusion model to obtain the final feature map used by RPN for target classification and boundary regression prediction.

[0074] Finally, in order to enhance detail restoration and image reconstruction in the image segmentation process, the single-sequence feature fusion module IFPN of the present invention introduces transposed convolution as an upsampling method when upsampling the feature map p5 generated in the C5 stage. The upsampling weights are adaptively optimized through the training process, and the texture and edges of the image are effectively reconstructed to obtain clearer and continuous images, generate richer feature representations, and avoid the edge blurring that may be caused by conventional upsampling, thereby providing better accuracy and continuity for the bridge disease segmentation task.

[0075] The extracted feature map is further input into RPN and RoI Align to generate fixed-size feature regions, and the corresponding masks, bounding boxes and category labels are output using full convolutional layers and fully connected layers.

[0076] When training the CITNet bridge surface subtle damage detection model, we employed a reasonable optimization method to accelerate model convergence. Adam was used as the training optimizer, and stochastic gradient descent (SGD) was used for model training.

[0077] The model has three prediction tasks, and its loss function can be expressed as:

[0078]

[0079] in, represents the classification loss value, Represents the regression loss value of the bounding box; Indicates the loss value of the masked part.

[0080] To improve model training efficiency, the bridge surface subtle damage detection model is trained using a transfer learning method to transfer the training weights of the COCO dataset to initialize the model. Next, a phased training approach is employed, first adjusting the parameters of the model's head layer through training while freezing the other layers, and then training all layers while unfreezing the other layers.

[0081] S3: Use the trained bridge apparent subtle damage detection model to automatically identify bridge apparent subtle damage.

[0082] Corresponding to the aforementioned embodiment of the method for identifying apparent subtle damage to a bridge based on a multi-scale feature single-sequence fusion algorithm, the present invention also provides an embodiment of an apparatus for identifying apparent subtle damage to a bridge based on a multi-scale feature single-sequence fusion algorithm.

[0083] An embodiment of the present invention provides a device for identifying apparent subtle damage to a bridge based on a multi-scale feature single-sequence fusion algorithm, comprising one or more processors for implementing the method for identifying apparent subtle damage to a bridge based on a multi-scale feature single-sequence fusion algorithm in the above embodiment.

[0084] The embodiment of the bridge apparent subtle damage identification device based on the multi-scale feature single sequence fusion algorithm of the present invention can be applied to any device with data processing capabilities, and the any device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware, or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located to read the corresponding computer program instructions in the non-volatile memory into the memory and run them. From the hardware level, the bridge apparent subtle damage identification device based on the multi-scale feature single sequence fusion algorithm of the present invention is a hardware structure of any device with data processing capabilities. In addition to the processor, memory, network interface, and non-volatile memory, the any device with data processing capabilities in which the device in the embodiment is located can also include other hardware according to the actual function of the device with data processing capabilities, which will not be described in detail.

[0085] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0086] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0087] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for identifying bridge apparent subtle damage based on a multi-scale feature single-sequence fusion algorithm in the above embodiment is implemented.

[0088] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device, such as a plug-in hard disk, a smart memory card (SmartMedia card, SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0089] Example

[0090] In this example, we selected 2,275 images of bridges with subtle surface damage, including 294 images of exposed rebar, 1,500 images of cracks, 375 images of holes, and 106 images of roughness. Considering the hardware resource cost and computational complexity of neural network data processing, we cropped the sample images to a resolution of 800 x 600 and annotated them using Labelme software. Figure 3 The MS Defect dataset constructed in this embodiment is presented. The dataset is divided into non-overlapping training and validation datasets. Each dataset uses the original image as input and the annotated mask image as the prediction comparison value for testing and training network performance.

[0091] The model was trained using images of bridge surface defects from the dataset as input data. The ReLU activation parameter was set to 0.2, and the number of iterations was 200-500. If the performance requirements were not met, hyperparameter optimization was continued. If the performance requirements were met, the training weights were saved to obtain the trained bridge surface defect recognition model CITNet. The hyperparameter values ​​set in this example are detailed in Table 1.

[0092] Table 1 CITNet model hyperparameter settings

[0093]

[0094] In this embodiment, IOU, precision P, recall R, F1 score and mean average precision (mAP) are used to evaluate the recognition performance of the bridge apparent subtle damage detection model.

[0095] IOU (the Intersection over Union) represents the ratio of the overlapping area to the total area of ​​the actual area and the predicted area, and is used to evaluate the accuracy of instance segmentation. A higher IoU value indicates better instance segmentation performance. The formula is:

[0096]

[0097] in, is the overlapping area of ​​the actual area and the predicted area, is the total area of ​​the actual area and the predicted area.

[0098] Precision P refers to the proportion of true positive samples among all samples predicted to be positive, and is used to evaluate the model's ability to distinguish negative samples. Its calculation formula is:

[0099]

[0100] Positive samples predicted by the model as positive are called TP (True Positives); negative samples predicted by the model as negative are called TN (True Negatives); negative samples predicted by the model as positive are called FP (False Positives); and positive samples predicted by the model as negative are called FN (False Negatives). Compared to classification tasks, instance segmentation has higher requirements and therefore a higher standard for correct identification. A detection is considered correct only when the defect type in the detection area matches the ground truth area, and the IOU between the two areas reaches at least a certain value.

[0101] The recall rate R is the proportion of actual positive samples that are correctly identified as positive samples, which is used to evaluate the ability of the model to identify positive samples. Its calculation formula is:

[0102]

[0103] F1score is the harmonic mean of precision and recall, and is a comprehensive indicator for measuring the accuracy of model segmentation:

[0104]

[0105] Average precision (mAP) is a comprehensive measure of the overall accuracy of the model. After obtaining the precision and recall values, a precision-recall (PR) curve is plotted with precision on the vertical axis and recall on the horizontal axis. The area enclosed by the PR curve and the coordinate axis represents the AP (average precision) value for a specific defect type, expressed as

[0106]

[0107] mAP is the average of all class AP values ​​and is expressed as

[0108]

[0109] Where n represents the number of categories, Represents the AP value of the i-th category. The bounding box mAP calculated with an IOU threshold of 0.5 is expressed as , and the mAP of the mask calculated with an IOU threshold of 0.5 is expressed as .

[0110] like Figure 4 As shown, CITNet performs well in automatically identifying images with various defects. The recognition accuracy of each defect has improved significantly: the mAP for pitting increased from 20.75% to 45.28%, a 118.18% improvement; the mAP for cracks increased from 5.26% to 28.49%, a 441.32% improvement; the mAP for holes increased from 0 to 77.81%; and the mAP for exposed reinforcement increased from 0 to 42.31%. Model comparison experiments (see Table 2 for the experimental configuration) show that, for pitting and holes, each module in CITNet has a positive impact on the model, and when all modules are integrated, the best experimental results are achieved. For pitting, all four models successfully detected the target, but the baseline model's segmentation results showed a high degree of misidentification. For C and CT, most defect locations were detected, but the resulting segmentation masks were smaller than the ground truth or incorrect. CFT segmented the defects more accurately. For holes, the baseline model failed to detect the defects. After inserting the CBAM block, IFPN block, and transposed convolution, more defects were successfully detected, and mAP values ​​increased to varying degrees. As with the pockmarked surface, the CBAM block, IFPN block, and transposed convolution all had a positive impact on the segmentation task, with CFT performing the best among the four models. It can be seen that CITNet also performs well in complex and relatively small backgrounds.

[0111] Table 2 Comparative experimental settings

[0112]

[0113] At the same time, for different CBAM connection methods, Figure 5 The model conditions under different CBAM connection modes are shown in the figure. The connection ratio ( Figure 5 a) and parameters ( Figure 5 b) shows the complexity of the model, mAP ( Figure 5 c) and mIOU ( Figure 5 e) demonstrates the accuracy of the model, Figure 5Figure d shows the specific connection details of different connection schemes, where 2-5 refers to stage2-stage5, I refers to the identity block, and C refers to the conv block. Through experiments, we found that some sparsely connected networks can surpass the accuracy of the fully added CBAM network, for example, when the connection ratio is between 0.03 and 0.09. At the same time, some networks perform worse than the original network without CBAM, indicating that certain connection schemes may impair model performance. In addition, even with the same connection ratio, the accuracy and network complexity may vary, with some improving network performance while others impairing it. Therefore, choosing a connection scheme with smaller connections and higher accuracy can not only reduce network complexity, but also avoid the negative impact of the attention mechanism module in some cases, minimizing computational costs while ensuring prediction results.

[0114] Those skilled in the art will understand that the foregoing descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art will still be able to modify the technical solutions described in the foregoing examples or substitute equivalents for some of the technical features therein. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the invention shall be included within the scope of protection of the invention.

Claims

1. A bridge apparent subtle damage identification method based on a multi-scale feature single sequence fusion algorithm, characterized by: The method comprises the following steps: S1: Establish a bridge apparent subtle damage dataset; S2: Build and train a bridge apparent subtle damage detection model; the bridge apparent subtle damage detection model includes a multi-scale feature fusion model CITFPN, RPN, RoI Align, and a network head architecture that outputs prediction results; the multi-scale feature fusion model includes a feature extraction network that introduces an attention module and a single-sequence feature fusion module IFPN that introduces transposed convolution; The multi-scale feature fusion model is used to perform a one-time single-sequence superposition fusion of the high-semantic feature map generated by the last convolution stage of the feature extraction network and the feature maps generated by the remaining convolution stages except the first convolution stage output by the feature extraction network, and then further convolution processing is performed on the fused fusion feature map; at the same time, the high-semantic feature map is downsampled; then, all the feature maps after the further convolution processing and the downsampled high-semantic feature map are fused again as the output of the multi-scale feature fusion model; the output of the multi-scale feature fusion model is used as the input of the RPN, and a fixed-size feature region is generated through the RPN and RoI Align, and the corresponding mask, bounding box and category label are output through the network head architecture that outputs the prediction result; S3: Use the trained bridge apparent subtle damage detection model to automatically identify bridge apparent subtle damage.

2. The bridge apparent subtle damage identification method based on multi-scale feature single sequence fusion algorithm according to claim 1 is characterized in that: The feature extraction network is Resnet101, which includes five convolution stages C1, C2, C3, C4, and C5. The single-sequence feature fusion module IFPN upsamples the feature map p5 generated in the C5 stage, and performs a one-time single-sequence superposition fusion with the feature maps generated by C2, C3, and C4 to obtain p2, p3, and p4. Then, p2, p3, p4, and p5 are subjected to 3×3 convolution processing to obtain P2, P3, P4, and P5; finally, P5 is downsampled to obtain P6, and P2, P3, P4, P5, and P6 are fused again as the output of the multi-scale feature fusion model.

3. The bridge apparent subtle damage identification method based on multi-scale feature single sequence fusion algorithm according to claim 1 is characterized in that: The identity block and convolution block in the feature extraction network Resnet101 have built-in attention modules, which include a channel attention submodule and a spatial attention submodule, which compress and weight features in the channel dimension and spatial dimension respectively, so as to enhance detail features and suppress noise.

4. The bridge apparent subtle damage identification method based on multi-scale feature single sequence fusion algorithm according to claim 3 is characterized in that: The specific steps of the attention module to compress and weight features in the channel dimension and spatial dimension are as follows: (1) After the input feature F1 is compressed by the average pooling layer and the maximum pooling layer, it passes through two layers of perceptrons to obtain two independent channel features. The two channel features are further added element by element, and the channel attention weight Mc is obtained through the convolution layer with Sigmoid activation function; (2) Multiplying the channel attention weight Mc by the input feature F1 to obtain the channel attention feature F2, inputting the channel attention feature F2 into the average pooling layer and the maximum pooling layer in series, and then passing through the convolution layer with the Sigmoid activation function to obtain the spatial attention weight Ms; (3) Multiply the spatial attention weight Ms by the channel attention feature F2 to obtain the final attention enhancement feature F3.

5. The bridge apparent subtle damage identification method based on multi-scale feature single sequence fusion algorithm according to claim 2 is characterized in that: The single-sequence feature fusion module IFPN upsamples the feature map p5 generated in the C5 stage using transposed convolution.

6. The bridge apparent subtle damage identification method based on multi-scale feature single sequence fusion algorithm according to claim 1 is characterized in that: When training the bridge apparent subtle damage detection model, the transfer learning method is first used to transfer the training weights of the COCO dataset to initialize the model; then, a staged training method is adopted, that is, the parameters of the model head layer are first adjusted through training while the other layers are frozen, and then all layers are trained while the other layers are unfrozen.

7. A bridge apparent subtle damage identification system based on a multi-scale feature single sequence fusion algorithm, characterized by: The method comprises one or more processors for implementing the bridge apparent subtle damage identification method based on the multi-scale feature single sequence fusion algorithm according to any one of claims 1 to 6.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the electronic device, enables the electronic device to implement the bridge apparent subtle damage identification method based on the multi-scale feature single-sequence fusion algorithm as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, the method for identifying bridge apparent subtle damage based on a multi-scale feature single sequence fusion algorithm as described in any one of claims 1 to 6 is implemented.

Images