Bridge apparent slenderness defect recognition method based on parallel convolution adaptive segmentation model

Abstract

The present invention discloses a method for identifying apparent slenderness defects in bridges based on a parallel convolution adaptive segmentation model. The method comprises a feature extraction network with a built-in parallel feature extraction module (DCM), an adaptive spatial feature fusion model (ASFF), an RPN, a RoI Align, and a network head architecture. The DCM is a parallel module embedded in each stage of the feature extraction network, fusing standard convolution and dynamic snake convolution. Each stage is interspersed in parallel. One branch uses dynamic snake convolution to dynamically adjust the convolution kernel through iterative offset to further perceive the morphological characteristics of apparent slenderness defects in bridges, while the other branch uses standard convolution to perform global information constraints. The ASFF layer-by-layer superposition and fusion of the high-semantic feature map of the last convolution stage with the remaining feature maps except the first convolution stage, and achieves feature adaptive fusion through learnable parameters. The present invention can achieve automated and refined detection of apparent slenderness defects in bridges.

Description

Technical Field

[0001] The present invention relates to the field of bridge structure appearance information detection, and in particular to a method for identifying bridge appearance slenderness defects based on a parallel convolution adaptive segmentation model. Background Art

[0002] In recent years, many researchers at home and abroad have conducted extensive research on automatic identification of surface defects based on deep learning, particularly in the detection of various bridge defects such as cracks, spalling, and holes. However, existing bridge inspection technologies still face various challenges in identifying and quantifying slender bridge defects in complex real-world scenarios:

[0003] (1) In actual engineering applications, due to the special features of bridge structures and the safety requirements of shooting equipment, the number of pixels of the defect targets in drone images is often small, and there is a significant imbalance between the number of pixels occupied by the defect targets and the background. In addition, most images have complex backgrounds and a lot of noise. Existing bridge defect instance segmentation models are mostly trained based on crack datasets collected in laboratory environments or magnified defect images, which are significantly different from actual scenes. As a result, existing models are difficult to directly apply to recognition tasks in actual engineering, and their application is limited.

[0004] (2) The recognition accuracy of slender diseases needs to be further improved. Most automatic detection methods are not general enough in terms of the morphology of slender diseases.

[0005] (3) The existing identification methods involve a single type of disease, most of which focus on typical diseases such as cracks, and rarely identify other slender diseases such as exposed steel bars. In addition, there are difficulties in identifying other conventional diseases at the same time. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, the present invention proposes a bridge apparent slenderness defect recognition method based on a parallel convolution adaptive segmentation model. The specific technical solution is as follows:

[0007] A method for identifying apparent slenderness defects of bridges based on a parallel convolution adaptive segmentation model comprises the following steps:

[0008] S1: Construct a dataset of bridge apparent slenderness defects;

[0009] S2: Construct a bridge apparent slenderness disease detection model DCANet, which includes a feature extraction network with a built-in parallel feature extraction module DCM, an adaptive spatial feature fusion model ASFF, RPN, RoI Align, and a network head architecture that outputs prediction results. The parallel feature extraction module DCM is a parallel module that integrates standard convolution Conv and dynamic snake convolution DSConv. The feature extraction of each stage is interspersed in parallel, where one branch uses the dynamic snake convolution method to further perceive the morphological characteristics of the bridge apparent slenderness disease, and the other branch uses standard convolution to constrain global information. The parallel feature extraction module DCM is embedded in the identity block and convolution block of each stage of the feature extraction network.

[0010] The adaptive spatial feature fusion model ASFF performs layer-by-layer superposition and fusion of the high semantic feature map generated by the last convolution stage of the feature extraction network and the feature maps generated by the remaining convolution stages except the first convolution stage output by the feature extraction network to obtain a multi-scale feature map Pi, i = 2~k; then performs an upsampling or downsampling operation F on Pi il , and according to the characteristics of the feature maps at all levels, the spatial weight vectors at all levels are obtained Finally, adaptively fuse the feature maps at all levels to obtain the feature maps P′i at all levels; then perform convolution on P′i to obtain P″i; downsample P″k to obtain P″(k+1), and fuse P″2 to P″(k+1) again as the output of the adaptive spatial feature fusion model ASFF;

[0011] The dynamic snake convolution DSConv dynamically adjusts the convolution kernel through iterative offset so that it can better cover the shape of the disease and extract features from the disease image area.

[0012] S3: Use the trained bridge apparent slenderness defect detection model DCANet to automatically identify bridge apparent slenderness defects.

[0013] Furthermore, the parallel feature extraction module DCM performs the following steps:

[0014] (1) Initializing the parallel convolution module in the parallel feature extraction module DCM;

[0015] (2) Feature extraction is performed on the input disease feature map through standard convolution and dynamic snake convolution respectively. The parameters of dynamic snake convolution need to be continuously adjusted according to the image features and model performance;

[0016] (3) The feature maps generated by the two convolutions are integrated, and the output weights of the two convolutions are adjusted and optimized according to the model performance and image characteristics to ensure effective feature fusion and enhance the model's ability to recognize slender and complex diseases.

[0017] Furthermore, the execution steps of the dynamic snake convolution DSConv are as follows:

[0018] (1) Determine the center coordinate starting point K i±c =(x i±c ,y i±c ), c = {0, 1, 2, 3, 4}, c is the horizontal distance from the center grid, and the center coordinate starting point is used as the starting point of the convolution kernel iterative offset;

[0019] (2) Select the next position of each target to be processed for observation in turn, so as to ensure the continuity of attention and not spread the perception range too far due to large deformation offset. i Initially, each grid K i±c The position depends on the previous grid K i±c-1 Position, that is, each grid K i±c In the previous grid K i±c-1 Add an offset Δ={δ|δ∈[-1,1]} to the basis, and the same applies to the y-axis direction;

[0020] (3) If the convolution kernel sensing area formed by the point selected in step (2) contains part of the location of the disease, then select this point as the starting point of the next convolution iteration; otherwise, return to step (2) and continue to select a new direction for offset;

[0021] (4) Smoothing the convolution kernel perception area selected in step (3) by bilinear interpolation;

[0022] (5) Repeat steps (2) to (4) so ​​that the convolution kernel is iteratively shifted to a position that can better cover the shape of the disease, and perform feature extraction of the corresponding image area.

[0023] Furthermore, the feature extraction network is Resnet101, which includes five convolution stages C1, C2, C3, C4, and C5. The specific workflow of the adaptive spatial feature fusion model ASFF is as follows:

[0024] (1) The Resnet101 network with the embedded DCM module upsamples the feature map P5 generated in the C5 stage, and then superimposes and fuses it layer by layer with the feature maps generated by C2, C3, and C4 to obtain multi-scale feature maps P2, P3, and P4;

[0025] (2) For the first level feature map output Pl, the remaining three feature maps are upsampled or downsampled F il , in order to carry out subsequent adaptive feature fusion;

[0026] (3) According to the characteristics of the feature maps at each level, the spatial weight vectors at each level are obtained

[0027] (4) Adaptively fuse the feature maps at all levels according to the formula Obtain the characteristic maps P'2~P'5 at each level of scale;

[0028] (5) During the training process, the weight vector is continuously optimized and updated to achieve adaptive fusion effect.

[0029] Furthermore, the loss function of the bridge apparent slenderness defect detection model DCANet during training is a weighted sum of the classification loss value, the bounding box regression loss value, and the mask loss value; the weight of the loss function for each category is different.

[0030] Furthermore, when training the bridge apparent slenderness defect detection model DCANet, the transfer learning method is first used to transfer the training weights of the COCO dataset to initialize the model; then, a staged training method is adopted, that is, first, the parameters of the model head layer are adjusted through training while freezing the other layers, and then all layers are trained while unfreezing the other layers.

[0031] A bridge apparent slenderness defect recognition system based on a parallel convolution adaptive segmentation model includes one or more processors for implementing a bridge apparent slenderness defect recognition method based on a parallel convolution adaptive segmentation model.

[0032] An electronic device, comprising:

[0033] one or more processors;

[0034] A storage device is used to store one or more programs. When the one or more programs are executed by the electronic device, the electronic device implements a method for identifying bridge apparent slenderness defects based on a parallel convolution adaptive segmentation model.

[0035] A computer-readable storage medium stores a program, which, when executed by a processor, implements a method for identifying apparent slenderness defects of bridges based on a parallel convolution adaptive segmentation model.

[0036] The beneficial effects of the present invention are as follows:

[0037] (1) The present invention designs a parallel feature extraction model (DCM), which realizes multi-level feature extraction of the input image through continuous parallel convolution and pooling operations fused with standard convolution and dynamic snake convolution, further perceives the slender and complex morphological characteristics of the apparent slender defects of the bridge using the dynamic snake convolution method, constrains the global information using standard convolution, and simultaneously captures the global and local information of the defect image through the parallel structure, which not only improves the efficiency of feature extraction, but also better covers the defect shape by dynamically adjusting the shape and position of the snake convolution kernel, enhances the processing ability of the global and detailed information of the slender defects, improves the detection accuracy, and is more suitable for the identification needs of minor and slender defects in on-site bridge inspections.

[0038] (2) A learnable and optimizable adaptive spatial feature fusion model ASFF is introduced into the bridge apparent slenderness defect detection model DCANet to perform adaptive feature fusion, which alleviates the inconsistency problem between different feature scales and the conflict information between spatial levels to a certain extent, obtains defect features that retain more detailed features, and improves the accuracy of defect detection.

[0039] (3) The intelligent recognition technology for apparent slender defects of bridges can perform image segmentation on slender defect images with large scale, unbalanced categories, large background noise, and complex defect morphology taken during on-site bridge inspections. This alleviates the impact of problems such as large defect size, small background noise, and relatively simple defect morphology in traditional data set images. It facilitates the use of equipment such as drones to achieve high-precision and high-efficiency defect recognition, significantly improving the automation and refined detection capabilities of bridge defects, and providing more reliable, objective, and comprehensive data support for bridge health monitoring and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of the bridge apparent slenderness defect identification method based on the parallel convolution adaptive segmentation model of the present invention.

[0041] Figure 2 This is a network structure diagram of the bridge apparent slenderness disease detection model of the present invention.

[0042] Figure 3 Schematic diagram of the MSDefect dataset according to an embodiment of the present invention.

[0043] Figure 4 This is an example diagram of the identification of typical diseases obtained in an embodiment of the present invention.

[0044] Figure 5 Comparison chart of recognition results for different Dsconv positions. DETAILED DESCRIPTION

[0045] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become more apparent. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0046] Technical term explanation

[0047] ResNet, Residual Neural Network, residual network;

[0048] FPN, Feature Pyramid Network, Feature Pyramid Network;

[0049] RPN, Region Proposal Network, Region Proposal Network;

[0050] RoI Align,Region of Interest Align;

[0051] Dsconv, Dynamic Snake Convolution, dynamic snake convolution;

[0052] Conv, Convolution, standard convolution;

[0053] ASFF, Adaptive Spatial Feature Fusion, adaptive spatial feature fusion;

[0054] Bilinear Interpolation, bilinear interpolation;

[0055] Baseline, baseline model;

[0056] Identity Block, identity block;

[0057] Conv Block,Convolutional Block,Convolutional Block;

[0058] FCN, Fully Convolutional Network, Fully Convolutional Network;

[0059] Dense Layers,Dense Layers,Fully connected layers;

[0060] Adam, Adaptive Moment Estimation, adaptive moment estimation;

[0061] SGD, Stochastic Gradient Descent, stochastic gradient descent;

[0062] IoU, the Intersection over Union, intersection over union ratio;

[0063] P, Precision, accuracy;

[0064] R,Recall, recall rate;

[0065] F1 Score,F1 Score,F1 score;

[0066] mAP, mean Average Precision, average precision.

[0067] like Figure 1 As shown, the bridge apparent slenderness defect recognition method based on the parallel convolution adaptive segmentation model of the present invention includes the following steps:

[0068] S1: Construct a dataset of bridge apparent slenderness defects.

[0069] Using drones equipped with cameras and other shooting equipment to collect images of bridge surface subtle damage, a dataset of bridge surface slender defects is established based on photos taken during on-site bridge inspections as raw data. The dataset has the following characteristics: (1) Each image contains at least one slender defect, especially slender defects with small and fragile local structures and complex and variable global morphologies, such as cracks and exposed rebar. Each type of defect has a certain number of image samples in the dataset to meet the needs of model training, for example, no less than 100 images.

[0070] A series of data augmentation operations, including angular rotation, translation, shearing, scaling, and horizontal flipping, were then performed on the images in the Bridge Apparent Slenderness Defects Dataset to enhance the model's adaptability to different deformations and viewpoints. The dataset was then divided into non-overlapping training and validation sets. Each set used the original images as input, while the annotated mask images served as prediction comparison values ​​for testing and training the network's performance.

[0071] S2: Construct the bridge apparent slenderness defect detection model DCANet.

[0072] Focusing on the problem of identifying slender defects in complex background faced by bridge defect detection, this paper proposes a bridge apparent slender defect detection model DCANet, such as Figure 2As shown in the figure, the parallel feature extraction model (DCM) effectively enhances the processing capability of global and detailed information of slender defects, improves the detection accuracy, and is more suitable for the identification of minor and slender defects in field bridge inspections, alleviating the problem of complex and diverse defect morphologies in bridge defect identification. In the bridge apparent slender defect detection model DCANet, a learnable and optimizable adaptive spatial feature fusion model ASFF is introduced to perform adaptive feature fusion, which alleviates the inconsistency problem between different feature scales and the conflict information between spatial levels to a certain extent. It is more suitable for the identification of small target defects, unbalanced categories, and complex backgrounds in field bridge inspections.

[0073] In order to accurately obtain the number, area and contour shape of defects, such as Figure 2 As shown in the figure, the bridge apparent slenderness defect detection model DCANet includes a feature extraction network with a built-in parallel feature extraction module DCM, an adaptive spatial feature fusion model ASFF, RPN, RoI Align, and a network head architecture that outputs prediction results.

[0074] Taking Resnet101 as an example, it contains five convolution stages C1, C2, C3, C4, and C5. In the process of disease recognition and segmentation, the apparent image data of the bridge is first input into the feature extraction network. For the parallel feature extraction module DCM, the parallel feature extraction module DCM is a parallel module that integrates standard convolution Conv and dynamic snake convolution DSConv. The feature extraction of each stage is interspersed in parallel. One branch uses the dynamic snake convolution method to further perceive the morphological characteristics of the apparent slender disease of the bridge, and the other branch uses the standard convolution kernel to constrain the global information; the parallel feature extraction module DCM is embedded in the identity block and convolution block of each stage of the feature extraction network. Dynamic snake convolution Dsconv has great advantages in flexibly focusing on the complex geometric features of the disease. Dynamic snake convolution has good dynamic structural adaptability in the x-axis and y-axis directions, and can pay more attention to the local features of the object, especially the slender structure. Define the center coordinate of the convolution kernel as K i =(x i ,y i ), the 3*3 convolution kernel with a dilation rate of 1 can be expressed as:

[0075] K={(x-1,y-1),(x-1,y),…,(x+1,y+1)

[0076] The dynamic snake convolution does not always exist as a standard 3*3 convolution kernel, but iteratively shifts. By sequentially selecting the next position of each target to be processed for observation, it ensures the continuity of attention and does not spread the perception range too far due to large deformation shifts.

[0077] The execution steps of dynamic snake convolution DSConv are as follows:

[0078] (1) Determine the center coordinate starting point K i±c =(x i±c ,y i±c ), c = {0, 1, 2, 3, 4}, c is the horizontal distance from the center grid, and the center coordinate starting point is used as the starting point of the convolution kernel iterative offset;

[0079] (2) Select the next position of each target to be processed for observation in turn, so as to ensure the continuity of attention and not spread the perception range too far due to large deformation offset. i Initially, each grid K i±c The position depends on the previous grid K i±c-1 Position, that is, each grid K i±c In the previous grid K i±c-1 Add an offset Δ={δ|δ∈[-1,1]} to the basis, and the same applies to the y-axis direction;

[0080]

[0081] (3) If the convolution kernel sensing area formed by the point selected in step (2) contains part of the location of the disease, then select this point as the starting point of the next convolution iteration; otherwise, return to step (2) and continue to select a new direction for offset;

[0082] (4) Since the offset may be a decimal, while the coordinates are usually in integer form, it is necessary to smooth the convolution kernel perception area selected in step (3) through bilinear interpolation;

[0083] K=∑ K′ B(K′,K)·K′

[0084] B(K,K′)=b(K x ,K x ′)·b(K y ,K y ′)

[0085] Where K′ is an enumeration of all integer spatial positions, B(K′,K) is a bilinear weight function, and b(K x ,K x ′) and b(K y ,K y ′) correspond to the one-dimensional interpolation kernel functions in the x and y directions respectively.

[0086] (5) Repeat steps (2) to (4) so ​​that the convolution kernel is iteratively shifted to a position that can better cover the shape of the disease, and perform feature extraction of the corresponding image area.

[0087] Compared to standard convolution, dynamic snake convolution dynamically adjusts the shape and position of the convolution kernel to better cover the overall shape of the defect, capturing both its overall and local characteristics. This enhances the network's ability to perceive the target, especially for slender defects. This improves DCANet's accuracy when segmenting slender defects such as cracks and exposed rebar. Furthermore, due to DSConv's adaptive advantage in identifying defects of varying sizes, DCANet achieves greater accuracy and adaptability when segmenting the same defect with similar characteristics but varying sizes.

[0088] In practical applications, DCM first extracts features from the input image (or feature map) using both standard convolution and dynamic snake convolution. The parameters of the dynamic snake convolution are continuously adjusted based on image features and model performance. The feature maps generated by the two convolutional layers are then integrated, and their output weights are adjusted and optimized based on model performance and image characteristics to ensure effective feature fusion and enhance the model's ability to recognize long, thin, and complex diseases. This parallel structure allows the model to simultaneously capture both global and local information from disease images, improving not only feature extraction efficiency but also detection accuracy by dynamically adjusting the shape and position of the convolution kernel to better cover the disease shape.

[0089] The parallel feature extraction module DCM is embedded in the identity block and convolution block of each stage of the feature extraction network to optimize feature capture and extraction. Resnet is a residual network structure that optimizes image processing performance through a deep network architecture while ensuring that the performance is at least equal to that of a shallow network. In order to solve the problem of performance degradation of deep networks, a shortcut connection of identity mapping between blocks is established. Specifically, each block accepts an input feature x, and the feature objective function H(x) = F(x) + x that originally needs to be learned by the layer. F(x) represents the residual part that the network block needs to learn, and x is the input feature directly added to the output through a shortcut connection. Then:

[0090] Output of layer l: x l+1 =x l +F(x l ,w l ), x l Input for layer l

[0091] Output of layer l+1: x l+2 =x l+1 +F(x l+1 ,w l+1 )=x l +F(x l ,w l )+F(x l+1 ,w l+1 )

[0092] Output of layer L:

[0093] Use the chain rule to calculate the gradient back propagation:

[0094]

[0095] The gradient of each layer is transmitted not only through the residual function F(x) but also directly through shortcut connections of the identity mapping. This structural design ensures that the overall gradient does not vanish in deep networks, avoiding the problem of gradient loss of shallow parameters caused by the chain rule. Furthermore, when adding network layers fails to significantly improve performance, the residual F(x) tends to 0. At this point, the output feature y is the same as the input feature x, equivalent to not adding new layers to the network, thus ensuring that network performance does not degrade.

[0096] Through continuous convolution and pooling within the feature extraction network of the built-in parallel feature extraction module (DCM), multi-level feature extraction is achieved for the input image, resulting in a series of multi-scale feature maps with decreasing resolution and increasing semantic information. The adaptive spatial feature fusion model (ASFF) then performs adaptive feature fusion on the disease features, enhancing the network's ability to refine disease processing. Because the adaptive spatial feature fusion model (ASFF) has learnable features, it can suppress conflicting information and enhance feature scale invariance by learning spatial inter-layer weights without incurring additional computational cost, thereby improving the recognition accuracy of minor, slender diseases.

[0097] The adaptive spatial feature fusion model ASFF stacks and fuses the high semantic feature map generated by the last convolution stage of the feature extraction network with the feature maps generated by the remaining convolution stages of the feature extraction network except the first convolution stage layer by layer to obtain a multi-scale feature map Pi, i = 2~k; then performs an upsampling or downsampling operation F on Pi il , and according to the characteristics of the feature maps at all levels, the spatial weight vectors at all levels are obtained Finally, the feature maps at all levels are adaptively fused to obtain the scale feature maps P′i at all levels; P′i is then convolved to obtain P″i; P″k is downsampled to obtain P″(k+1), and P″2~P″(k+1) are fused again as the output of the adaptive spatial feature fusion model ASFF.

[0098] Taking the feature extraction network Resnet101 as an example, it contains five convolution stages C1, C2, C3, C4, and C5. The multi-scale feature maps P2 to P5 (corresponding to the convolution stages C2 to C5, respectively) generated by the Resnet101 network with the embedded DCM module are input into the adaptive spatial feature fusion model ASFF, and ASFF performs upsampling or downsampling operations on P2 to P5.il , i = 2 to 5, in order to carry out subsequent adaptive feature fusion. According to the characteristics of the feature maps at each level, the spatial weight vectors at each level are obtained

[0099]

[0100] Adaptively fuse the feature maps at all levels to obtain the feature maps P′2 to P′5 at all levels. The formula is:

[0101]

[0102] The traditional FPN uses a layer-by-layer fusion method to perform i The generated feature maps are integrated. Specifically, taking p2 and p3 as an example, before the p2 feature map is fused with p3, p3 itself has already integrated the high-level feature information from p4 and p5. In this process, the feature maps of each layer are purely added layer by layer, resulting in the information of each layer being equally integrated into the final feature map. As a result, some detailed information may be suppressed, some unimportant information is externalized, and there are inconsistencies between different feature scales. To solve this problem, ASFF adds a learnable coefficient as a weight based on FPN, automatically learning the relationship between each layer, enabling adaptive feature fusion, automatically adjusting the weight relationship between feature maps of each layer, improving the invariance of feature scale, and helping to improve disease recognition performance.

[0103] The four feature maps P′2 to P′5 obtained are further extracted through 3×3 convolution to obtain P″2 to P″5. P″5 is downsampled again to generate a reference feature map P″6 that is more conducive to generating prediction results. Finally, P″2 to P″6 are fused again as the output of the adaptive spatial feature fusion model ASFF, and input into the region generation network RPN for the final feature map of target classification and boundary regression prediction. The extracted feature map is further input into RPN and RoI Align to generate a fixed-size feature region, and the full convolution layer and fully connected layer are used to output the corresponding mask, bounding box and category label.

[0104] During the training of the bridge apparent slenderness defect detection model DCANet, a reasonable optimization method should be considered to accelerate model convergence. Adam is used as the training optimizer, and stochastic gradient descent (SGD) is used for model training.

[0105] The loss function is used to guide the model training, and its formula is:

[0106] L=w cls L cls +w box L box +w maskL mask

[0107] Among them, L cls Represents the classification loss value, L box Represents the regression loss value of the bounding box; L mask Indicates the loss value of the mask part. w cls 、w box 、w mask By assigning different loss function weights to different types of diseases, we can more fully learn the hard-to-detect parts, such as the instance information of slender diseases, and improve the sensitivity of the model.

[0108] To improve model training efficiency, the bridge apparent slenderness defect detection model DCANet was trained by first using a transfer learning method to transfer the training weights of the COCO dataset to initialize the model. Then, a phased training method was adopted. That is, the parameters of the model head layer were first adjusted through training while freezing the other layers, and then all layers were trained while unfreezing the other layers.

[0109] S3: Use the trained bridge apparent slenderness defect detection model DCANet to automatically identify bridge apparent slenderness defects.

[0110] An embodiment of the present invention provides a device for identifying apparent slenderness defects of bridges based on a parallel convolution adaptive segmentation model, comprising one or more processors for implementing the method for identifying apparent slenderness defects of bridges based on a parallel convolution adaptive segmentation model in the above embodiment.

[0111] The embodiments of the bridge apparent slenderness defect identification method based on a parallel convolutional adaptive segmentation model can be applied to any device with data processing capabilities, such as a computer or other device. Device embodiments can be implemented using software, hardware, or a combination of software and hardware.

[0112] Taking software implementation as an example, as a logical device, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions from non-volatile memory into the internal memory and executing them. From a hardware perspective, the bridge apparent slenderness defect identification device based on the parallel convolution adaptive segmentation model of the present invention has a hardware structure of any device with data processing capabilities. In addition to the processor, memory, network interface, and non-volatile memory, the device with data processing capabilities in which the device is located in the embodiments may also include other hardware based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0113] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0114] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0115] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for identifying apparent slenderness defects of bridges based on a parallel convolution adaptive segmentation model in the above embodiment is implemented.

[0116] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device, such as a plug-in hard disk, a smart memory card (SmartMedia card, SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0117] Example

[0118] Due to the unique characteristics of bridge structures and the safety of the camera equipment, most images contain defects with a low pixel count and high background complexity. This example selected 1,794 images of bridge defects with apparent slenderness, including 294 images of exposed rebar and 1,500 images of cracks. All images were taken from construction sites and did not include defect images collected in a laboratory setting. Considering the hardware resource cost and computational complexity of neural network data processing, we cropped the sample images to a resolution of 800 x 600 and annotated them using Labelme software. Figure 3The MSDefect dataset constructed in this embodiment is presented. The dataset is divided into non-overlapping training and validation datasets. Each dataset uses the original image as input data and the annotated mask image as the prediction comparison value for testing and training network performance.

[0119] The model was trained using images of bridge surface defects from the dataset as input data for 500 iterations and 30 pre-training cycles. If the model did not meet the performance requirements, hyperparameter optimization was continued. If it did, the trained weights were saved to produce the trained DCANet bridge surface slenderness defect detection model. In this model, the weights for most defects were set to [1, 1, 1], while the weights for slenderness defects were set to [0.5, 0.5, 1]. The relevant hyperparameters were determined experimentally and are shown in Table 1. The ReLU activation parameter was set to 0.2, and training was performed for 200-500 epochs.

[0120] Table 1 DCANet model hyperparameter settings

[0121]

[0122] In this embodiment, IOU, precision P, recall R, F1 score and mean average precision (mAP) are used to evaluate the recognition performance of the bridge apparent subtle damage detection model.

[0123] IOU (the Intersection over Union) represents the ratio of the overlapping area to the total area of ​​the actual area and the predicted area, and is used to evaluate the accuracy of instance segmentation. The higher the IoU value, the better the instance segmentation performance. The formula is:

[0124]

[0125] Among them, S overlap is the overlapping area between the actual area and the predicted area, S union is the total area of ​​the actual area and the predicted area.

[0126] Precision P refers to the proportion of true positive samples among all samples predicted to be positive, and is used to evaluate the model's ability to distinguish negative samples. Its calculation formula is:

[0127]

[0128] Among them, positive samples predicted by the model as positive are TP (true positive); negative samples predicted by the model as negative are TN (true negative); negative samples predicted by the model as positive are FP (false positive); and positive samples predicted by the model as negative are FN (false negative). Compared to classification tasks, instance segmentation has higher requirements and therefore higher standards for correct identification. A detection is considered correct only when the defect type in the detection area is consistent with the ground truth area and the IOU between the two areas reaches at least a certain value.

[0129] The recall rate R is the proportion of actual positive samples that are correctly identified as positive samples, which is used to evaluate the ability of the model to identify positive samples. Its calculation formula is:

[0130]

[0131] F1score is the harmonic mean of precision and recall, and is a comprehensive indicator for measuring the accuracy of model segmentation:

[0132]

[0133] Average precision (mAP) is a comprehensive measure of the overall accuracy of the model. After obtaining the precision and recall values, a precision-recall (PR) curve is plotted with precision as the vertical axis and recall as the horizontal axis. The area enclosed by the PR curve and the coordinate axis represents the AP (average precision) value for a specific defect type, expressed as

[0134]

[0135] mAP is the average of all class AP values ​​and is expressed as

[0136]

[0137] Where n represents the number of categories, AP i Represents the AP value of the i-th category. The bounding box mAP calculated with an IOU threshold of 0.5 is denoted as mAP bb , and the mAP of the mask calculated with an IOU threshold of 0.5 is expressed as mAP m .

[0138] like Figure 4As shown, DCANet performs well in automatically identifying various defect images, achieving significant improvements in defect recognition accuracy compared to the baseline. For cracks, for example, mAP increases from 5.26% to 34.92%, a 563.54% improvement; and F1score increases from 7.17% to 34.12%, a 375.69% improvement. For exposed rebar, for example, mAP increases from 0 to 67.49%, and F1score increases from 0 to 50.85%. Model comparison experiments (see Table 2 for the experimental configuration) show that each module in DCANet has a positive impact on the model, and when all modules are integrated, the best experimental results are achieved. All three models successfully detect cracks, but the baseline segmentation results often miss or misidentify them. For AF, most defect locations are detected, but the resulting segmentation masks are smaller than the ground truth or incorrect. DCA achieves relatively accurate segmentation results, performing the best among the three models. The segmentation results of exposed rebar are similar to those of cracks. The baseline cannot generate segmentation results. AF can detect some instances, but there are some incorrect segmentations. DCA successfully detects the defect range more accurately. It can be seen that DCANet can also perform well in different scenes with complex backgrounds and small proportions. In addition, it can be found that compared with DCA, more images of baseline and AF are concentrated in the parts with low thresholds and accuracy ( Figure 4 b cracks, Figure 4 c exposed ribs), which highlights the ability of DCA in handling complex geometric defect features.

[0139] Table 2 Comparative experimental settings

[0140]

[0141] At the same time, for different Dsconv positions, Figure 5Figure 2 shows model recognition results for different DsConv positions. DSConv was inserted into various conv blocks or identity blocks in ResNet101, both at the lower and higher levels. Within the same stage, conv blocks showed better accuracy and more successful segmentation cases than identity blocks, indicating that inserting DSConv within conv blocks generally leads to better model performance. This is because conv blocks primarily focus on feature perception, while identity blocks primarily focus on feature transfer. Accurate feature perception is crucial for improving segmentation accuracy. Therefore, embedding DSConv within conv blocks is more beneficial for optimizing model performance. Within the same block, lower levels showed better accuracy and more successful detection image distributions than higher levels, indicating that inserting DSConv within lower levels generally leads to better model performance. This is because the parameters of lower levels are more sensitive to the input, causing changes in lower-level features that in turn affect higher-level features. Therefore, integrating DSConv within lower levels significantly enhances model performance.

[0142] Those skilled in the art will understand that the foregoing descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art will still be able to modify the technical solutions described in the foregoing examples or substitute equivalents for some of the technical features therein. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the invention shall be included within the scope of protection of the invention.

Claims

1. A bridge apparent slenderness defect recognition method based on a parallel convolution adaptive segmentation model, characterized in that: The method comprises the following steps: S1: Construct a dataset of bridge apparent slenderness defects; S2: Construct a bridge apparent slenderness disease detection model DCANet, which includes a feature extraction network with a built-in parallel feature extraction module DCM, an adaptive spatial feature fusion model ASFF, RPN, RoIAlign, and a network head architecture that outputs prediction results; the parallel feature extraction module DCM is a parallel module that integrates standard convolution Conv and dynamic snake convolution DSConv. The feature extraction of each stage is interspersed in parallel, where a single branch uses the dynamic snake convolution method to further perceive the morphological characteristics of the bridge apparent slenderness disease, and the other branch uses standard convolution to constrain global information; the parallel feature extraction module DCM is embedded in the identity block and convolution block of each stage of the feature extraction network; The adaptive spatial feature fusion model ASFF performs layer-by-layer superposition and fusion of the high semantic feature map generated by the last convolution stage of the feature extraction network and the feature maps generated by the remaining convolution stages except the first convolution stage output by the feature extraction network to obtain a multi-scale feature map Pi, i = 2~k; then performs an upsampling or downsampling operation F on Pi il , and according to the characteristics of the feature maps at all levels, the spatial weight vectors at all levels are obtained Finally, adaptively fuse the feature maps at all levels to obtain the feature maps P′i at all levels; then perform convolution on P′i to obtain P″i; downsample P″k to obtain P″(k+1), and fuse P″2 to P″(k+1) again as the output of the adaptive spatial feature fusion model ASFF; The dynamic snake convolution DSConv dynamically adjusts the convolution kernel through iterative offset so that it can better cover the shape of the disease and extract features from the disease image area. S3: Use the trained bridge apparent slenderness defect detection model DCANet to automatically identify bridge apparent slenderness defects.

2. The bridge apparent slenderness defect recognition method based on the parallel convolution adaptive segmentation model according to claim 1 is characterized in that: The execution steps of the parallel feature extraction module DCM are as follows: (1) Initializing the parallel convolution module in the parallel feature extraction module DCM; (2) Feature extraction is performed on the input disease feature map through standard convolution and dynamic snake convolution respectively. The parameters of dynamic snake convolution need to be continuously adjusted according to the image features and model performance; (3) The feature maps generated by the two convolutions are integrated, and the output weights of the two convolutions are adjusted and optimized according to the model performance and image characteristics to ensure effective feature fusion and enhance the model's ability to recognize slender and complex diseases.

3. The bridge apparent slenderness defect recognition method based on the parallel convolution adaptive segmentation model according to claim 1 is characterized in that: The execution steps of the dynamic snake convolution DSConv are as follows: (1) Determine the center coordinate starting point K i±c =(x i±c ,y i±c ), c = {0, 1, 2, 3, 4}, c is the horizontal distance from the center grid, and the center coordinate starting point is used as the starting point of the convolution kernel iterative offset; (2) Select the next position of each target to be processed for observation in turn, so as to ensure the continuity of attention and not spread the perception range too far due to large deformation offset, from the center position K i Initially, each grid K i±c The position depends on the previous grid K i±c-1 Position, that is, each grid K i±c In the previous grid K i±c-1 Add an offset Δ={δ|δ∈[-1,1]} to the basis, and the same applies to the y-axis direction; (3) If the convolution kernel sensing area formed by the point selected in step (2) contains part of the location of the disease, then select this point as the starting point of the next convolution iteration; otherwise, return to step (2) and continue to select a new direction for offset; (4) Smoothing the convolution kernel perception area selected in step (3) by bilinear interpolation; (5) Repeat steps (2) to (4) so ​​that the convolution kernel is iteratively shifted to a position that can better cover the shape of the disease, and perform feature extraction of the corresponding image area.

4. The bridge apparent slenderness defect recognition method based on the parallel convolution adaptive segmentation model according to claim 1 is characterized in that: The feature extraction network is Resnet101, which includes five convolution stages C1, C2, C3, C4, and C5. The specific workflow of the adaptive spatial feature fusion model ASFF is as follows: (1) The Resnet101 network with the embedded DCM module upsamples the feature map P5 generated in the C5 stage, and then superimposes and fuses it layer by layer with the feature maps generated by C2, C3, and C4 to obtain multi-scale feature maps P2, P3, and P4; (2) For the first level feature map output Pl, the remaining three feature maps are upsampled or downsampled F il , in order to carry out subsequent adaptive feature fusion; (3) According to the characteristics of the feature maps at each level, the spatial weight vectors at each level are obtained (4) Adaptively fuse the feature maps at all levels according to the formula Obtain the characteristic maps P'2~P'5 at each level of scale; (5) During the training process, the weight vector is continuously optimized and updated to achieve adaptive fusion effect.

5. The bridge apparent slenderness defect recognition method based on the parallel convolution adaptive segmentation model according to claim 1 is characterized in that: The loss function of the bridge apparent slenderness defect detection model DCANet during training is the weighted sum of the classification loss value, the bounding box regression loss value and the mask loss value; the weight of the loss function of each category is different.

6. The bridge apparent slenderness defect recognition method based on the parallel convolution adaptive segmentation model according to claim 1 is characterized in that: When training the bridge apparent slenderness defect detection model DCANet, the model is first initialized by transferring the training weights of the COCO dataset using a transfer learning method. Then, a staged training method is adopted. That is, the parameters of the model head layer are first adjusted through training while the other layers are frozen. Then, all layers are trained while the other layers are unfrozen.

7. A bridge apparent slenderness defect recognition system based on a parallel convolution adaptive segmentation model, characterized by: The method comprises one or more processors for implementing the bridge apparent slenderness defect recognition method based on the parallel convolution adaptive segmentation model according to any one of claims 1 to 6.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the electronic device, enables the electronic device to implement the bridge apparent slenderness defect identification method based on the parallel convolution adaptive segmentation model as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, the method for identifying bridge apparent slenderness defects based on a parallel convolution adaptive segmentation model as claimed in any one of claims 1 to 6 is implemented.

Images