Bridge apparent disease semantic segmentation method and system based on improved Deeplabv3 +

Abstract

The invention discloses an improved bridge apparent disease semantic segmentation method and system of a Deeplabv3 + model, and belongs to the field of visual semantic segmentation, and the method comprises the steps: collecting a bridge apparent disease picture; marking the bridge apparent disease pictures to make a data set; an improved Deeplabv3 + semantic segmentation model is constructed; and training the constructed model on the data set, and evaluating the performance of the model. According to the method, the test precision of the model on a bridge apparent disease data set is improved by improving the Deeplabv3 + model, and research of a deep learning model in the field of bridge diseases is promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of apparent defect detection, and in particular to a method and system for semantic segmentation of bridge apparent defect images based on an improved DeepLabv3+ model. Background Art

[0002] As infrastructure ages and ages, the detection and maintenance of surface defects (such as cracks, spalling, and rust) in bridges becomes increasingly important. Traditional bridge defect detection methods rely primarily on manual inspections, which are not only time-consuming and labor-intensive but also susceptible to subjective factors, making it difficult to ensure the accuracy and consistency of detection results. In recent years, automated detection technologies based on computer vision and deep learning have become a research hotspot. In particular, deep learning models have demonstrated strong potential in the field of image semantic segmentation.

[0003] In the task of image semantic segmentation, many models have attracted much attention due to their excellent performance. However, in the specific task of bridge surface disease detection, there are few related studies, the detection effect needs to be improved urgently, and some challenges still exist. Complex background interference: Bridge surface images usually contain complex backgrounds (such as vegetation, shadows, stains, etc.). These interference factors can easily lead to model misjudgment and reduce segmentation accuracy. Diversity of diseases: The morphology and scale of bridge diseases vary greatly, from small cracks to large areas of peeling, and it is difficult to fully cover the feature extraction of a single scale. High data annotation cost: The annotation of bridge disease datasets requires professional knowledge and a lot of manpower, resulting in a scarcity of high-quality annotated data, which limits the training effect of the model.

[0004] Therefore, to address the above problems, a semantic segmentation method for bridge apparent defects based on the improved DeepLav3+ model is proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for detecting apparent defects of bridge diseases based on DeepLabV3+, so as to solve the problem of low accuracy in detecting apparent defects of bridge diseases.

[0006] To solve the above technical problems, the present invention proposes a method and system for detecting apparent defects of bridge diseases based on DeepLabV3+. The method proposed in the present invention is as follows:

[0007] The following steps are involved:

[0008] S1: Collect the surface defect images of the bridge to be inspected and mark the defects on the images;

[0009] S2: Divide the labeled bridge surface defect images into training and test sets;

[0010] S3: The training set images are input into the improved deeplabv3+ model for training. The trained model is used for semantic segmentation of bridge surface defects.

[0011] A bridge apparent disease semantic segmentation system based on improved DeepLabV3+, including:

[0012] Backbone network module, spatial pyramid pooling module, attention convolution module, feature fusion module;

[0013] The backbone network module is used to extract semantic information contained in the image to obtain a first feature map;

[0014] The spatial pyramid module is used to further obtain deep semantic information based on the first feature map to obtain a second feature map;

[0015] The attention convolution module is used to focus on valuable semantic information in the first feature map to obtain a third feature map;

[0016] The feature fusion module is used to fuse and filter the acquired second feature map and third feature map to obtain a fourth feature map.

[0017] Preferably, the first feature map is obtained from the final output layer of the backbone network Mobilenetv2; shallow semantic information is obtained from the fourth layer of the backbone network Mobilenetv2 as a shallow feature map; the shallow feature map and the fourth feature map are spliced and then upsampled to obtain the final semantic segmentation result map.

[0018] Preferably, the attention convolution module is a module with similar functions and structures simulated by a convolutional neural network based on the execution process and structural characteristics of the Transformer module.

[0019] Preferably, the calculation method of the attention convolution module is:

[0020]

[0021] Q = Conv1(X)

[0022] K = Conv2(X)

[0023] V = Conv3(X)

[0024] Where X represents the input feature vector, which is fed into three different 1×1 attention convolution modules to obtain the query Q, key K, and value V respectively. CBR() represents the convolution, batch normalization, and regularization operations on the input content, and finally obtains the output result of the attention convolution module.

[0025] Preferably, the feature fusion module is composed of a multi-scale pooling unit, a channel attention unit and a spatial attention unit;

[0026] Specifically, the feature representation of the channel dimension is first extracted through global average pooling (GAP) and global maximum pooling (GMP), and then the channel attention weight is calculated through a fully connected network, and the original input features are weighted; then, the spatial attention weight is calculated on the channel-enhanced feature map, and the spatial attention map is extracted using a 7×7 convolution operation, and finally a weighted adjustment of the spatial dimension is performed to achieve feature enhancement.

[0027] Preferably, the multi-scale pooling unit performs global average pooling and global maximum pooling on the input feature map:

[0028] F avg =AdaptiveAvgPool2d(X)

[0029] F max =AdaptiveMaxPool2d(X)

[0030] f sum =Flatten(F avg )+Flatten(F max )

[0031] Among them, X represents the input feature vector, F avg is the result of global average pooling of the input feature map, F max is the result of global maximum pooling of the input feature map, f sum Is F avg and F max Flatten the vectors and add them together.

[0032] Preferably, the channel attention mechanism is calculated as follows:

[0033] W channel =σ(W2·RELU(W1·f sum ))

[0034] X channel =X⊙W channel

[0035] Among them, W1 is the weight matrix of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, σ is the Sigmoid activation function, and ⊙ represents element-by-element multiplication.

[0036] Preferably, the spatial attention mechanism is calculated as follows:

[0037] F cat=Concat(Mean(X channel ,dim=1),Max(X channel ,dim=1))

[0038] W spatial =σ1(Conv(F cat ))

[0039] X output =X channel ⊙W spatial

[0040] Among them, X channel Represents the input feature vector of the spatial attention mechanism, Mean() means finding the mean of the input feature vector in the first dimension, Max() means finding the maximum value of the input feature vector in the first dimension, Concat() means concatenating two vectors, Conv() means performing a convolution operation on the input vector, σ1 means the Sigmoid activation function, and ⊙ means element-by-element multiplication.

[0041] Preferably, a loss function is used in the model training process, and the loss function calculation formula is:

[0042] Loss = focal_loss

[0043]

[0044] Among them, K means that this is a K classification problem, p i Indicates that the predicted value is the softmax predicted probability, y i =1 means that the i-th category is the correct category, p i is the model's predicted logit value for category i, γ≥0 is used to control the weight change of difficult and easy samples, α i is the category balance factor, which is used to adjust the weights of different categories.

[0045] Specifically:

[0046] Obtain a dataset of bridge apparent damage and label images corresponding to the dataset images;

[0047] Divide the data set and the corresponding labeled images into training set and test set;

[0048] Build an improved deeplabv3+ model;

[0049] Use the training set to train on the constructed deeplabv3+ model to obtain the improved deeplabv3+ model weights, and use the test set to test on the weights to obtain the semantic segmentation results of the image.

[0050] Preferably, the improved deeplabv3+ model includes a backbone network module, a spatial pyramid pooling module, an attention convolution module, and a feature fusion module. The backbone network module is used to extract the semantic information contained in the image to obtain a first feature map; the spatial pyramid module is used to further obtain deep semantic information on the basis of the first feature map to obtain a second feature map; the attention convolution module is used to focus on valuable semantic information in the first feature map to obtain a third feature map; the feature fusion module is used to fuse and filter the obtained second feature map and third feature map to obtain a fourth feature map.

[0051] Preferably, the backbone network module adopts the MobileNetv2 network, obtains the first feature map from the final output layer of the backbone network Mobilenetv2; obtains shallow semantic information from the fourth layer of the backbone network Mobilenetv2 as a shallow feature map; splices the shallow feature map and the fourth feature map and then upsamples them to obtain the final semantic segmentation result map.

[0052] Optionally, the attention convolution module is a module with similar functions and structures simulated by a convolutional neural network based on the execution process and structural characteristics of the Transformer module. The calculation method of the attention convolution module is:

[0053]

[0054] Q = Conv1(X)

[0055] K = Conv2(X)

[0056] V = Conv3(X)

[0057] Where X represents the input feature vector, which is fed into three different 1×1 attention convolution modules to obtain the query Q, key K, and value V respectively. CBR() represents the convolution, batch normalization, and regularization operations on the input content, and finally obtains the output result of the attention convolution module.

[0058] Optionally, the feature fusion module is composed of a multi-scale pooling unit, a channel attention unit, and a spatial attention unit. Specifically, it first extracts the feature representation of the channel dimension through global average pooling (GAP) and global maximum pooling (GMP). Then, the channel attention weight is calculated through a fully connected network and the original input features are weighted. Then, the spatial attention weight is calculated on the channel-enhanced feature map, and a 7×7 convolution operation is used to extract the spatial attention map. Finally, a weighted adjustment of the spatial dimension is performed to achieve feature enhancement.

[0059] Optionally, the multi-scale pooling unit performs global average pooling and global maximum pooling on the input feature map:

[0060] F avg =AdaptiveAvgPool2d(X)

[0061] F max =AdaptiveMaxPool2d(X)

[0062] f sum =Flatten(F avg )+Flatten(F max )

[0063] Among them, X represents the input feature vector, F avg is the result of global average pooling of the input feature map, F max is the result of global maximum pooling of the input feature map, f sum Is F avg and F max Flatten the vectors and add them together.

[0064] Optionally, the channel attention mechanism is calculated as follows:

[0065] W channel =σ(W2·RELU(W1·f sum ))

[0066] X channel =X⊙W channel

[0067] Among them, W1 is the weight matrix of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, σ is the Sigmoid activation function, and ⊙ represents element-by-element multiplication.

[0068] Optionally, the spatial attention mechanism is calculated as follows:

[0069] F cat =Concat(Mean(X channel ,dim=1),Max(X channel ,dim=1))

[0070] W spatial =σ1(Conv(F cat ))

[0071] X output =X channel ⊙W spatial

[0072] Among them, Xchannel Represents the input feature vector of the spatial attention mechanism, Mean() means finding the mean of the input feature vector in the first dimension, Max() means finding the maximum value of the input feature vector in the first dimension, Concat() means concatenating two vectors, Conv() means performing a convolution operation on the input vector, σ1 means the Sigmoid activation function, and ⊙ means element-by-element multiplication.

[0073] Optionally, a loss function is required during model training. The loss function calculation formula is:

[0074] Loss = focal_loss

[0075]

[0076] Among them, K means that this is a K classification problem, p i Indicates that the predicted value is the softmax predicted probability, y i =1 means that the i-th category is the correct category, p i is the model's predicted logit value for category i, γ≥0 is used to control the weight change of difficult and easy samples, α i is the category balance factor, which is used to adjust the weights of different categories.

[0077] Compared with the prior art, the present invention has the following beneficial effects:

[0078] The present invention improves the average intersection-over-union (IoU) of bridge surface defect image datasets by modifying the model architecture of Deeplabv3+, enabling the trained neural network model to obtain higher accuracy, thereby improving the application of neural network models in bridge surface defect detection and positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0080] Figure 1 It is a schematic diagram of the basic method flow of the present invention;

[0081] Figure 2 The overall network architecture design of the embodiment of the present invention;

[0082] Figure 3 This is a structural diagram of a convolutional attention module according to an embodiment of the present invention;

[0083] Figure 4 This is a structural diagram of the feature fusion module of an embodiment of the present invention. DETAILED DESCRIPTION

[0084] The specific embodiments of the present invention are intended to help those skilled in the art understand the technical solutions of the present invention, but do not limit the scope of the present invention. Without departing from the spirit of the present invention, those skilled in the art may make appropriate modifications or equivalent substitutions therefor. Therefore, the scope of protection of the present invention shall be subject to the claims.

[0085] The terms used in this specification are only used to describe specific embodiments and do not limit the scope of the present invention. Unless the context indicates otherwise, terms in the singular also include the plural form. In addition, "and / or" represents any or all combinations of the listed items. The terms "first", "second", etc. are only used to distinguish similar elements and do not represent a specific order or importance. For example, "first component" can be called "second component" and vice versa. In addition, the term "if" can be understood as "when..." or "based on a determined result" depending on the context.

[0086] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0087] according to Figures 1 to 4 As shown, the present invention provides a method for detecting apparent defects of bridge diseases based on DeepLabV3+, comprising the following steps:

[0088] Obtain a dataset of bridge surface damage images and corresponding labels, and divide the dataset and corresponding labels into a training set and a test set. The dataset of bridge surface damage images used in this paper was collected and annotated by ourselves on the Internet. It contains a total of 1100 images, which are divided into 5 categories: cracks, spalling, rust, whitening, and background. The images are randomly shuffled and 800 of them are selected as the training set, 150 images as the test set, and the remaining 150 images as the test set. All images are of size 1024×1024 and have 3 channels. Construct an improved DeepLabV3+ model:

[0089] The model includes a backbone network module, a spatial pyramid pooling module, an attention convolution module, and a feature fusion module. The backbone network module is used to extract semantic information contained in the image to obtain a first feature map; the spatial pyramid module is used to further obtain deep semantic information based on the first feature map to obtain a second feature map; the attention convolution module is used to focus on valuable semantic information in the first feature map to obtain a third feature map; the feature fusion module is used to fuse and filter the obtained second and third feature maps to obtain a fourth feature map.

[0090] Specifically:

[0091] (1) Backbone network module:

[0092] When selecting the backbone network, we chose the lightweight MobileNetv2 network. Compared with other backbone networks such as Xception, ResNet50, and ResNet101, the MobileNetv2 model has fewer parameters, which helps speed up training and facilitate deployment.

[0093] MobileNetV2 is a lightweight deep neural network optimized for mobile and compute-constrained devices. Its core improvements include depthwise separable convolution, an inverted residual architecture, and a linear bottleneck. Depthwise separable convolution decomposes the standard convolution into pointwise convolution between channels (1×1) and depthwise convolution within channels (3×3), significantly reducing computation and parameter count. Traditional ResNets use residual connections, which perform better with high-dimensional inputs. However, MobileNetV2 introduces an inverted residual architecture, which first uses 1×1 convolutions for dimensionality increase, then 3×3 depthwise convolutions for feature extraction, and finally 1×1 convolutions for dimensionality reduction. This design allows for more efficient transfer of residual information from low-dimensional features, improving feature representation. After the 1×1 convolutions for dimensionality reduction, MobileNetV2 uses a linear activation function instead of ReLU. This is because ReLUs can lose critical information in low-dimensional space, while linear activations better preserve features. The entire network consists of multiple stacked inverted residual modules, and finally uses global average pooling and a fully connected layer to output the classification result.

[0094] The model structure of MobileNetv2 is shown in Table 1.

[0095] Table 1:

[0096]

[0097] (2) Spatial pyramid pooling module;

[0098] The spatial pyramid pooling module uses dilated convolutions at different sampling rates to capture information at different scales. It also incorporates global features, enabling the model to better understand complex structures in images. The spatial pyramid pooling module consists of five parallel branches: four dilated convolution branches, typically using 1×1 standard convolutions, and 3×3 dilated convolutions. The 3×3 dilated convolutions employ different dilation rates (6, 12, and 18) to extract features at different scales. This expands the receptive field without increasing computational effort, enabling the model to capture a wider range of contextual information. The global average pooling branch captures global contextual information across the entire image. Its output undergoes feature compression using 1×1 convolutions, followed by bilinear interpolation to match the spatial dimensions of the other branches. The design of the spatial pyramid pooling module effectively mitigates the scale variation caused by pooling or fixed convolution kernel sizes, enhancing the ability to capture object shapes and boundaries, thereby improving segmentation accuracy.

[0099] (3) Attention convolution module;

[0100] Because the dataset itself has a complex background and contains many subtle features that are not easy to observe, such as cracks, this patent proposes using an attention mechanism to allow the model to focus on certain features in a targeted manner. Therefore, an attention module is introduced in the spatial pyramid module. However, considering that the MobileNetv2 network is a lightweight module and the Transformer module is relatively complex, a convolutional neural network can achieve functions similar to the Transformer module. The feature vector input to the attention convolution module is subjected to three 1×1 convolutions. The resulting parameters are used as the query Q, key K, and value V in the attention mechanism. The dot product between the Q vector and the K vector is calculated to obtain an attention score. This score reflects the similarity between Q and the key. After convolution, batch normalization, and regularization operations, the attention weight of each part is obtained. These attention weights are weighted and summed with the corresponding V to obtain a weighted sum vector. This weighted sum vector is used as a new representation of the image, containing richer contextual information.

[0101] The calculation method of the attention convolution module is:

[0102]

[0103] Q = Conv1(X)

[0104] K = Conv2(X)

[0105] V = Conv3(X)

[0106] Where X represents the input feature vector, which is fed into three different 1×1 attention convolution modules to obtain the query Q, key K, and value V respectively. CBR() represents the convolution, batch normalization, and regularization operations on the input content, and finally obtains the output result of the attention convolution module.

[0107] (4) Feature fusion module;

[0108] After adding the attention convolution module in parallel to the dilated convolution module, the original deeplabv3+ module will directly concatenate all feature vectors. If the features of different modules are highly overlapped, the model may learn redundant information, which will affect the generalization ability and cause the model accuracy to be lower than the original model performance. In addition, directly concatenating the feature vectors of different modules cannot effectively extract the valuable semantic information from the features used. Considering the above reasons, the feature fusion module is introduced, and its structure is as follows: Figure 4 shown.

[0109] The feature fusion module consists of three parts: multi-scale pooling, channel attention, and spatial attention. Multi-scale pooling uses global average pooling and global maximum pooling to extract channel-level features. Channel attention uses a fully connected layer to calculate channel weights to improve the expression of important features. Spatial attention obtains a spatial attention map through 7×7 convolution to improve the model's perception of the target area.

[0110] The calculation formula for multi-scale pooling is as follows:

[0111] F avg =AdaptiveAvgPool2d(X)

[0112] F max =AdaptiveMaxPool2d(X)

[0113] f sum =Flatten(F avg )+Flatten(F max )

[0114] Among them, X represents the input feature vector, F avg is the result of global average pooling of the input feature map, F max is the result of global maximum pooling of the input feature map, f sum Is F avg and F max Flatten the vectors and add them together.

[0115] The channel attention mechanism is calculated as follows:

[0116] W channel =σ(W2·RELU(W1·f sum ))

[0117] X channel =X⊙W channel

[0118] Among them, W1 is the weight matrix of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, σ is the Sigmoid activation function, and ⊙ represents element-by-element multiplication.

[0119] The spatial attention mechanism is calculated as follows:

[0120] F cat =Concat(Mean(X channel ,dim=1),Max(X channel ,dim=1))

[0121] W spatial =σ1(Conv(F cat ))

[0122] X output =X channel ⊙W spatial

[0123] Among them, X channel Represents the input feature vector of the spatial attention mechanism, Mean() means finding the mean of the input feature vector in the first dimension, Max() means finding the maximum value of the input feature vector in the first dimension, Concat() means concatenating two vectors, Conv() means performing a convolution operation on the input vector, σ1 means the Sigmoid activation function, and ⊙ means element-by-element multiplication.

[0124] Multi-scale pooling can obtain more comprehensive channel information, and 7×7 convolution is used in the spatial attention module to enhance spatial correlation, so that the model pays more attention to the target area.

[0125] Loss function design:

[0126] The loss function used in this invention is an improved cross entropy loss focal_loss, which is mainly used to solve the problem of class imbalance, mainly by reducing the loss weight of easy-to-classify samples and increasing the focus on difficult-to-classify samples. focal_loss introduces an adjustment factor (1-p i ) γ To reduce the impact of easy-to-classify samples and increase attention to difficult-to-classify samples, the calculation formula is as follows:

[0127] Loss = focal_loss

[0128]

[0129] Among them, K means that this is a K classification problem, p i Indicates that the predicted value is the softmax predicted probability, y i =1 means that the i-th category is the correct category, p i is the model's predicted logit value for category i, γ≥0 is used to control the weight change of difficult and easy samples, α i It is a class balance factor, which is used to adjust the weights of different classes to increase the influence of minority class samples. i close to 1), (1-p i ) γ The weight becomes smaller, reducing the loss contribution of the sample; when the sample is difficult to classify p i The smaller the value, the larger the weight, making the loss greater, thus guiding the model to pay more attention to difficult-to-classify samples.

[0130] Model training:

[0131] The model initialization weights were trained on the ImageNet dataset to accelerate model convergence. The Bridge Surface Defects Dataset consists of 1,100 images collected online and labeled using the LabelMe annotation tool. 800 images were randomly shuffled to form the training set, 150 images as the validation set, and the remaining 150 images as the test set. Each image has a size of 1024×1024. The images were fed into the model for training, with a training epoch of 100.

[0132] The model was trained using the stochastic gradient descent method. After training, the trained model weights were saved. The model performance was tested on the test set using the model weights, and the average intersection-over-union ratio of the evaluation index was calculated to be 57.6%.

[0133] The experimental results of the deeplabv3+ model before and after improvement on the test set are shown in Table 2, where w / indicates the addition of a module to the model, FF indicates the feature fusion module, and CT indicates the attention convolution module.

[0134] Table 2:

[0135]

[0136] It can be seen that compared with the original deeplabv3+ model, the average intersection-over-union ratio of the improved model is 1.8% higher than that of the original model. However, due to the complex background of bridge surface defect images in the natural environment and the overlap between different defects, the overall recognition accuracy of the model still has a lot of room for improvement.

[0137] The present invention provides a method for identifying apparent bridge defects based on an improved deeplabv3+ model, which solves the problems of lack of research content and poor experimental results in the prior art and has broad application prospects. Those skilled in the art can make various deformations or modifications to the present invention without departing from the core idea of the present invention, and these deformations or modifications should be deemed to fall within the scope of protection of the present invention. The present invention has been described in detail above with reference to specific embodiments, but the present invention is not limited to the above-mentioned embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various deformations and adjustments to the present invention, and all these modifications are within the scope of protection of the present invention and are subject to the scope defined by the claims.

Claims

1. A semantic segmentation method for bridge apparent defects based on improved deeplabv3+, characterized by: The following steps are involved: S1: Collect the surface defect images of the bridge to be inspected and mark the defects on the images; S2: Divide the labeled bridge surface defect images into training and test sets; S3: The training set images are input into the improved deeplabv3+ model for training. The trained model is used for semantic segmentation of bridge surface defects.

2. A bridge apparent disease semantic segmentation system based on improved deeplabv3+, characterized by: include: Backbone network module, spatial pyramid pooling module, attention convolution module, feature fusion module; The backbone network module is used to extract semantic information contained in the image to obtain a first feature map; The spatial pyramid module is used to further obtain deep semantic information based on the first feature map to obtain a second feature map; The attention convolution module is used to focus on valuable semantic information in the first feature map to obtain a third feature map; The feature fusion module is used to fuse and filter the acquired second feature map and third feature map to obtain a fourth feature map.

3. The bridge apparent defect semantic segmentation system based on improved deeplabv3+ according to claim 2 is characterized by: The first feature map is obtained from the final output layer of the backbone network Mobilenetv2; the shallow semantic information is obtained from the fourth layer of the backbone network Mobilenetv2 as the shallow feature map; the shallow feature map and the fourth feature map are spliced and upsampled to obtain the final semantic segmentation result map.

4. The bridge apparent defect semantic segmentation system based on improved deeplabv3+ according to claim 2 is characterized by: The attention convolution module uses a convolutional neural network to simulate and produce a module with similar functions and structures based on the execution process and structural characteristics of the Transformer module.

5. The bridge apparent defect semantic segmentation system based on improved deeplabv3+ according to claim 4 is characterized by: The calculation method of the attention convolution module is: Q = Conv1(X) K = Conv2(X) V = Conv3(X) Where X represents the input feature vector, which is fed into three different 1×1 attention convolution modules to obtain the query Q, key K, and value V respectively. CBR() represents the convolution, batch normalization, and regularization operations on the input content, and finally obtains the output result of the attention convolution module.

6. The bridge apparent defect semantic segmentation system based on improved deeplabv3+ according to claim 2 is characterized by: The feature fusion module is composed of a multi-scale pooling unit, a channel attention unit and a spatial attention unit; Specifically, the feature representation of the channel dimension is first extracted through global average pooling (GAP) and global maximum pooling (GMP), and then the channel attention weight is calculated through a fully connected network and the original input features are weighted; Then, the spatial attention weight is calculated on the channel-enhanced feature map, a 7×7 convolution operation is used to extract the spatial attention map, and finally a weighted adjustment of the spatial dimension is performed to achieve feature enhancement.

7. The bridge apparent defect semantic segmentation system based on improved deeplabv3+ according to claim 6 is characterized by: The multi-scale pooling unit performs global average pooling and global maximum pooling on the input feature map: F avg =AdaptiveAvgPool2d(X) F max =AdaptiveMaxPool2d(X) f sum =Flatten(F avg )+Flatten(F max ) Among them, X represents the input feature vector, F avg is the result of global average pooling of the input feature map, F max is the result of global maximum pooling of the input feature map, f sum Is F avg and F max Flatten the vectors and add them together.

8. The bridge apparent defect semantic segmentation system based on improved deeplabv3+ according to claim 6 is characterized by: The channel attention mechanism is calculated as follows: IN channel =σ(W2·RELU(W1·f sum )) X channel =X⊙W channel Among them, W1 is the weight matrix of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, σ is the Sigmoid activation function, and ⊙ represents element-by-element multiplication.

9. The bridge apparent defect semantic segmentation system based on improved deeplabv3+ according to claim 6 is characterized by: The spatial attention mechanism is calculated as follows: F cat =Concat(Mean(X channel ,dim=1),Max(X channel ,dim=1)) W spatial =σ1(Conv(F cat )) X output =X channel ⊙W spatial Among them, X channel Represents the input feature vector of the spatial attention mechanism, Mean() means finding the mean of the input feature vector in the first dimension, Max() means finding the maximum value of the input feature vector in the first dimension, Concat() means concatenating two vectors, Conv() means performing a convolution operation on the input vector, σ1 means the Sigmoid activation function, and ⊙ means element-by-element multiplication.

10. The bridge apparent defect semantic segmentation method based on improved deeplabv3+ according to claim 1 is characterized by: The loss function is used in the model training process. The loss function calculation formula is: Loss = focal_loss Among them, K means that this is a K classification problem, p i Indicates that the predicted value is the softmax predicted probability, y i =1 means that the i-th category is the correct category, p i is the model's predicted logit value for category i, γ≥0 is used to control the weight change of difficult and easy samples, α i is the category balance factor, which is used to adjust the weights of different categories.

Images