Bridge concrete defect multi-label classification method based on improved OfficientNetV2 model

By improving the EfficientNetV2 model, combining SCARM and SPPF modules and asymmetric loss function, the problems of irrelevant information interference and multi-defect type detection in bridge concrete defect detection are solved, and efficient and accurate multi-label classification is achieved to adapt to the needs of complex scenarios.

CN119006928APending Publication Date: 2024-11-22ANHUI UNIVERSITY OF TECHNOLOGY
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411234860.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-04
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The prior art is difficult to effectively reduce irrelevant information interference such as stains and holes in the detection of bridge concrete defects, and detect multiple defect types. The traditional methods rely on manpower and are not efficient and accurate enough.

Method used

Using the improved EfficientNetV2 model, multi-label classification of bridge concrete defects is performed by inserting SCARM and SPPF modules, combining asymmetric loss function, reducing redundant information, enhancing feature representation capabilities, and improving the attention mechanism and multi-scale feature fusion Model robustness and accuracy.

Benefits of technology

It realizes efficient multi-label classification of bridge concrete defects, reduces redundant information interference during the inspection process, improves detection efficiency and accuracy, adapts to complex and changeable practical application scenarios, and reduces manpower and material investment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006928A_ABST
    Figure CN119006928A_ABST
Patent Text Reader

Abstract

The invention discloses a bridge concrete defect multi-label classification method based on an improved OfficientNetV2 model, and belongs to the technical field of concrete defect classification. According to the method, an SCARM module is added into an original OfficientNetV2 network, redundant information on space and channels of a feature map extracted by the network is removed, redundant information in the model training process is reduced, and learning of representative features is promoted through an attention mechanism; in view of the fact that images in a data set have different sizes and aspect ratios, an SPPF module is adopted to perform multi-scale feature fusion before a classification layer, and a network can be helped to extract spatial feature information of various sizes, so that the robustness of a model to object deformation is improved; in order to solve the problem that positive and negative samples in a data set are unbalanced, an asymmetric loss function is adopted for training, the weight and a hard threshold can be dynamically adjusted to reduce the influence of simple negative samples, and the samples possibly marked wrongly can be filtered.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of concrete defect classification, and in particular to a multi-label classification method for bridge concrete defects based on an improved EfficientNetV2 model. Background Art

[0002] As time goes by, bridges will develop various structural and non-structural defects such as cracks, corrosion, and spalling due to environmental erosion, material aging, and load overload. If these defects are not discovered and repaired in time, they may lead to structural performance degradation or even collapse accidents, causing significant casualties and economic losses. Therefore, ensuring the safety and stability of bridges and extending their service life is an important task of bridge maintenance and management.

[0003] Traditional bridge defect detection and assessment mainly rely on on-site visual inspection by professionals or the use of some basic detection tools. Since many bridges are located in elevated or inaccessible geographical locations, such as above rivers or between valleys, it is very dangerous to conduct a comprehensive inspection on them. Professionals often need to work at height and use scaffolding, hanging baskets or climbing equipment to inspect the bridges one by one, which not only increases the time and cost of the inspection process, but also greatly increases the safety risks of operators. This traditional method is not only time-consuming and labor-intensive, but also difficult to guarantee accuracy and comprehensiveness. Although some high-tech equipment, such as infrared scanning, has been introduced to assist in detection work, these methods still rely on a lot of manpower and material resources, and the data processing and analysis process mostly depends on the experience and judgment of operators. In recent years, with the advancement of technology, image recognition technology based on deep learning has been favored in the field of defect detection. With the help of advanced image acquisition tools such as drones, it can automatically analyze images without direct contact with the bridge, identify defects such as cracks and corrosion, and greatly reduce the risk of operation. Compared with traditional detection methods, this method not only greatly improves the efficiency and accuracy of detection, but also can continuously improve the recognition performance through continuous learning and optimization, and better adapt to complex and changeable practical application scenarios.

[0004] In recent years, deep convolutional neural networks (CNNs) have been widely used for image and video classification. Driven by this progress, many studies have begun to adopt the CNN framework for applications in different fields such as structural engineering, such as determining the bearing capacity of soil, structural optimization of concrete arch bridges, and analyzing road damage. Similarly, in the task of civil structure defect classification, many classification methods based on the CNN framework have emerged. However, although researchers have achieved remarkable results in the detection and classification of bridge cracks, few studies have focused on the situation where multiple types of bridge concrete defects are superimposed. To this end, this paper proposes a multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model. Summary of the invention

[0005] The technical problem to be solved by the present invention is: how to reduce the interference of irrelevant information such as stains, holes, poster residues, graffiti, paintings, etc. on the surface of bridges in real scenes, and at the same time detect multiple defect types that appear together in an image, and provide a multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model.

[0006] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps:

[0007] S1: Sample pretreatment

[0008] Preprocess the image samples of the bridge concrete defect multi-label classification dataset;

[0009] S2: Model construction

[0010] Insert the SCARM and SPPF modules into the original EfficientNetV2 model, build an improved EfficientNetV2 model, determine the baseline model, and perform the above improvements on the baseline model to obtain a multi-label classification model for bridge concrete defects;

[0011] S3: Loss function optimization

[0012] An asymmetric loss function is used to replace the cross entropy loss function, and then the loss function is optimized. The asymmetric loss function is used to train the multi-label classification model of bridge concrete defects, and a multi-label classification model of bridge concrete defects that meets the performance indicators after training is obtained;

[0013] S4: Output the test results

[0014] The image to be classified is input into the multi-label classification model of bridge concrete defects that meets the performance indicators after training to obtain the multi-label classification results.

[0015] Furthermore, in step S1, the preprocessing process is as follows:

[0016] S11: Perform data augmentation on image samples of the bridge concrete defect multi-label classification dataset;

[0017] S12: Balance the number of image samples for each category in the dataset by replicating fewer categories;

[0018] S13: Divide the training set, validation set and test set into a set according to the set ratio;

[0019] S14: Adjust the image sizes in the training set, validation set, and test set to the set sizes.

[0020] Furthermore, in step S11, the categories of the image samples include no defect, crack, spalling, corrosion, wind separation and exposed steel bars.

[0021] Furthermore, in the step S2, based on the original EfficientNetV2 model, improvements are made on the original backbone network EfficientNetV2, and the SCARM module and the SPPF module are sequentially added before the classification layer.

[0022] Furthermore, the SCARM module includes a SARM module and a CARM module. For the feature map output by the backbone network, it is first processed by the SARM module to obtain a spatially refined attention feature map, and then processed by the CARM module to obtain a channel-refined attention feature map.

[0023] Furthermore, in the SARM module, the specific processing process is as follows:

[0024] S211: Given an intermediate feature map Where N is the number of samples in the batch, C is the number of channels, H and W are the feature height and width respectively;

[0025] S212: Normalize the feature map F as follows:

[0026]

[0027] Among them, μ and σ are the mean and standard deviation of the feature map F, ε is a positive constant, γ and β are trainable affine transformations;

[0028] S213: Normalize γ by the following formula to obtain the weight value

[0029]

[0030] S214: Use the sigmoid function to convert the weight value Wγ Map to the range of (0, 1), and perform separation operations by setting thresholds. Set the weights above the threshold to 1 to obtain the rich information weight W1, and set the weights below the threshold to 0 to obtain the redundant information weight W2. Multiply the original input feature map F with the weighted feature weights W1 and W2 to obtain the rich information representation F1 and the redundant information representation F2. The specific operations are as follows:

[0031] W i =Gate(Sigmoid(Wγ(GN(F)))),i=1,2

[0032] F i =F×W i ,i=1,2;

[0033] S215: Divide F1 and F2 into two parts F 11 、F 12 and F 21 、F 22 , then F 11 With F 22 Add, F 12 With F 21 Add together to get the reconstructed and Finally and The two parts are spliced ​​to obtain the spatial refinement feature F'. The specific operation process is as follows:

[0034] F1=F 11 ∪F 12 ,F2=F 21 ∪F 22 ;

[0035]

[0036] S216: Perform average pooling and maximum pooling operations on the spatial refinement feature F' along the channel axis to generate two context representations on different channels, and use the convolution layer to generate a spatial attention map Finally, the spatial refinement feature F' and M s (F) Multiply to obtain spatial attention reconstruction features That is, we get a spatially refined attention feature map. The specific operations are as follows:

[0037]

[0038] Furthermore, in the SARM module, the specific processing process is as follows:

[0039] S221: The spatial attention reconstruction feature F″ is divided into the upper X up and lower X down ;

[0040] S222: For upper X up , the GWC convolution layer and the 1×1 PWC convolution layer are used for processing, and the two parts of the output are added and merged to form the upper layer feature X1. The specific operation is as follows:

[0041] X1=COV GWC (X up )+COV PWC (X up )

[0042] Among them, COV GWC and COV PWC Represent GWC and PWC convolution operations respectively;

[0043] S223: For the lower X down , processed by a 1×1 PWC convolution layer and compared with the original X down Concatenate to get the lower-level feature X2. The specific operation is as follows:

[0044] X2=COV PWC (X down )∪X down ;

[0045] S224: Perform global average pooling on X1 and X2, and concatenate the obtained channel importance description vectors and perform SoftMax to generate feature importance vectors β1 and β2. Under the guidance of feature importance vectors β1 and β2, merge the upper-layer feature X1 and the lower-layer feature X2 to obtain the channel refinement feature X. The specific operations are as follows:

[0046] X=β1X1+β2X2;

[0047] S225: First, the average pooling features and the maximum pooling features are generated through the AvgPool layer and the MaxPool layer, and then sent to the MLP with one hidden layer. Then, the main elements of the output processed by the MLP are added to merge the output channel attention map Then the generated channel refinement feature X and channel attention map Multiply to obtain channel attention reconstruction features That is, the channel-refined attention feature map is obtained. The specific operations are as follows:

[0048] M s (X)=sigmoid(MLP(AvgPool(X))+MLP(MaxPool(X)));

[0049] X′=X×M s (X).

[0050] Furthermore, in the SPPF module, the feature map is sequentially subjected to three layers of maximum pooling operations with a size of 5×5, a step size of 1, and a padding of 2; then the result features output by each layer of maximum pooling operation will be spliced ​​into a feature vector of a fixed size.

[0051] Furthermore, in step S2, the selected benchmark model is the EfficientNetV2-s model.

[0052] Furthermore, in step S3, the asymmetric loss function is defined as follows:

[0053]

[0054] Among them, L + and L - are the positive loss and negative loss parts respectively, p is the label probability predicted by the network, γ + and γ - are the positive and negative focusing parameters, respectively, and p m is the introduced probability offset mechanism, and m is the probability margin.

[0055] Compared with the prior art, the present invention has the following advantages: the multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model, by adding the SCARM module to the original EfficientNetV2 network, removes redundant information in space and channels from the feature map extracted by the network, so as to reduce redundant information in the model training process, and promotes the learning of representative features through the attention mechanism; in view of the fact that the images in the data set have different sizes and aspect ratios, the SPPF module is used before the classification layer for multi-scale feature fusion, which can help the network extract spatial feature information of various sizes, thereby improving the robustness of the model to object deformation; in order to solve the problem of imbalance between positive and negative samples in the data set, an asymmetric loss function is used for training, which can dynamically adjust the weights and hard thresholds to reduce the influence of simple negative samples, and can filter out samples that may be incorrectly labeled. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of a multi-label classification method for bridge concrete defects based on the EfficientNetV2 model in an embodiment of the present invention;

[0057] FIG2( a ) is an example diagram of a sample of a normal background data set in the CODEBRIM data set according to an embodiment of the present invention;

[0058] FIG2( b ) is an example diagram of a crack defect dataset sample in the CODEBRIM dataset according to an embodiment of the present invention;

[0059] FIG2( c ) is an example diagram of a sample of a spalling defect dataset in the CODEBRIM dataset according to an embodiment of the present invention;

[0060] FIG2( d ) is an example diagram of a weathering defect dataset sample in the CODEBRIM dataset according to an embodiment of the present invention;

[0061] FIG2( e ) is an example diagram of a sample of a steel bar exposed defect dataset in the CODEBRIM dataset according to an embodiment of the present invention;

[0062] FIG2( f ) is an example diagram of a corrosion defect dataset sample in the CODEBRIM dataset according to an embodiment of the present invention;

[0063] Figure 3 is a schematic diagram of the structure of the improved EfficientNetV2 model in an embodiment of the present invention;

[0064] Figure 4 Schematic diagram of the structure of the SARM module in the SCARM module in the embodiment of the present invention;

[0065] Figure 5 Schematic diagram of the structure of the CARM module in the SCARM module in an embodiment of the present invention;

[0066] Figure 6 is a schematic diagram of the structure of the SPPF module in an embodiment of the present invention;

[0067] FIG. 7( a ) is an original image of a sample example of a visualization effect image in an embodiment of the present invention, which includes three defects: exposed steel bars, peeling, and corrosion;

[0068] FIG7( b ) is a visualization diagram of the exposed steel bar defect in FIG7( a );

[0069] FIG7( c ) is a visualization diagram of the spalling defect in FIG7( a );

[0070] Figure 7(d) is a visualization of the corrosion defect in Figure 7(a). DETAILED DESCRIPTION

[0071] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0072] This embodiment provides a technical solution: a multi-label classification method for bridge concrete defects based on an improved EfficientNetV2 model, comprising the following steps:

[0073] S1: Sample pretreatment

[0074] In step S1, the following contents are included:

[0075] S11: Obtain a multi-label classification dataset of bridge concrete defects (CODEBRIM), and perform necessary preprocessing on the images in the dataset according to the task requirements, including random cropping, rotation, scaling, brightness adjustment, and mosaic enhancement. The dataset includes 2507 images with cracks, 1898 images with spalling defects, 833 images with weathering defects, 1507 images with exposed bars, and 1559 images with corrosion defects. The typical five types of defect samples, namely cracks, spalling, corrosion, efflorescence, and exposed bars, are shown in the following examples: Figure 2(b)-2(f) As shown in Table 1, there are some imbalances between the categories in the dataset, so the number of image samples of each category in the dataset is balanced by duplicating fewer categories. After expansion, there are a total of 10,458 images. The data distribution of the training set, validation set, and test set is shown in Table 1.

[0076] S12: The image sizes of the training set, validation set, and test set are uniformly adjusted to 480×480. The purpose of unifying the size of all images in the training set, validation set, and test set is to adapt to the input size requirements of the model. Different input sizes will make it impossible for the computing device to build a unified batch for training.

[0077] Table 1 Dataset distribution

[0078]

[0079] S2: Insert the spatial and channel attention reconstruction module (SCARM) and the spatial pyramid feature fusion mechanism (SPPF) module into the original EfficientNetV2 model to construct the improved EfficientNetV2 network model.

[0080] In step S2, the following contents are included:

[0081] S21: Based on the original EfficientNetV2 model, improvements are made on the original backbone network EfficientNetV2. The SCARM module and SPPF module are added before the classification layer (classifier) ​​in sequence. The redundant information in space and channels of the feature map extracted by the backbone network EfficientNetV2 is removed to reduce the redundant information in the model training process, and the learning of representative features is promoted through the attention mechanism. The SPPF module can effectively solve the aspect ratio and size difference problems in the data set, and improve the model accuracy of the concrete defect classification network.

[0082] S22: EfficientNetV2(s, m, l) was compared on the dataset and it was found that EfficientNetV2-s performed best, so it was used as the benchmark model for multi-label classification of concrete defects.

[0083] Specifically, in step S21, the SCARM module is a module designed to reduce the redundant information interference of the bridge concrete defect data set and add an attention mechanism. It can suppress the redundant information in space and channels of the feature map extracted by the network to reduce the interference of redundant information in the model training process and promote the learning of representative features through the attention mechanism. SCARM consists of two parts: the spatial attention reconstruction module (SARM) and the channel attention reconstruction module (CARM). Specifically, for the feature map output by the network, we first use the SARM module to obtain the spatially refined attention feature map, and then use the CARM module to obtain the channel-refined attention feature map to reduce information redundancy and enhance the feature representation ability of the network model.

[0084] As more specifically, in step S21, in the SARM module, given an intermediate feature map Where N is the number of samples in the batch, C is the number of channels, H and W are the feature height and width respectively. First, the feature map F is normalized (GN) as follows:

[0085]

[0086] Among them, μ and σ are the mean and standard deviation of the feature map F, ε is a small positive constant, and γ and β are trainable affine transformations.

[0087] The present invention uses trainable parameters As an indicator for measuring the pixel variance on each channel, the richer the spatial information, the larger the corresponding γ value will be. Then γ is normalized by the following formula to obtain the weight value It indicates the importance of different channels in the feature map.

[0088]

[0089] Then use the sigmoid function to convert the weight value W γ Mapped to the range of (0, 1), and separated by setting a threshold (the threshold is set to 0.5 in this embodiment). The present invention sets the weights above the threshold to 1 to obtain the rich information weight W1, and sets the weights below the threshold to 0 to obtain the redundant information weight W2. Then, the original input feature map F is multiplied by the weighted feature weights W1 and W2 to obtain the rich information representation F1 and the redundant information representation F2. The operation process is shown in the following equation:

[0090] W i =Gate(Sigmoid(Wγ(GN(F))))

[0091] F i =F×W i ,i=1,2

[0092] It should be noted that in the reconstruction operation, the rich information representation F1 and the redundant information representation F2 are not directly added together, but a cross-reconstruction operation is used to fuse different information features. First, we divide F1 and F2 into two parts F 11 、F 12 and F 21 、F 22 Then, F 11 With F 22 Add, F 12 With F 21 Add together to get the reconstructed and Finally and The two parts are spliced ​​to obtain the spatial refinement feature F'. The whole process can be expressed as follows:

[0093] F1=F 11 ∪F 12 ,F2=F 21 ∪F 22

[0094]

[0095]

[0096] We then perform average pooling and maximum pooling operations on the spatially refined features F' along the channel axis to generate two contextual representations on different channels.

[0097] Next, we use a convolutional layer to generate a spatial attention map Finally, the spatial refinement feature F' and M s (F) Multiply to obtain spatial attention reconstruction features The operation process can be expressed as follows.

[0098]

[0099] F″=F ′ ×M s (F ′ )

[0100] More specifically, in step S21, in the CARM module, for the spatial attention reconstruction feature F″ after SARM processing, we first divide F″ into the upper X up and lower X down .X up As the input of the upper conversion stage, its role is to act as a "rich feature extractor", using GWC convolution layer and 1×1 PWC convolution layer to extract high-level representative information and reduce computational costs, replacing the expensive standard k×k convolution. The GWC convolution layer can reduce the number of parameters and computations by grouping channels and then convolving, but it will cut off the information flow between channels, while the PWC convolution layer can make up for the information loss and help ensure the integrity of the information on the channel. We add and merge the two outputs to form the representative feature X1 (upper layer feature), and the whole process can be expressed as:

[0101] X1=COV GWC (X up )+COV PWC (X up )

[0102] Among them, COV GWC and COV PWC Represent GWC and PWC convolution operations respectively.

[0103] X down After a 1×1 PWC operation, a feature map with shallow semantic representation is generated and compared with the original X down The output X2 (lower layer features) is concatenated to supplement the feature extraction part. The operation process is as follows:

[0104] X2=COV PWC (X down )∪X down

[0105] In the fusion part, firstly, global average pooling is performed on X1 and X2, and the obtained channel importance description vector (the vector here is the result of global average pooling of X1 and X2) is concatenated and then SoftMax is performed to generate feature importance vectors β1 and β2. Finally, under the guidance of feature importance vectors β1 and β2, the upper layer feature X1 and the lower layer feature X2 are merged to obtain the channel refinement feature X. The specific operation is as follows:

[0106] X=β1X1+β2X2

[0107] We then first generate the average pooling feature and the maximum pooling feature through the AvgPool layer and the MaxPool layer, and feed these two context descriptors into a multi-layer perceptron (MLP) with one hidden layer. We then perform principal element addition on the outputs processed by the MLP to merge the output channel attention maps. Finally, we combine the generated channel refinement feature X with the channel attention map Multiply to obtain channel attention reconstruction features The operation process is as follows:

[0108] M s (X)=sigmoid(MLP(AvgPool(X))+MLP(MaxPool(X)))

[0109] X′=X×M s (X)

[0110] More specifically, in step S21, the SPPF module sequentially processes the feature map through three layers of maximum pooling operations of size 5×5, with a step size of 1 and a padding of 2. Then, the resulting features output by each layer of maximum pooling operations are concatenated into a feature vector of a fixed size. This method not only solves the problem of inconsistent image sizes, but also helps the neural network capture global and local information of different scales.

[0111] More specifically, in step S22, a cosine learning rate scheme is adopted, the initial learning rate is set to 0.001, the learning rate is reduced by a factor of 10 every 30 training cycles, the batch size is set to 128, and a total of 300 rounds of training are performed.

[0112] More specifically, in step S22, accuracy and average accuracy are used as evaluation indicators, and a baseline model is determined after experimental comparison.

[0113] S3: Optimizing loss functions for multi-label classification

[0114] In step S3, the following contents are included:

[0115] S31: It is found that the loss function based on cross entropy cannot solve the problem of imbalance between positive and negative samples in multi-label classification problems. It is planned to use asymmetric loss function (ASL) as an improved method for optimization. The specific form of ASL has been explained above. ASL has the following characteristics: This loss function can dynamically adjust weights and hard thresholds to reduce the influence of simple negative samples, and can filter samples that may be mislabeled.

[0116] As more specific, the asymmetric loss function is defined as follows:

[0117]

[0118] Among them, L + and L - are the positive loss and negative loss parts respectively, p is the label probability predicted by the network, γ + and γ - are the positive and negative focusing parameters, respectively, and p m is the introduced probability offset mechanism, and m is the probability margin.

[0119] S32: The performance of the model after adding the SCARM module, SPPF module and ASL is verified on an independent test set. The results are shown in Table 2. It can be seen from the table that the above improvements are helpful in improving the model performance.

[0120] Table 2 Results of performance improvement of the model by three improvements

[0121]

[0122]

[0123] S4: Verify the performance of the proposed model on an independent test set. The detailed plan is as follows:

[0124] S41: In order to evaluate the effectiveness of the present invention, the accuracy of the model of the present invention (ours) was statistically analyzed on the validation set and the independent test set and compared with other detection methods. The test results are shown in Table 3:

[0125] Table 3 Comparison of the method of the present invention with other defect classification methods

[0126]

[0127] It can be seen from Table 3 that the multi-label classification model for bridge concrete defects designed in the present invention (improved EfficientNetV2-s model) achieved a classification accuracy of 79.91% on the test set, which is ahead of other models.

[0128] S42: In order to understand the classification ability of the network model for various defect categories, we conducted further experiments. Table 4 gives the classification accuracy of a single class.

[0129] Table 4 Classification accuracy of single-class defects in the present invention

[0130]

[0131]

[0132] In summary, the multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model in the above embodiment performs necessary preprocessing on the images in the data set, including random cropping, rotation, scaling, brightness adjustment, and mosaic enhancement of the images, and then designs a SCARM module to handle feature redundancy and apply an attention mechanism. Then, an SPPF module is added to the model to fuse multi-scale features and solve the problem of inconsistent image size and aspect ratio in the data set; in addition, the ASL loss function is used to train the model, which can reduce the imbalance problem of positive and negative samples in multi-label classification. Finally, the performance of different models is verified and compared on an independent test set, and the results show that our proposed method has better performance in the multi-label classification task of concrete defects.

[0133] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model, characterized in that: The following steps are involved: S1: Sample pretreatment Preprocess the image samples of the bridge concrete defect multi-label classification dataset; S2: Model construction Insert the SCARM and SPPF modules into the original EfficientNetV2 model, build an improved EfficientNetV2 model, determine the baseline model, and perform the above improvements on the baseline model to obtain a multi-label classification model for bridge concrete defects; S3: Loss function optimization An asymmetric loss function is used to replace the cross entropy loss function, and then the loss function is optimized. The asymmetric loss function is used to train the multi-label classification model of bridge concrete defects, and a multi-label classification model of bridge concrete defects that meets the performance indicators after training is obtained; S4: Output the test results The image to be classified is input into the multi-label classification model of bridge concrete defects that meets the performance indicators after training to obtain the multi-label classification results.

2. According to claim 1, a multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model is characterized in that: In step S1, the preprocessing process is as follows: S11: Perform data augmentation on image samples of the bridge concrete defect multi-label classification dataset; S12: Balance the number of image samples for each category in the dataset by replicating fewer categories; S13: Divide the training set, validation set and test set into a set according to the set ratio; S14: Adjust the image sizes in the training set, validation set, and test set to the set sizes.

3. According to claim 2, a multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model is characterized in that: In step S11, the categories of image samples include no defect, crack, peeling, corrosion, wind separation and exposed steel bars.

4. According to claim 1, a multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model is characterized in that: In the step S2, based on the original EfficientNetV2 model, improvements are made on the original backbone network EfficientNetV2, and the SCARM module and the SPPF module are sequentially added before the classification layer.

5. The multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model according to claim 4 is characterized in that: The SCARM module includes a SARM module and a CARM module. For the feature map output by the backbone network, the SARM module is first used to obtain a spatially refined attention feature map, and then the CARM module is used to obtain a channel-refined attention feature map.

6. The multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model according to claim 5 is characterized in that: In the SARM module, the specific processing process is as follows: S211: Given an intermediate feature map Where N is the number of samples in the batch, C is the number of channels, H and W are the feature height and width respectively; S212: Normalize the feature map F as follows: Among them, μ and σ are the mean and standard deviation of the feature map F, ε is a positive constant, γ and β are trainable affine transformations; S213: Normalize γ by the following formula to obtain the weight value S214: Use the sigmoid function to convert the weight value W γ Map to the range of (0, 1), and perform separation operations by setting thresholds. Set the weights above the threshold to 1 to obtain the rich information weight W1, and set the weights below the threshold to 0 to obtain the redundant information weight W2. Multiply the original input feature map F with the weighted feature weights W1 and W2 to obtain the rich information representation F1 and the redundant information representation F2. The specific operations are as follows: W i =Gate(Sigmoid(Wγ(GN(F)))) F i =F×W i ,i=1,2; S215: Divide F1 and F2 into two parts F 11 、F 12 and F 21 、F 22 , then F 11 With F 22 Add, F 12 With F 21 Add together to get the reconstructed and Finally and The two parts are spliced ​​to obtain the spatial refinement feature F'. The specific operation process is as follows: F1=F 11 ∪F 12 ,F2=F 21 ∪F 22 ; S216: Perform average pooling and maximum pooling operations on the spatial refinement feature F' along the channel axis to generate two context representations on different channels, and use the convolution layer to generate a spatial attention map Finally, the spatial refinement feature F' and M s (F) Multiply to obtain spatial attention reconstruction features That is, we get a spatially refined attention feature map. The specific operations are as follows: F″=F ′ ×M s (F ′ )。 7. The multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model according to claim 6 is characterized in that: In the SARM module, the specific processing process is as follows: S221: The spatial attention reconstruction feature F″ is divided into the upper X up and lower X down ; S222: For upper X up , the GWC convolution layer and the 1×1 PWC convolution layer are used for processing, and the two parts of the output are added and merged to form the upper layer feature X1. The specific operation is as follows: X1=COV GWC (X up )+COV PWC (X up ) Among them, COV GWC and COV PWC Represent GWC and PWC convolution operations respectively; S223: For the lower X down , processed by a 1×1 PWC convolution layer and compared with the original X down Concatenate to get the lower-level feature X2. The specific operation is as follows: X2=COV PWC (X down )∪X down ; S224: Perform global average pooling on X1 and X2, and concatenate the obtained channel importance description vectors and perform SoftMax to generate feature importance vectors β1 and β2. Under the guidance of feature importance vectors β1 and β2, merge the upper-layer feature X1 and the lower-layer feature X2 to obtain the channel refinement feature X. The specific operations are as follows: X=β1X1+β2X2; S225: First, the average pooling features and the maximum pooling features are generated through the AvgPool layer and the MaxPool layer, and then sent to the MLP with one hidden layer. Then, the main elements of the output processed by the MLP are added to merge the output channel attention map Then the generated channel refinement feature X and channel attention map Multiply to obtain channel attention reconstruction features That is, the channel-refined attention feature map is obtained. The specific operations are as follows: M s (X)=sigmoid(MLP(AvgPool(X))+MLP(MaxPool(X))); X′=X×M s (X)。 8. The multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model according to claim 7 is characterized in that: In the SPPF module, the feature map is sequentially subjected to three layers of maximum pooling operations with a size of 5×5, a step size of 1, and a padding of 2; then the resulting features output by each layer of maximum pooling operation will be concatenated into a feature vector of a fixed size.

9. The multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model according to claim 1 is characterized in that: In step S2, the selected benchmark model is the EfficientNetV2-s model.

10. The multi-label classification method for bridge concrete defects based on the improved EfficientNetV2 model according to claim 1 is characterized in that: In step S3, the asymmetric loss function is defined as follows: Among them, L + and L - are the positive loss and negative loss parts respectively, p is the label probability predicted by the network, γ + and γ - are the positive and negative focusing parameters, respectively, and p m is the introduced probability offset mechanism, and m is the probability margin.

Citation Information

Cited By

  • Bridge concrete apparent damage identification method and system based on texture analysis

    CN121811260A

  • Bridge concrete surface damage identification method and system based on texture analysis

    CN121811260B