Tunnel bolt corrosion area identification method and system

Through the improved tunnel bolt corrosion detection model of encoder-decoder structure and attention mechanism, the problem of high miss detection and error detection rate of bolt corrosion detection in the existing technology is solved, and accurate identification and efficient detection of bolt corrosion areas are achieved. .

CN119048880APending Publication Date: 2024-11-29JINAN SHENGYUAN DETECTION TECHNOLOGY CO LTD +1

Patent Information

Application Number
CN202411138420.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The existing bolt corrosion detection methods have the problem of high missed detection rates and error detection rates, especially when the bolt corrosion areas are small, traditional image processing and deep learning models are difficult to identify smaller corrosion defects, and the sensor costs are high and the maintenance is complex. , manual inspection is inefficient.

Method used

Using the improved encoder-decoder structure, multi-level depth feature extraction is performed through CBL blocks, maximum pooling layer, dense connection module Dense Block and Transition connection blocks, and an attention mechanism is introduced into the decoder. Perform upsampling and feature fusion, dynamically adjust the weight of the feature map, and realize pixel-level segmentation of the bolt rust area.

Benefits of technology

It improves the accuracy and efficiency of bolt corrosion detection, can effectively identify smaller defect areas, reduces the computational complexity and overfitting risks, and enhances the generalization ability and operation efficiency of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048880A_ABST
    Figure CN119048880A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of tunnel safety and defect identification, and provides a tunnel bolt corrosion area identification method and system.The method comprises the steps that firstly, a tunnel bolt corrosion detection model adopting an encoder-decoder structure is constructed, and an encoder backbone network of the improved tunnel bolt corrosion detection model comprises a maximum pooling layer; the dense connection modules and the Transformation connection blocks are sequentially and alternately connected; the Transformation connection block is connected with two adjacent dense connection modules and is used for controlling the size and the channel number of the feature map; and carrying out up-sampling in a decoder, dynamically adjusting the weight of a space channel feature map based on an attention mechanism, and carrying out feature fusion on the feature map based on the weight and depth features output by a corresponding layer of an encoder to obtain a bolt corrosion region segmentation map. The constructed tunnel bolt corrosion detection model can realize pixel-level segmentation of a bolt corrosion part, and can achieve a better prediction result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical fields related to tunnel safety and defect identification, and in particular to a method and system for identifying rusted areas of tunnel bolts. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] In modern tunnel engineering, bolts are key components for connecting and fixing structural parts. Their safety and stability have a vital impact on the overall safety of the tunnel. However, as the tunnel ages, bolts are susceptible to rust due to erosion by environmental factors (such as moisture, corrosive media, etc.), which leads to a decrease in bolt performance and even safety hazards. Therefore, accurate and rapid detection of the degree of rust in tunnel bolts is of great significance to ensure the safe operation of the tunnel.

[0004] Traditional bolt corrosion detection methods mainly rely on manual inspection and sensor technology. Manual inspection usually requires professionals to check the bolts one by one visually or with simple tools. This method is not only time-consuming and labor-intensive, but also due to human subjective factors, the accuracy and consistency of the detection results are low. In the short 3-hour tunnel maintenance window, even skilled tunnel inspection workers can only inspect nearly 3 kilometers. Manual inspection is inefficient, with a high missed detection rate and a high false alarm rate. Sensor technology monitors the corrosion status of bolts through sensors installed on them, but this method is costly and complex to maintain, and the reliability and life of the sensors themselves may also be affected by the environment.

[0005] With the development of computer vision technology, bolt corrosion detection methods based on image processing have gradually attracted attention. These methods usually use camera acquisition equipment to capture images of tunnel bolts, and then identify the rusted areas on the bolt surface through traditional image processing techniques such as image segmentation, edge detection and morphological operations. Traditional image processing algorithms rely on manually designed features and hard-coded algorithms, which may show low performance under changes in lighting, direction and viewpoint, limiting the anti-interference ability of tunnel bolt detection.

[0006] In recent years, deep learning has made significant progress in the fields of image processing and computer vision. By building a multi-layer neural network, deep learning methods can automatically learn features from data, effectively overcoming the limitations of manually designed features in traditional methods. However, bolt corrosion detection based on deep learning has the problem of high missed detection rate and false detection rate. When the bolt corrosion area is very small, the corrosion feature accounts for too small a proportion in the entire image. After a series of convolution and pooling operations, the features of the corrosion area may disappear or be ignored. It can be seen that the existing deep learning model cannot identify smaller corrosion areas, and there are problems in identifying smaller corrosion defects on bolts. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes a tunnel bolt corrosion area identification method and system. The constructed tunnel bolt corrosion detection model can realize pixel-level segmentation of bolt corrosion areas and achieve better prediction results.

[0008] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:

[0009] One or more embodiments provide a method for identifying a tunnel bolt corrosion area, comprising the following steps:

[0010] Acquire a bolt image to be identified;

[0011] The image is input into the improved tunnel bolt corrosion detection model with encoder-decoder structure, and multi-level deep feature extraction is performed through the encoder backbone network;

[0012] The encoder backbone network of the improved tunnel bolt corrosion detection model includes a CBL block, a maximum pooling layer, and multiple dense connection modules Dense Block and Transition connection blocks that are alternately connected in sequence; the Transition connection block connects two adjacent dense connection modules Dense Block to control the size and number of channels of the feature map;

[0013] The attention mechanism is introduced into the decoder to upsample the extracted multi-level deep features, and the attention mechanism is used to dynamically adjust the weights of the spatial and channel feature maps to obtain the weighted feature maps after weighting.

[0014] The weighted feature map is fused with the deep features output by the corresponding layer of the encoder to obtain a feature map with the same size as the input image and then converted to obtain a segmentation map of the bolt corrosion area.

[0015] One or more embodiments provide a tunnel bolt corrosion area identification system, including:

[0016] An acquisition module: configured to acquire an image of a bolt to be identified;

[0017] Feature extraction module: configured to input the image into the improved tunnel bolt corrosion detection model with encoder-decoder structure, and perform multi-level deep feature extraction through the encoder backbone network;

[0018] The encoder backbone network of the improved tunnel bolt corrosion detection model includes a CBL block, a maximum pooling layer, and multiple dense connection modules and transition connection blocks that are alternately connected in sequence; the transition connection block connects two adjacent dense connection modules to control the size and number of channels of the feature map;

[0019] Upsampling module: It is configured to introduce an attention mechanism in the decoder, upsample the extracted multi-level deep features, and use the attention mechanism to dynamically adjust the weights of the spatial and channel feature maps to obtain weighted feature maps after weighting;

[0020] Fusion recognition module: It is configured to fuse the weighted feature map with the deep features output by the corresponding layer of the encoder to obtain a feature map with the same size as the input image and convert it to obtain a segmentation map of the bolt corrosion area.

[0021] An electronic device comprises a memory and a processor and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the above-mentioned method for identifying the rusted area of ​​a tunnel bolt are completed.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] In the present disclosure, a tunnel bolt corrosion detection model is constructed, and the backbone network of the encoder includes a structure in which dense connection modules Dense Block and Transition connection blocks are alternately connected. Based on the dense connection module Dense Block, features are extracted from the input image. While using fewer parameters, a high accuracy rate can be maintained, and features can be effectively reused, and smaller defect areas can be effectively identified, thereby avoiding the problem of information and gradient disappearance, reducing overfitting, and increasing the generalization ability of the model; based on the Transition connection block, the dimension of the feature map can be reduced, thereby reducing the computational complexity; thereby improving the feature extraction effect and maintaining the operating efficiency of the model system.

[0024] The advantages of the present disclosure and the advantages of additional aspects will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and the description thereof are used to explain the present disclosure but do not constitute a limitation of the present disclosure.

[0026] Figure 1 This is a flow chart of a tunnel bolt corrosion detection method according to Embodiment 1 of the present disclosure;

[0027] Figure 2 This is an example schematic diagram of a bolt rust area marked in red in Example 1 of the present disclosure;

[0028] Figure 3 This is a schematic diagram of the overall structure of the tunnel bolt corrosion detection model of Example 1 of the present disclosure;

[0029] Figure 4 This is a Dense Block structure diagram of Example 1 of the present disclosure;

[0030] Figure 5 This is a structural diagram of a Dense layer module of Embodiment 1 of the present disclosure;

[0031] Figure 6 This is a structural diagram of a Transition connection block in Embodiment 1 of the present disclosure;

[0032] Figure 7 This is a diagram of an upsampling structure of a decoder according to Embodiment 1 of the present disclosure;

[0033] Figure 8 It is a structural diagram of the CBAM attention mechanism module of Example 1 of the present disclosure;

[0034] Fig. 9 This is an algorithm flow chart of the CBAM attention mechanism of Embodiment 1 of the present disclosure;

[0035] Fig.10 Schematic diagram of the model training method flow of embodiment 1 of the present disclosure. DETAILED DESCRIPTION

[0036] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0037] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0038] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof. It should be noted that, in the absence of conflict, the various embodiments in the present disclosure and the features in the embodiments can be combined with each other. The embodiments will be described in detail below in conjunction with the accompanying drawings.

[0039] Example 1

[0040] In the technical solutions disclosed in one or more embodiments, Figures 1 to 10 As shown, a method for identifying a tunnel bolt corrosion area comprises the following steps:

[0041] Step 1: Obtain the bolt image to be identified;

[0042] Step 2: Input the image into the improved tunnel bolt corrosion detection model using an encoder-decoder structure, and perform multi-level deep feature extraction through the encoder backbone network;

[0043] The encoder backbone network of the improved tunnel bolt corrosion detection model includes a CBL block, a maximum pooling layer, and multiple dense connection modules Dense Block and Transition connection blocks that are alternately connected in sequence; the Transition connection block connects two adjacent dense connection modules Dense Block to control the size and number of channels of the feature map;

[0044] Step 3: Introduce the attention mechanism in the decoder, upsample the extracted multi-level deep features, and use the attention mechanism to dynamically adjust the weights of the spatial and channel feature maps to obtain the weighted feature maps after weighting;

[0045] Step 4: Fuse the weighted feature map with the depth feature output by the corresponding layer of the encoder to obtain a feature map with the same size as the input image and convert it to obtain a bolt corrosion area segmentation map;

[0046] In this embodiment, a tunnel bolt corrosion detection model is constructed, and the backbone network of the encoder includes a structure in which dense connection modules Dense Block and Transition connection blocks are alternately connected. Based on the dense connection module Dense Block, features are extracted from the input image. While using fewer parameters, a high accuracy rate can be maintained, and features can be effectively reused. Smaller defect areas can be effectively identified, and the problem of information and gradient disappearance can be avoided, overfitting can be reduced, and the generalization ability of the model can be increased. Based on the Transition connection block, the dimension of the feature map can be reduced, thereby reducing the computational complexity; thereby improving the feature extraction effect and maintaining the operating efficiency of the model system.

[0047] In some embodiments, the tunnel bolt corrosion detection model uses an encoder-decoder network, such as Figure 3 As shown in the figure, the encoder is composed of the backbone network Densenet121, and the decoder includes multiple layers of upsampling modules and convolutional layers. The upsampling module of the previous layer is connected to the convolutional layer of the next layer. The upsampled feature map is concatenated and fused with the feature map of the corresponding layer of the encoder through the convolution operation.

[0048] A possible structure such as Figure 3 As shown, the backbone network of the encoder is used to extract multi-layer features of the input image, including a CBL block, a maximum pooling layer (Maxpool), multiple dense connection modules (DenseBlock) and multiple Transition connection blocks connected in sequence; the dense connection modules are connected through the Transition connection blocks; the multiple dense connection modules output deep features respectively to obtain the extracted multi-level deep features;

[0049] Specifically, in this embodiment, 4 dense connection blocks (Dense Block) and 3 Transition connection blocks are set.

[0050] Specifically, CBL includes a convolutional layer, a batch normalization layer, and a ReLU activation function layer connected in sequence.

[0051] Furthermore, if Figure 4 As shown in the figure, the dense connection module Dense Block includes multiple cascaded Dense layer modules. Each Dense Layer module is directly connected to all the previous Dense Layer modules in the cascade order. The output features of the last Dense Layer module of the Denselayer module are concatenated with the input features of the Dense layer module to form the final output of the Dense Block of the corresponding encoder layer.

[0052] like Figure 4 As shown, the connection structure of the dense connection module Dense Block includes 4 Dense Layer modules;

[0053] Each layer in each densely connected block is directly connected to each other, which can make full use of the features of the previous layers, maximize the reuse of features, and alleviate the gradient vanishing problem.

[0054] Specifically, the structure of the Dense layer module is as follows: Figure 5 As shown, it includes an input layer (Input), a batch normalization layer (Batch Normalization), a ReLU non-linear activation layer, a 1×1 convolution layer, a batch normalization layer, a 3×3 convolution layer and an output layer connected in sequence.

[0055] In one achievable implementation, each Dense layer module processes input features, including the following steps:

[0056] Step 1.1, obtain input features and concatenate all input features;

[0057] Specifically, the input feature map includes the output of all previous layers in the current Dense Block. Assuming that the current DenseLayer is the lth layer, the input feature is [x0, x1, x2, ..., x l-1 ];

[0058] Step 1.2: The received data is batch normalized and ReLU nonlinearly activated, and then the first convolution operation is performed through a 1×1 convolutional layer to reduce the dimensionality of the data;

[0059] Specifically, the first convolution block BN_ReLU_Conv2D is used to perform standardization operations to improve training stability and speed, the ReLU activation function is used for nonlinear activation to increase the nonlinear representation ability of the network, and the 1×1 convolution layer is used to perform dimensionality reduction operations to reduce the dimension of the feature map, thereby reducing the computational complexity.

[0060] Step 1.3: Perform batch normalization on the reduced-dimensional data, and then perform feature extraction through the second convolution operation of the 3×3 convolution layer;

[0061] Step 1.4: Concatenate the features extracted in step 1.3 with the input features of the current Dense Layer module and transmit them to each Dense Layer module connected to the backend;

[0062] In this embodiment, specifically, the Dense Layer module first uses BN_ReLU to receive the feature map from the previous layer, then uses a 1×1 Conv2D to set the channel dimension of the feature map to 128, and then uses a 3×3 Conv2D to compress the feature map to 32 for use by the subsequently connected Dense layer module.

[0063] In a further technical solution, the number of Denselayers in the densely connected modules of different coding layers of the encoder is different, and increases according to a set multiple from low to high coding layers;

[0064] Specifically, the number of Dense layers in the four Dense Blocks in this embodiment is different, and the number of Dense layer modules in order is 6, 12, 24, and 16. The output features of the last Dense Layer are concatenated with the input features to form the final output of the Dense Block.

[0065] Optionally, the growth rate of the feature map during each Dense Block processing can be set, and the density of the feature map is increased according to the set growth rate;

[0066] A preferred technical solution is that the growth rate of the feature map in the Dense Block can be 32, and each DenseLayer adds 32 feature maps. If the number of input feature maps of the Dense Block is 64 (assuming the initial input) and the number of DenseLayer is 4, the number of output feature maps is: 64+4×32=192.

[0067] In the above scheme, the feature extraction of the encoder is realized, such as Figure 3 As shown, the feature map obtained after passing through the CBL block is F1, the feature map after passing through Dense Block1 is F2, the feature map after passing through Dense Block2 is F3, the feature map after passing through Dense Block3 is F4, and the feature map after passing through Dense Block4 is F5.

[0068] In this embodiment, a Transition connection block is used between every two Dense Blocks to control the size and number of channels of the feature map, thereby optimizing the complexity of the model and the computing resource requirements, while maintaining sufficient feature information for subsequent layers.

[0069] The main function of Transition is to connect two adjacent Dense Blocks to control the size and number of channels of the feature map to achieve model compression and reduce the amount of calculation. The Transition block between densely connected blocks uses BN_ReLU for standardization and Conv2D_Pool to adjust the number of feature channels and downsample the feature map for output.

[0070] In some embodiments, the structure of the Transition connection block can be as follows: Figure 6 As shown in the figure, it includes an input layer (Input), a batch normalization layer (Batch Normalization), a ReLU activation layer, a convolution layer Conv1×1 and an average pooling layer (AvgPool2×2) connected in sequence; the Transition connection block performs feature normalization on the input data through a batch normalization layer, increases its nonlinearity through the ReLU activation function, and then adjusts the number of channels of the feature map through a 1×1 convolution. Finally, a 2×2 average pooling layer is used to reduce the height and width of the feature map, further reducing the computational complexity of the model.

[0071] In this embodiment, Transition effectively connects two Dense Blocks through convolution and pooling operations, and controls the size and number of channels of the feature map to achieve model compression and reduce the amount of calculation, thereby improving the efficiency and performance of the model while ensuring network performance.

[0072] The backbone network of the encoder described above performs feature extraction on the input image and can maintain a high accuracy rate while using fewer parameters. It can effectively reuse features to avoid the problem of information and gradient disappearance, reduce overfitting, and increase the generalization ability of the model.

[0073] The multi-level deep features extracted in step 3 are input to the decoder for upsampling (up).

[0074] In some embodiments, the decoder includes multiple upsampling modules cascaded in sequence, the bottom-level upsampling module is connected to a convolution module (UpConv), the input of the upsampling module is connected to the feature output end of the corresponding layer of the encoder, the output features of the upsampling module are spliced ​​with the output features of the encoder of the previous layer, and the spliced ​​features are input to the upsampling module of the current layer for upsampling.

[0075] Each layer in the decoder is connected to the layer of the corresponding depth in the encoder through a skip connection. The skip connection passes the feature map of the encoder directly to the decoder, which can preserve the detailed information: the low-level feature map contains rich edge and texture information, which is essential for reconstructing high-resolution segmentation maps. At the same time, the high-level feature map provides global semantic information, while the low-level feature map retains the location and detail information. The skip connection enables the model to utilize both types of information at the same time, so that not only local details but also global context are taken into account during the reconstruction process.

[0076] Specifically, the jump connection concatenates the features F1, F2, F3, F4, and F5 extracted in step 2 with the feature map obtained by the upsampling operation to obtain a one-fold downsampled feature map, and then a convolution layer with a convolution module (UpConv) is used as the output layer to restore the size to the original image size and convert the feature map into a bolt corrosion area segmentation map.

[0077] Achievable, such as Figure 7 As shown in the figure, the upsampling operation includes the following steps: first, bilinear interpolation is performed to double the size of the feature map, and then it is concatenated with the feature map of the previous layer in the channel dimension. The concatenated feature map uses a 3×3 convolution to extract features, and then the CBAM attention mechanism is used to dynamically adjust the weight of the spatial channel feature map to increase the weight of the bolt corrosion feature, and then a convolution layer is used to extract features to obtain the output feature map.

[0078] The decoder of this embodiment generates an output of the same size as the input image through a series of upsampling operations and convolutional layers. At the same time, the decoder also fuses the feature maps of the corresponding layers of the encoder (skip connection) to retain and utilize low-level features, enhance bolt corrosion details and positioning information, and improve the accuracy and robustness of bolt corrosion segmentation.

[0079] Specifically, the implementation structure module of the CBAM attention mechanism is as follows Figure 8 As shown in Figure 1, it includes a channel attention module and a spatial attention module connected in sequence. These two mechanisms are applied in sequence to capture the importance of features in different dimensions.

[0080] The channel attention module adjusts the weight of the feature map by evaluating the importance of each channel. The input feature map (Input Feature) is subjected to global average pooling and global maximum pooling operations to obtain the global average and global maximum of each channel, respectively. Then, they are input into a shared multi-layer perceptron (MLP), which consists of two fully connected layers with a ReLU activation function in the middle. Finally, the outputs of the two MLPs are fused through element-by-element addition to obtain the channel attention weights. Finally, the channel attention weights are applied to the original feature map through element-by-element multiplication to generate a weighted feature map (Refined Feature).

[0081] The spatial attention module adjusts the weight of the feature map by evaluating the importance of each pixel. First, the input feature map is subjected to global maximum pooling and global average pooling in the channel dimension to obtain two two-dimensional feature maps. Then, the two two-dimensional feature maps are channel-wise concatenated and processed through a convolutional layer to generate spatial attention weights. Finally, the spatial attention weights are applied to the input feature map through element-by-element multiplication to generate the final weighted feature map.

[0082] A specific implementation method uses the CBAM attention mechanism to dynamically adjust the weights of the spatial channel feature maps to obtain a weighted feature map, such as Fig. 9 As shown, the following steps are included:

[0083] Step 3.1: The input features are pooled by global average pooling (Avgpool) and global maximum pooling (Maxpool) to obtain global information from two different perspectives from the feature map;

[0084] In this step, the input feature dimension of the input layer (Input) is H×W×C; the feature dimensions after pooling are 1×1×C;

[0085] Step 3.2, convert the two global information obtained into the importance weight of each channel through a multi-layer perceptron (MLP);

[0086] Step 3.3, add the importance weights element by element and apply the Sigmoid activation function to get the final channel attention weight matrix;

[0087] Step 3.4, perform matrix multiplication of the channel attention weight matrix and the input feature to obtain the first feature map adjusted in the channel dimension; the first feature map obtained in this step has the same dimension as the input feature, which is H×W×C;

[0088] Step 3.5, use the average pooling layer and the maximum pooling layer to compress the first feature map in the channel dimension, highlight the spatial features and obtain two second feature maps with dimensions of H×W×1;

[0089] Step 3.6: Concatenate the two output feature maps on the channel to obtain a feature map H×W×2, input it into the multi-layer perceptron for fusion, remap it to H×W×1 dimensional features, and then use the Sigmoid activation function to obtain the spatial attention weight matrix;

[0090] Step 3.7: Multiply the spatial attention weight matrix by the first feature map of the input to obtain the feature map adjusted in the channel and spatial dimensions, that is, the weight-based feature map.

[0091] In the upsampling stage of the model decoder of this embodiment, the feature maps of the encoder and decoder parts are fused through jump connections to improve the accuracy of segmentation. CBAM attention is introduced in the upsampling operation, the importance of each channel is obtained through global average pooling and global maximum pooling operations, and a spatial attention weight map is generated through the convolution layer. The CBAM attention mechanism focuses on different dimensions of the feature map respectively, thereby improving the network's attention to important features.

[0092] Further technical solutions also include the process of training the tunnel bolt corrosion detection model, such as Fig.10 As shown, the following steps are included:

[0093] Step S1, obtaining bolt corrosion images, performing preprocessing and annotation, and constructing a bolt corrosion dataset;

[0094] Step S2, the data of the bolt corrosion data set is transmitted to the constructed tunnel bolt corrosion detection model for training with the preprocessed bolt corrosion image as input to obtain a mask image of the corrosion area;

[0095] Step S3, comparing the mask image of the identified rusted area with the annotated data, calculating the loss function, adjusting the model parameters, and iterating until the training iteration cutoff condition is met to obtain a trained tunnel bolt corrosion detection model;

[0096] In step S1, specifically, a bolt corrosion data set is established, more than 1000 bolt corrosion color images are collected, and the collected images are preprocessed including rotation correction, translation scaling, and contrast adjustment to images of a set size, and the set size can be set to an image of 640×640 size to constitute the original image of the data set;

[0097] It is feasible to annotate the rust on each original image, determine the contour and range of the rust in the image, and generate label data; Figure 2 As shown in the figure, each tunnel image in the established tunnel bolt corrosion dataset has a corresponding mask label, and the bolt corrosion area is marked in red.

[0098] Specifically, use Labelme to mark the rusted area, obtain the label file in Json format, and then convert the Json file into a file with the same size as the original. Figure 1 The labeled PNG file is created, and the grayscale of the image is adjusted so that the pixel value of the background area is 0 and the pixel value of the rusted area is 1. The labeled image file is obtained, and multiple folders are constructed to store the dataset established in the VOC format:

[0099] JPEG Images folder: used to store the original images used to create the dataset in VOC format;

[0100] Segmentation Class folder: used to store labeled mask files in PNG format;

[0101] Set up a subfolder Segmentation under the ImageSets folder: It is used to store t×t files corresponding to the indexes of the training set and the validation set, and establish a one-to-one correspondence between the original image and the label file;

[0102] The ImageSets folder is used to provide the image list required for the training and validation stages;

[0103] In the task of detecting bolt corrosion in tunnels, there is a class imbalance problem between bolt corrosion areas and non-corrosion areas, that is, the proportion of pixels in the corrosion area is relatively small, while the non-corrosion area occupies most of the entire image. In order to solve the class imbalance, a multi-task learning strategy combining Dice Loss and Focal Loss is used to design the loss function, optimize the model training process, enhance the adaptability to unbalanced samples and difficult-to-classify samples, and obtain the optimal bolt corrosion detection model.

[0104] Dice Loss is good at dealing with class imbalance problems because it directly optimizes for overlapping areas and is insensitive to the imbalance of foreground and background categories. Dice Loss promotes the model to learn more precise boundaries by maximizing the intersection between the prediction and the true label, which is crucial for segmentation tasks.

[0105] Focal Loss strengthens difficult-to-classify samples, allowing the model to pay more attention to small targets and edge cases, while Dice Loss ensures that the model can accurately capture object boundaries in segmentation tasks. This combination is particularly beneficial for improving the performance of the model in small target segmentation and dense target detection.

[0106] Dice Loss can be expressed as follows:

[0107]

[0108] Among them, pi is the predicted value output by the model, which indicates the probability that the i-th pixel value is a rusted area; ti indicates the true category of the i-th pixel. The smaller the Dice Loss, the closer the model's prediction result is to the true label.

[0109] Focal Loss can solve the problem of category imbalance and effectively suppress the loss of positive samples that are easy to classify, thereby improving the performance of the network.

[0110] Focal Loss can be expressed as follows:

[0111] Focal Loss=-αt(1-pt)γlog(pt);

[0112] Among them, pt is the probability value predicted by the network, αt is the sample weight, which can be adjusted according to the degree of imbalance in the data set category, and γ represents the degree of focus of the loss function.

[0113] The scaling factor in Focal Loss (1-p t ) γ It can make the model focus on samples that are more difficult to classify, reduce the impact of easy-to-classify samples, and enhance the learning ability of difficult-to-classify samples.

[0114] The total loss function after this implementation is expressed as:

[0115] Loss=α×Dice Loss+β×Focal Loss;

[0116] Among them, α and β are the weight coefficients of Dice Loss and Focal Loss, which are used to balance the importance of the two loss functions.

[0117] This combined loss function can comprehensively consider the segmentation accuracy and class imbalance problem, thereby improving the performance and robustness of the network. Through such a loss function design, class imbalance and difficult-to-classify samples can be better handled in the tunnel bolt corrosion detection task, thereby improving the performance and generalization ability of the model.

[0118] Based on the above-trained tunnel bolt corrosion detection model, the segmented image of the bolt corrosion area can be obtained by inputting the tunnel image to be identified into the model, and the segmentation and positioning of the bolt corrosion part of the tunnel image can be realized. Compared with the existing methods, this method not only significantly improves the detection speed, but also maintains a high accuracy rate.

[0119] This embodiment improves the U-type network structure (UNet) based on the encoder-decoder structure, improves the backbone network structure of the encoder, continuously improves the density of the feature map, can continuously supplement the shallow resolution information for the deep semantic feature map, and realizes the retrieval of edge features through feature splicing. The segmentation and positioning of the rusted area is simple and efficient, and can be trained from a small data set to better complete the pixel-level positioning of the bolt rusted area. The degree of rust on the bolt is evaluated according to the area of ​​the rusted area to better assist manual inspections.

[0120] The method proposed in this embodiment is stronger than the original encoder and decoder U-type network structure (UNet) and Deeplabv3+ in bolt corrosion segmentation and positioning performance. The average pixel accuracy in bolt corrosion identification reaches 78.67%, and the average intersection over union ratio reaches 68.35%, which is a significant improvement compared with traditional methods, and the inference speed is greatly improved.

[0121] Among them, Deeplabv3+ is a convolutional neural network (CNN) architecture improved on the basis of Deeplabv3, which is specially used for image semantic segmentation. It improves the segmentation performance by introducing spatial pyramid pooling and encoder-decoder structure.

[0122] Example 2

[0123] Based on Example 1, this embodiment provides a tunnel bolt corrosion area identification system, including:

[0124] An acquisition module: configured to acquire an image of a bolt to be identified;

[0125] Feature extraction module: configured to input the image into the improved tunnel bolt corrosion detection model with encoder-decoder structure, and perform multi-level deep feature extraction through the encoder backbone network;

[0126] The encoder backbone network of the improved tunnel bolt corrosion detection model includes a maximum pooling layer, and multiple dense connection modules and transition connection blocks that are alternately connected in sequence; the transition connection block connects two adjacent dense connection modules to control the size and number of channels of the feature map;

[0127] Upsampling module: It is configured to introduce an attention mechanism in the decoder, upsample the extracted multi-level deep features, and use the attention mechanism to dynamically adjust the weights of the spatial and channel feature maps to obtain weighted feature maps after weighting;

[0128] Fusion recognition module: It is configured to fuse the weighted feature map with the deep features output by the corresponding layer of the encoder to obtain a feature map with the same size as the input image and convert it to obtain a segmentation map of the bolt corrosion area.

[0129] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation process is the same, which will not be repeated here.

[0130] Example 3

[0131] Based on Example 1, this embodiment provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, the steps in the tunnel bolt corrosion area identification method described in Example 1 are completed.

[0132] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

[0133] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A method for identifying the rusted area of ​​a tunnel bolt, characterized in that: The steps include: Acquire a bolt image to be identified; The image is input into the improved tunnel bolt corrosion detection model with encoder-decoder structure, and multi-level deep feature extraction is performed through the encoder backbone network; The encoder backbone network of the improved tunnel bolt corrosion detection model includes a CBL module, a maximum pooling layer, and multiple dense connection modules and transition connection blocks that are alternately connected in sequence; the transition connection block connects two adjacent dense connection modules to control the size and number of channels of the feature map; The attention mechanism is introduced into the decoder to upsample the extracted multi-level deep features, and the attention mechanism is used to dynamically adjust the weights of the spatial and channel feature maps to obtain the weighted feature maps after weighting. The weighted feature map is fused with the deep features output by the corresponding layer of the encoder to obtain a feature map with the same size as the input image and then converted to obtain a segmentation map of the bolt corrosion area.

2. A tunnel bolt corrosion area identification method according to claim 1, characterized in that: The backbone network of the encoder is used to extract multi-layer features of the input image, including a CBL block, a maximum pooling layer, multiple dense connection modules and multiple Transition connection blocks connected in sequence; the dense connection modules are connected through Transition connection blocks; multiple dense connection modules output deep features respectively to obtain extracted multi-level deep features.

3. A tunnel bolt corrosion area identification method according to claim 1, characterized in that: The dense connection module includes multiple cascaded Dense layer modules. Each Dense Layer module is directly connected to all previous Dense Layer modules in the cascade order. Alternatively, the number of Dense layers in the densely connected modules of different encoding layers of the encoder is different, and increases according to the set multiples from low to high encoding layers.

4. A tunnel bolt corrosion area identification method according to claim 3, characterized in that: The Dense layer module includes an input layer, a batch normalization layer, a ReLU nonlinear activation layer, a 1×1 convolutional layer, a batch normalization layer, a 3×3 convolutional layer, and an output layer connected in sequence.

5. A tunnel bolt corrosion area identification method according to claim 3, characterized in that: Each Denselayer module processes the input features in the following steps: Get input features and concatenate all input features; The received data is batch normalized and activated with ReLU nonlinearity, and then the first convolution operation is performed through a 1×1 convolutional layer to reduce the dimensionality of the data. The first convolutional block BN_ReLU_Conv2D is used for normalization, the ReLU activation function is used for nonlinear activation, and the dimensionality reduction is performed through a 1×1 convolutional layer; The reduced-dimensional data is batch normalized and then subjected to a second convolution operation of a 3×3 convolutional layer for feature extraction. The extracted features are concatenated with the input features of the current Dense Layer module and transmitted to each DenseLayer module connected to the back end.

6. A tunnel bolt corrosion area identification method according to claim 1, characterized in that: The transition connection block includes the input layer, batch normalization layer, ReLU activation layer, convolution layer and average pooling layer connected in sequence; The Transition connection block normalizes the input data through a batch normalization layer, increases its nonlinearity through the ReLU activation function, then adjusts the number of channels of the feature map through a 1×1 convolution, and finally uses a 2×2 average pooling layer to reduce the height and width of the feature map; The decoder includes multiple upsampling modules cascaded in sequence. The bottom-level upsampling module is connected to a convolution module. The input of the upsampling module is connected to the feature output end of the corresponding layer of the encoder. The output features of the upsampling module are spliced ​​with the output features of the encoder of the previous layer. The spliced ​​features are input to the upsampling module of the current layer for upsampling.

7. A tunnel bolt corrosion area identification method according to claim 6, characterized in that: Each upsampling module performs an upsampling operation, including the following steps: The acquired input features are bilinearly interpolated to double the size of the feature map, and then concatenated with the feature map of the previous layer in the channel dimension; a 3×3 convolution is used to extract features from the concatenated feature map, and then the CBAM attention mechanism is used to dynamically adjust the weights of the spatial channel feature map, and then a convolution layer is used to extract features to obtain the output feature map.

8. A tunnel bolt corrosion area identification method according to claim 6, characterized in that: During the training of the tunnel bolt corrosion detection model, the data set is constructed, including the following steps: Annotate each original image for rust, determine the outline and range of rust in the image, generate label data, and obtain a label file in Json format; Convert the Json file into a PNG label file with the same size as the original image, and adjust the grayscale of the image so that the pixel value of the background area is 0 and the pixel value of the rusted area is 1, and obtain the labeled image file; During the training process, a loss function combining Dice Loss and Focal Loss is used.

9. A tunnel bolt corrosion area identification system, characterized in that: include: An acquisition module: configured to acquire an image of a bolt to be identified; Feature extraction module: configured to input the image into the improved tunnel bolt corrosion detection model with encoder-decoder structure, and perform multi-level deep feature extraction through the encoder backbone network; The encoder backbone network of the improved tunnel bolt corrosion detection model includes a CBL module, a maximum pooling layer, and multiple dense connection modules and transition connection blocks that are alternately connected in sequence; the transition connection block connects two adjacent dense connection modules to control the size and number of channels of the feature map; Upsampling module: It is configured to introduce an attention mechanism in the decoder, upsample the extracted multi-level deep features, and use the attention mechanism to dynamically adjust the weights of the spatial and channel feature maps to obtain weighted feature maps after weighting; Fusion recognition module: It is configured to fuse the weighted feature map with the deep features output by the corresponding layer of the encoder to obtain a feature map with the same size as the input image and convert it to obtain a segmentation map of the bolt corrosion area.

10. An electronic device, characterized in that: The method comprises a memory and a processor and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the method for identifying a tunnel bolt corrosion area according to any one of claims 1 to 8 are completed.

Citation Information

Patent Citations

  • Metal corrosion image segmentation method and device

    CN115170579A

  • Tunnel bolt positioning and bolt corrosion area identification method

    CN115661534A

Cited By

  • Gas facility corrosion identification method based on semantic segmentation

    CN121259336A

  • Under-vehicle bolt corrosion detection method and device based on improved RT-DETR and medium

    CN121564697A

  • Under-vehicle bolt corrosion detection method and device based on improved RT-DETR, and medium

    CN121564697B