An image-based shield tunnel bolt health degree evaluation method

Abstract

The application provides an image-based shield tunnel bolt health degree evaluation method, and relates to the technical field of shield tunnel engineering, and comprises the following steps: acquiring a bolt scanning image in a tunnel; constructing a deep learning model, including a backbone network and a segmentation network; inputting the bolt scanning image after preprocessing into the deep learning model for corrosion detection; extracting a feature map by using the backbone network; setting a region of interest for the feature map; screening the region of interest; inputting the screened region of interest into the segmentation network; obtaining a corrosion detection result of the bolt scanning image, including position information, category information and segmentation information of corrosion; calculating a corrosion area according to the corrosion detection result; evaluating the bolt based on the corrosion area according to a health degree evaluation rule to obtain a health degree evaluation result of the bolt scanning image. The application solves the corrosion efficiency of the segment bolt, reduces the influence of artificial subjectivity on the accuracy and the time and artificial cost.

Description

Technical Field

[0001] This invention relates to the field of shield tunnel engineering technology, and in particular to an image-based method for evaluating the health of shield tunnel bolts. Background Technology

[0002] With the acceleration of urbanization and the rapid development of the national economy in China, urban rail transit systems, especially subway networks, have experienced explosive growth.

[0003] As a core component of subway infrastructure, shield tunnels are characterized by massive investment and high maintenance and reconstruction difficulties, posing severe challenges to their long-term service performance and lifespan. However, shield tunnels are exposed to a complex underground environment for extended periods, facing multiple factors such as groundwater seepage and ion erosion, leading to accelerated segment deterioration. These factors combined result in a series of structural defects during operation, including but not limited to: lining cracking and spalling, segment misalignment, and water leakage.

[0004] Domestic case studies highlight the potential risks to the long-term service performance of shield tunnels:

[0005] These cases highlight that the durability of shield tunnel structures has become a key factor affecting their long-term safe operation.

[0006] Currently, the detection of corrosion on bolts in shield tunnel segments still mainly relies on manual methods, which have many limitations:

[0007] Low efficiency: Manual inspection is slow and cannot meet the inspection needs of large-scale tunnel networks; Safety risks: Tunnel arch inspection requires special equipment, posing potential safety hazards; High cost: Manual inspection is costly and not conducive to long-term, large-scale periodic inspections; High subjectivity: Health assessment relies excessively on worker experience and lacks objective and unified evaluation standards; High professional requirements: High professional quality requirements are placed on inspection personnel, resulting in high training costs.

[0008] Given the important role of shield tunnels in urban rail transit and the challenges they face in long-term service performance, developing efficient, objective, and safe methods for detecting and evaluating corrosion of segment bolts has become an urgent task. Summary of the Invention

[0009] In view of this, the present invention proposes an image-based method for evaluating the health of shield tunnel bolts. This method uses a computer to quickly determine the location and area of ​​corrosion on the segment bolts and proposes a bolt health level. This method effectively solves the problem of corrosion efficiency of segment bolts, reduces the impact of human subjectivity on accuracy, and saves time and labor costs.

[0010] The technical solution of this invention is implemented as follows: This invention provides an image-based method for evaluating the health of bolts in shield tunnels, comprising:

[0011] S1 acquires bolt scan images inside the tunnel and preprocesses the bolt scan images;

[0012] S2 is used to build deep learning models, including backbone networks and segmentation networks;

[0013] S3 inputs the preprocessed bolt scan image into a deep learning model for rust detection, uses the backbone network to extract feature maps, sets regions of interest (ROIs) for the feature maps, filters the ROIs, and inputs the filtered ROIs into a segmentation network to obtain the rust detection results of the bolt scan image, including rust location information, category information, and segmentation information.

[0014] S4 calculates the corrosion area based on the corrosion detection results, and evaluates the bolt based on the corrosion area according to the health evaluation rules, thus obtaining the health evaluation result of the bolt scan image.

[0015] Based on the above technical solution, preferably, the backbone network includes a feature extraction module and a region generation module, and the segmentation network includes a first network branch and a second network branch, wherein:

[0016] The feature extraction module includes a 7*7 convolutional layer, a 3*3 max pooling layer, 4 residual blocks, and a feature pyramid. Each residual block contains multiple residual units, and each residual unit includes 2 1*1 convolutional layers and 1 3*3 convolutional layer. The feature pyramid adopts a top-down path.

[0017] The region generation module includes a shared convolutional layer, a classification layer, and a bounding box regression layer;

[0018] The first network branch includes two fully connected layers;

[0019] The second network branch includes 3x3 convolutional layers and transposed convolutional layers.

[0020] Based on the above technical solution, preferably, step S1 includes:

[0021] S11 converts the bolt scan image from the original RGB color space to the HSV color space to obtain an HSV image;

[0022] S12 separates the H, S, and V channels of an HSV image;

[0023] S13 performs exponential operations on the S and V channels respectively, where the value of the exponential factor ranges from 0.25 to 4;

[0024] S14 Keeps the H channel unchanged, and recombines the S and V channels processed in step S13 with the original H channel;

[0025] S15 converts the enhanced HSV image obtained in step S14 back to RGB space to obtain the preprocessed bolt scan image.

[0026] Based on the above technical solution, preferably, step S3 includes:

[0027] S31 inputs the preprocessed bolt scan image into the feature extraction module, uses a 7*7 convolutional layer to extract the first feature map C1, and then inputs 4 residual blocks in sequence to extract 4 feature maps, namely the second feature map C2, the third feature map C3, the fourth feature map C4, and the fifth feature map C5. The scale of each feature map is different, and the scale of the feature map increases sequentially according to the data flow.

[0028] S32 constructs the pyramid using a top-down path, resulting in a five-layer pyramid feature map;

[0029] S33 uses a fixed-size sliding window on a five-layer pyramid feature map. Multiple anchor boxes of preset size and aspect ratio are generated at the center of the sliding window. A 3*3 shared convolutional layer is applied to each sliding window position. A classification layer is used for binary classification. The probability of each anchor box being foreground or background is predicted. A bounding box regression layer is used for bounding box regression to predict the adjustment parameters of each anchor box. The N anchor boxes with the highest scores are selected based on the classification scores. NMS is applied to the N anchor boxes, and the NMS-reduced anchor boxes are used as the filtered regions of interest.

[0030] S34 applies RoI Align to each filtered region of interest, mapping regions of interest of different sizes to a feature map of a fixed size.

[0031] S35 inputs the feature map of the region of interest after RoI Align into the first network branch, uses the first fully connected layer to output multi-class classification, i.e. the category information of corrosion, and uses the second fully connected layer to output the bounding box result, i.e. the location information of corrosion.

[0032] S36 inputs the feature map of the region of interest after RoI Align into the second network branch, uses a 3*3 convolutional layer to enhance the feature map, and uses a transposed convolutional layer to upsample the feature map to generate a mask prediction for each category, i.e., the segmentation information of rust.

[0033] Based on the above technical solution, preferably, step S32 includes:

[0034] After performing a 1*1 convolution on C5, a second pyramid feature map P5 is constructed: P5 = Conv1*1(C5);

[0035] P5 is used to construct the highest-level pyramid feature map P6 through 3*3 convolution and max pooling: P6 = Maxpool2*2(Conv3*3(P5));

[0036] Upsample P5 to the size of C4, and add the result of 1*1 convolution between the upsampled P5 and C4 to construct the third pyramid feature map P4: P4 = Conv3*3(Upsample(P5)+Conv1*1(C4));

[0037] Upsample P4 to the size of C3, and add the result of 1*1 convolution between the upsampled P4 and C3 to construct the fourth pyramid feature map P3: P3 = Conv3*3(Upsample(P4)+Conv1*1(C3));

[0038] Upsample P3 to the size of C2, and add the result of 1*1 convolution between the upsampled P3 and C2 to construct the fifth pyramid feature map P2: P2 = Conv3*3(Upsample(P3)+Conv1*1(C2));

[0039] The final five-layer pyramid feature map includes P2, P3, P4, P5, and P6, with each layer having the same number of channels.

[0040] Based on the above technical solution, preferably, step S34 includes:

[0041] Obtain the coordinates of the filtered region of interest on the original image;

[0042] Set the target feature map size to N*N;

[0043] Map the coordinates of each region of interest from the original image scale to the scale of the target feature map;

[0044] Each mapped region of interest is uniformly divided into a grid the size of the target feature map, with each grid serving as a sub-window;

[0045] Within each sub-window, select sampling points, and for each sampling point, use an interpolation algorithm to calculate its feature value;

[0046] The feature values ​​of the sampling points within each sub-window are averaged to generate a single feature value.

[0047] The feature values ​​of all sub-windows are combined to form the final feature map of the region of interest after RoI Align, which has the same size as the target feature map.

[0048] Based on the above technical solution, preferably, step S35 includes:

[0049] Flatten the feature map of the region of interest after RoI Align into a one-dimensional vector;

[0050] The one-dimensional vector is input into the first fully connected layer, and after passing through the activation function, the original class score is output.

[0051] The softmax function is applied to the output of the first fully connected layer to convert the original class scores into a probability distribution, thus obtaining the probability of each class.

[0052] The flattened one-dimensional vector is input into the second fully connected layer to predict four bounding box adjustment values ​​(Δx, Δy, Δw, Δh) for each category;

[0053] Fine-tune the position and size of the region of interest based on the bounding box adjustment value corresponding to the highest probability value in each category;

[0054] The final output provides information on the type and location of the rust.

[0055] Based on the above technical solution, preferably, step S36 includes:

[0056] The feature map of the region of interest after RoI Align is input into the second network branch, and two 3*3 convolutional layers are used to extract and enhance the features to obtain the enhanced feature map.

[0057] Upsampling of the enhanced feature map is performed using a transposed convolutional layer;

[0058] The upsampled feature map is input into the final convolutional layer to generate mask predictions for each category.

[0059] The sigmoid activation function is applied to the output of the final convolutional layer to compress the value of each pixel to between 0 and 1, representing the probability that the pixel belongs to the rusted region.

[0060] Align the activated mask with the original region of interest and adjust it to the size of the original region of interest;

[0061] Output the final mask, which is the segmentation information of the rust.

[0062] Based on the above technical solution, preferably, step S4 includes:

[0063] S41 obtains the segmentation results of the rust, sets a threshold, binarizes the mask, and sets the value in the mask to 0 or 1;

[0064] S42 establishes a mapping relationship between pixels and actual size;

[0065] S43 calculates the number of values ​​of 1 in the mask, uses the mapping relationship to transform and calculate the corrosion area;

[0066] S44 obtains the total area of ​​the bolts, and calculates the percentage of corrosion based on the corrosion area and the total area of ​​the bolts.

[0067] S45 classifies the health level into health level 1, health level 2, health level 3, health level 4 and health level 5. Among them, health level 1 bolts have the lowest degree of corrosion, and health level 5 bolts have the highest degree of corrosion.

[0068] S46 divides the value range of the corrosion percentage into 5 value ranges, which correspond to health level 1, health level 2, health level 3, health level 4 and health level 5 in order from low to high.

[0069] S47 Based on the percentage of rust calculated in step S44, the health level of the bolt scan image is obtained.

[0070] Based on the above technical solution, preferably, step S42 includes:

[0071] Obtain camera parameters, including focal length f and sensor size. width and Sensor height Image resolution width and Image height ;

[0072] Use a rangefinder to measure the shooting distance D from the camera lens to the inspection bolt;

[0073] Calculate the horizontal and vertical field of view (FOV):

[0074] FOV horizontal = 2*arctan(Sensor) width / (2*f));

[0075] FOV vertical = 2*arctan(Sensor) height / (2*f));

[0076] Calculate the actual field of view size:

[0077] Real width =2*D*tan(FOV) horizontal / 2);

[0078] Real height =2*D*tan(FOV) vertical / 2);

[0079] Calculate the pixel-to-millimeter mapping:

[0080] X scale =Real width / Image width;

[0081] Y scale =Real height / Image height .

[0082] The present invention has the following advantages over the prior art:

[0083] (1) Automatic detection and health assessment of bolt corrosion were achieved using a deep learning model. This method not only significantly improves detection efficiency and accuracy but also overcomes the subjectivity and safety risks of traditional manual inspection. By combining image preprocessing, deep learning models, and health assessment rules, the entire process from image input to health assessment is automated, providing reliable technical support for the maintenance and management of shield tunnels and helping to extend the service life of tunnels.

[0084] (2) The HSV color space conversion and exponential operation preprocessing method used in this invention effectively enhances the contrast between the bolt corrosion area and the background. By performing exponential operations on the S and V channels, the saturation and brightness of the image are flexibly adjusted, making the corrosion area more visually prominent while maintaining the overall tone of the image. This preprocessing method not only improves the detection accuracy of the subsequent deep learning model, but also enhances the model's adaptability to bolt images taken under different lighting conditions, thereby improving the robustness and versatility of the entire system.

[0085] (3) The deep learning model designed in this invention includes a feature extraction module, a region generation module, and a dual-branch network structure, which realizes accurate localization, classification, and segmentation of bolt corrosion. The introduction of the feature pyramid network effectively integrates feature information at different scales, improving the model's ability to detect corrosion regions of different sizes. The application of RoI Align solves the feature misalignment problem and improves positional accuracy. The dual-branch network structure simultaneously outputs category information, position information, and segmentation information, realizing comprehensive analysis of corrosion regions;

[0086] (4) The health assessment rule established in this invention maps the percentage of rusted area to five health levels, thereby achieving a quantitative assessment of bolt health status. This assessment method not only objectively reflects the degree of bolt corrosion but also provides clear grading standards, facilitating managers to quickly determine bolt maintenance needs. By establishing a mapping relationship from pixels to actual dimensions, the practical significance of the assessment results is ensured. Attached Figure Description

[0087] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0088] Figure 1 This is a flowchart of the method of the present invention;

[0089] Figure 2 This is a flowchart illustrating the technical process of the present invention.

[0090] Figure 3 This is a structural diagram of the ResNet101 of the present invention;

[0091] Figure 4 This is a schematic diagram of the feature pyramid of the present invention;

[0092] Figure 5 This is a diagram of the segmentation network structure of the present invention;

[0093] Figure 6 This is a schematic diagram of RoI Align of the present invention. Detailed Implementation

[0094] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0095] like Figure 1 As shown, this invention provides an image-based method for evaluating the health of bolts in shield tunnels, including:

[0096] S1 acquires bolt scan images inside the tunnel and preprocesses the bolt scan images;

[0097] S2 is used to build deep learning models, including backbone networks and segmentation networks;

[0098] S3 inputs the preprocessed bolt scan image into a deep learning model for rust detection, uses the backbone network to extract feature maps, sets regions of interest (ROIs) for the feature maps, filters the ROIs, and inputs the filtered ROIs into a segmentation network to obtain the rust detection results of the bolt scan image, including rust location information, category information, and segmentation information.

[0099] S4 calculates the corrosion area based on the corrosion detection results, and evaluates the bolt based on the corrosion area according to the health evaluation rules, thus obtaining the health evaluation result of the bolt scan image.

[0100] like Figure 2 As shown, the technical implementation process of this invention is as follows: 1) Obtain images of rusted bolts in shield tunnel segments; 2) Perform contrast transformation on the images, i.e., preprocessing, to enhance the images; 3) Extract features from the images, using a feature extraction module of a deep learning model to achieve multi-scale feature extraction and pyramid feature construction; 4) Input the extracted features into a segmentation network, which has two branches: branch one is the class / box branch, i.e., the object detection branch, used to obtain the rust location and category information in the shield tunnel bolt rust image; branch two is the mask branch, i.e., the segmentation branch, used to obtain the image... 5) Obtain the segmentation and localization results of bolt corrosion based on the output of the segmentation network; 6) Binarize the mask of the corroded area, setting the mask to 1 or 0. A mask value of 1 indicates that the pixel belongs to the corroded area; 7) Calculate the number of pixels in the binarized mask, counting the number of white pixels with a value of 1, and calculate the corroded area; 8) Establish the mapping relationship between pixels and actual size to obtain the actual corroded area, obtain the total area of ​​the bolts, and conduct a safety rating of the bolt corrosion of the shield tunnel segment based on the proportion of the corroded area to the total bolt area.

[0101] Specifically, in one embodiment of the present invention, step S1 includes:

[0102] S11 converts the bolt scan image from the original RGB color space to the HSV color space to obtain an HSV image;

[0103] S12 separates the H, S, and V channels of an HSV image;

[0104] S13 performs exponential operations on the S and V channels respectively, where the value of the exponential factor ranges from 0.25 to 4;

[0105] S14 Keeps the H channel unchanged, and recombines the S and V channels processed in step S13 with the original H channel;

[0106] S15 converts the enhanced HSV image obtained in step S14 back to RGB space to obtain the preprocessed bolt scan image.

[0107] In this embodiment, the cvtColor function can be used to convert an RGB image to HSV format, such as: HSV image=cv2.cvtColor(rgb_image,cv2.COLOR_RGB2HSV); Use the split function to decompose the HSV image into three channels: H,S,V = cv2.split(HSV image ); Perform exponential operations on the S and V channels, with the exponential factor γ ranging from 0.25 to 4: S enhanced =np.power(S / 255.0,γ)*255,V enhanced =np.power(V / 255.0,γ)*255, convert the result back to uint8 type; use the Merge function to recombine the processed S and V channels with the original H channel: HSV enhanced =cv2.merge([H,S enhanced V enhanced Use the cvtColor function to convert the enhanced HSV image back to RGB format: RGB enhanced =cv2.cvtColor(HSV enhanced (cv2.COLOR_HSV2RGB).

[0108] This embodiment effectively enhances image contrast by performing exponential operations on the S (saturation) and V (luminance) channels. This processing method is particularly beneficial for highlighting the rusted areas on the bolt surface, making them more clearly distinguishable from the background. Keeping the H (hue) channel unchanged ensures that the basic color information of the original image is not altered. This is crucial for preserving the characteristic colors of the rusted areas, aiding in subsequent identification and classification. By adjusting the value of the exponential factor γ, the image can be flexibly adapted to images captured under different lighting conditions. This adaptability allows the preprocessing method to handle bolt images acquired in various environments. Adjusting the V channel can, to some extent, reduce the impact of uneven lighting on image quality.

[0109] Specifically, in one embodiment of the present invention, the backbone network includes a feature extraction module and a region generation module, and the segmentation network includes a first network branch and a second network branch, wherein:

[0110] The feature extraction module includes a 7*7 convolutional layer, a 3*3 max pooling layer, 4 residual blocks, and a feature pyramid. Each residual block contains multiple residual units, and each residual unit includes 2 1*1 convolutional layers and 1 3*3 convolutional layer. The feature pyramid adopts a top-down path.

[0111] The region generation module includes a shared convolutional layer, a classification layer, and a bounding box regression layer;

[0112] The first network branch includes two fully connected layers;

[0113] The second network branch includes 3x3 convolutional layers and transposed convolutional layers.

[0114] Specifically, in the deep learning model of this embodiment, the feature extraction module includes a feature extraction unit and a pyramid unit. The feature extraction unit adopts the ResNet101 architecture, removing the final average pooling layer and fully connected layer, retaining only the feature extraction architecture. The ResNet101 structure is as follows: Figure 3 As shown, the pyramid unit adopts an FPN structure, such as Figure 4 As shown, the segmentation network consists of two branches: an object detection branch (class / box branch) and a segmentation branch. The structure diagram of the segmentation network is shown below. Figure 5 As shown.

[0115] Specifically, after a deep learning model is built, it needs to be trained. The training process is as follows:

[0116] 1. Prepare a dataset of shield tunnel bolt images with rust location annotations, category labels, and segmentation masks. Enhance the dataset using the preprocessing method in step S1 by adding illumination variations.

[0117] 2. Set up the training strategy:

[0118] Initial stage: Freeze the feature extraction module and train only the region generation module and the segmentation network.

[0119] Fine-tuning phase: Unfreeze the high-level features of the feature extraction module, namely the third and fourth residual blocks, and perform end-to-end training.

[0120] Full fine-tuning: Unfreeze all layers and fine-tune with a small learning rate.

[0121] 3. Set training parameters:

[0122] Learning rate settings: A warmup strategy is used to gradually increase the learning rate to the initial value, and a cosine annealing strategy is used to decrease the learning rate.

[0123] Regularization: Weight decay is used to prevent overfitting, and Dropout is selectively applied to each fully connected layer.

[0124] 4. Perform joint training, with the overall loss function being: Ltotal = L RPN +L det +L seg , where L RPN The loss function for the backbone network, L det Let L be the loss function of the first network branch. seg This is the loss function for the second network branch.

[0125] Backbone network training:

[0126] Using a pre-trained ResNet101 as initial weights, the ResNet101 is fine-tuned to adapt to the bolt corrosion detection task. A Feature Pyramid Network (FPN) is built on top of the ResNet101. The output feature maps of the FPN are used to train a Region Generation Network (RPN). The loss function of the backbone network is: L RPN =L cls +λ1*L reg , where L cls It is a binary classification cross-entropy loss, i.e., prediction of foreground or background, L reg It is the smoothed L1 loss of bounding box regression.

[0127] First network branch training:

[0128] The region of interest (ROI) generated using RPN is combined with RoI Align for feature extraction. Classification and bounding box fine-tuning are then performed through two fully connected layers. The loss function of the first network branch is: L det =L clsdet +λ2*L regdet , where L clsdet It is a multi-class cross-entropy loss, used for predicting multiple categories of rust, L regdet It is the smoothed L1 loss of the rusted bounding box regression.

[0129] Second network branch training:

[0130] The region of interest generated using RPN is combined with RoI Align to extract features. Upsampling is performed using 3x3 convolutions and transposed convolutions. The loss function of the second network branch is: L seg =α1*L boundary +α2*L dice .

[0131] L boundary =ω·BCE(P boundary ,T boundary )+(1-ω)·BCE(P non-boundary ,T non-boundary )

[0132]

[0133] Among them, L boundary This is the boundary-aware loss, where ω is the boundary weight, which is dynamically adjusted based on the proportion of boundary pixels, and P boundary It predicts the boundary region, T boundary It is the true boundary region, P non-boundary It predicts non-boundary regions, T non-boundary It is a real non-boundary region, L diceLet X be the predicted segmentation mask, Y be the true segmentation mask, ε be the smoothing term, and BCE be the binary cross-entropy loss.

[0134] The entire model is optimized using backpropagation and stochastic gradient descent until the combined loss function reaches its minimum value, at which point training ends and a well-trained deep learning model is obtained.

[0135] Specifically, in one embodiment of the present invention, step S3 includes:

[0136] S31 inputs the preprocessed bolt scan image into the feature extraction module, uses a 7*7 convolutional layer to extract the first feature map C1, and then inputs 4 residual blocks in sequence to extract 4 feature maps, namely the second feature map C2, the third feature map C3, the fourth feature map C4, and the fifth feature map C5. The scale of each feature map is different, and the scale of the feature map increases sequentially according to the data flow.

[0137] S32 constructs the pyramid using a top-down path, resulting in a five-layer pyramid feature map;

[0138] S33 uses a fixed-size sliding window on a five-layer pyramid feature map. Multiple anchor boxes of preset size and aspect ratio are generated at the center of the sliding window. A 3*3 shared convolutional layer is applied to each sliding window position. A classification layer is used for binary classification. The probability of each anchor box being foreground or background is predicted. A bounding box regression layer is used for bounding box regression to predict the adjustment parameters of each anchor box. The N anchor boxes with the highest scores are selected based on the classification scores. NMS is applied to the N anchor boxes, and the NMS-reduced anchor boxes are used as the filtered regions of interest.

[0139] S34 applies RoI Align to each filtered region of interest, mapping regions of interest of different sizes to a feature map of a fixed size.

[0140] S35 inputs the feature map of the region of interest after RoI Align into the first network branch, uses the first fully connected layer to output multi-class classification, i.e. the category information of corrosion, and uses the second fully connected layer to output the bounding box result, i.e. the location information of corrosion.

[0141] S36 inputs the feature map of the region of interest after RoI Align into the second network branch, uses a 3*3 convolutional layer to enhance the feature map, and uses a transposed convolutional layer to upsample the feature map to generate a mask prediction for each category, i.e., the segmentation information of rust.

[0142] In this embodiment, step S31 includes:

[0143] The preprocessed bolt scan image is input into the feature extraction module, which uses a 7*7 convolutional layer to capture low-level features of the image, such as edges and textures. The 7*7 convolutional layer includes 64 convolutional kernels with a stride of 2. The first feature map C1 is output here, and its size is 1 / 2 of the original image.

[0144] A 3x3 max pooling layer with a stride of 2 is followed by a 7x7 convolutional layer to reduce the size of the first feature map C1 and increase the receptive field.

[0145] The pooled first feature map C1 is input into the residual structure, which consists of four residual blocks. Each residual block contains multiple residual units. The first residual block contains three residual units, its input is the feature map from the pooling layer, and its output is the second feature map C2, which is 1 / 4 the size of the original image. The second residual block contains four residual units, its input is C2, and its output is the third feature map C3, which is 1 / 8 the size of the original image. The third residual block contains 23 residual units. The input is C3, and the output is the fourth feature map C4, which is 1 / 16 the size of the original image. The fourth residual block consists of 3 residual units, with its input being C4 and its output being the fifth feature map C5, which is 1 / 32 the size of the original image. Each residual unit includes a 1*1 convolution (dimensionality reduction), a 3*3 convolution (feature extraction), a 1*1 convolution (dimensionality increase), and a residual connection (directly adding the input to the output). These four residual blocks generate feature maps C2, C3, C4, and C5 in sequence. The feature map scale is C1>C2>C3>C4>C5. Smaller feature maps (such as C5) have a larger receptive field and are suitable for detecting large-sized rust areas; larger feature maps (such as C2) retain more details and are suitable for detecting small-sized rust areas.

[0146] Specifically, in one embodiment of the present invention, step S32 includes:

[0147] After performing a 1*1 convolution on C5, a second pyramid feature map P5 is constructed: P5 = Conv1*1(C5); where the 1*1 convolution is used to adjust the number of channels, while preserving high-level semantic information to match subsequent layers.

[0148] The highest-level pyramid feature map P6 is constructed by convolution and max pooling of P5: P6 = Maxpool2*2(Conv3*3(P5)). In this embodiment, an additional P6 is introduced on top of the deepest feature output of the ResNet101 structure as the highest-level feature of the feature pyramid in order to further increase the receptive field and increase feature information.

[0149] Upsample P5 to the size of C4, and then add the results of the upsampled P5 and C4 after a 1*1 convolution to construct the third pyramid feature map P4: P4 = Conv3*3(Upsample(P5)+Conv1*1(C4)); This layer of features integrates high-level semantic information and mid-level features, which enhances feature expression.

[0150] Upsample P4 to the size of C3, and add the results of the upsampled P4 and C3 after a 1*1 convolution to construct the fourth pyramid feature map P3: P3 = Conv3*3(Upsample(P4)+Conv1*1(C3)); This layer further integrates the features of the middle and low layers to enhance the expression of details.

[0151] Upsample P3 to the size of C2, and add the results of the upsampled P3 and C2 after a 1*1 convolution to construct the fifth pyramid feature map P2: P2 = Conv3*3(Upsample(P3)+Conv1*1(C2)); This layer fuses low-level features to enhance fine-grained expression.

[0152] The final five-layer pyramid feature map includes P2, P3, P4, P5, and P6, with each layer having the same number of channels.

[0153] In this embodiment, a pyramid network is constructed after the feature extraction unit. The generated feature pyramid contains multiple resolution levels, which is beneficial for detecting targets of different sizes. By upsampling high-level semantic information and passing it to the low level, the semantic expressive power of the low-level features is enhanced. The detailed information of the low-level features is preserved and utilized, which helps to improve the performance of small target detection and fine segmentation.

[0154] Specifically, in one embodiment of the present invention, step S33 includes:

[0155] Apply a sliding window of the same size, such as a 3x3 window, to each feature map layer P2, P3, P4, P5, and P6. This sliding window moves across each feature map layer, covering all positions.

[0156] For each sliding window position, multiple anchor frames are generated, such as 3 scales and 3 aspect ratios, for a total of 9 preset anchor frames. The size of the anchor frames varies with the feature layer level, with larger anchor frames for higher layers and smaller anchor frames for lower layers.

[0157] A 3x3 shared convolutional layer is applied to each sliding window position. This shared convolutional layer shares parameters across all feature maps and all positions, and outputs a fixed-dimensional feature vector.

[0158] For each anchor box, predict its probability of being either foreground (containing the target) or background. Specifically, this can be achieved using a 1*1 convolution, with the number of output channels being twice the number of anchor boxes.

[0159] The adjustment parameters for each anchor frame are predicted using a 1x1 convolution, with the number of output channels being four times the number of anchor frames.

[0160] Sort all anchor frames by their foreground scores and select the top N anchor frames with the highest scores. In this embodiment, N is set to 1000.

[0161] Apply NMS (Non-maximum suppression) to N anchor boxes, calculate the IoU between the anchor boxes, keep the anchor box with the highest score, suppress other anchor boxes with high overlap with it, and repeat this process until all anchor boxes have been processed.

[0162] Use the anchor box after NMS as the filtered region of interest.

[0163] Specifically, such as Figure 6 As shown, in one embodiment of the present invention, step S34 includes:

[0164] Obtain the coordinates of the filtered region of interest on the original image;

[0165] Set the target feature map size to N*N;

[0166] Map the coordinates of each region of interest from the original image scale to the scale of the target feature map;

[0167] Each mapped region of interest is uniformly divided into a grid the size of the target feature map, with each grid serving as a sub-window;

[0168] Within each sub-window, select sampling points, and for each sampling point, use an interpolation algorithm to calculate its feature value;

[0169] The feature values ​​of the sampling points within each sub-window are averaged to generate a single feature value.

[0170] The feature values ​​of all sub-windows are combined to form the final feature map of the region of interest after RoI Align, which has the same size as the target feature map.

[0171] Specifically, in this embodiment, an adaptive sampling strategy is adopted when selecting sampling points, as follows:

[0172] The number of sampling points is calculated based on the area and aspect ratio of the region of interest. The calculation function is: Number of sampling points = base samples *(1+α*log(area / base area )+β*|aspect ratio -1|), where base samples It is the number of base sampling points, and area is the area of ​​the region of interest. areaIt is a reference area, aspect ratio α is the aspect ratio of the region of interest, and β are adjustable hyperparameters. The number of sampling points is rounded up to the nearest square.

[0173] In this embodiment, the interpolation algorithm is as follows:

[0174] For each sampling point, a learnable interpolation kernel K(x,y) is defined. K(x,y) is a small neural network. The input is the offset of the sampling point relative to its four nearest integer coordinates. The network outputs four weights w1, w2, w3, w4, corresponding to the four nearest integer coordinates. The feature value calculation formula is: f(x,y) = w1 * f 11 +w2*f 21 +w3*f 12 +w4*f 22 , where f 11 f 21 f 12 f 22 These are the eigenvalues ​​of the four nearest integer coordinate points.

[0175] In this embodiment, an adaptive sampling strategy is employed to dynamically adjust the sampling method based on the size and shape of the region of interest, making the extraction of different features more flexible and accurate. The interpolation process is transformed into a learnable process, allowing the network to automatically adjust the optimal feature extraction method according to task requirements. By adjusting the α and β parameters, the sensitivity of the sampling strategy to the size and shape of the region of interest can be controlled with fine granularity.

[0176] Specifically, in one embodiment of the present invention, step S35 includes:

[0177] The feature map of the region of interest after RoI Align is flattened into a one-dimensional vector; for example, a 7*7*256 feature map is flattened into a one-dimensional vector of length NC=12544, converting the spatial information into a sequence form.

[0178] The one-dimensional vector is input into the first fully connected layer, and after passing through the activation function, the original class score is output; the size of the weight matrix of the first fully connected layer is: (NC)*(num) classes +1), the activation function is ReLU to increase nonlinearity.

[0179] Apply the softmax function to the output of the first fully connected layer to transform the original class scores into a probability distribution, obtaining the probability of each class; ensure that the sum of all class probabilities is 1. Formula: Its output is the probability of each category, including the background class.

[0180] The flattened one-dimensional vector is input into the second fully connected layer to predict four bounding box adjustment values ​​(Δx, Δy, Δw, Δh) for each category; the size of the weight matrix of the second fully connected layer is: (NC)*(4*num) classes This fully connected layer does not apply an activation function, and the resulting adjustment value can be positive or negative.

[0181] Fine-tune the position and size of the region of interest (ROI) based on the bounding box adjustment value corresponding to the highest probability value in each category; select the non-background category with the highest probability, extract the bounding box adjustment value corresponding to that category, and use these values ​​to adjust the coordinates and size of the original ROI. The adjustment formula is: x new =x + w*Δx, y new =y + h*Δy, w new =w*e Δw h new =h*e Δh , where (x,y,w,h) are the original center coordinates, width, and height of the region of interest.

[0182] The final output provides the category and location information for the rust. The category information is the highest-probability non-background category and its softmax probability, while the location information is the fine-tuned bounding box coordinates (x, y, y). new ,y new ,w new ,h new ).

[0183] Specifically, in one embodiment of the present invention, step S36 includes:

[0184] The feature map of the region of interest after RoI Align is input into the second network branch, and two 3*3 convolutional layers are used to extract and enhance the features to obtain the enhanced feature map.

[0185] Specifically, the two convolutional layers are 3*3 convolutions with 256 filters and ReLU activation. The two convolutional layers are used to further extract and enhance features, increasing the network's receptive field.

[0186] The enhanced feature map is upsampled using a transposed convolutional layer.

[0187] Specifically, the transposed convolutional layer is a 2x2 transposed convolution with a stride of 2 and 256 filters. This layer enlarges the feature map size, recovering higher resolution spatial information, which is beneficial for generating more refined masks.

[0188] The upsampled feature maps are input into the final convolutional layer to generate mask predictions for each category.

[0189] Specifically, the final convolutional layer uses 1*1 convolution, and its output channel number is the number of categories (excluding the background), with each channel corresponding to a mask prediction for one category.

[0190] The sigmoid activation function is applied to the output of the final convolutional layer, compressing the value of each pixel to between 0 and 1, representing the probability that the pixel belongs to the rusted region. The sigmoid function is: f(x) = 1 / (1+e^x). -x This function compresses the pixel value to a range of 0-1, where 0 indicates that the pixel does not belong to the rusted area, and 1 indicates that it belongs to the rusted area.

[0191] Align the activated mask with the original region of interest and adjust it to the size of the original region of interest.

[0192] Bilinear interpolation can be used to adjust the mask to the size of the original region of interest to ensure that the mask accurately matches the detected rust area.

[0193] Output the final mask, which is the segmentation information of the rust.

[0194] In this embodiment, after sigmoid activation, morphological operations can be performed on the mask to effectively refine the segmentation results, remove noise, fill small holes, and make the mask smoother and more accurate. The morphological process is as follows:

[0195] The mask is binarized using Otsus, and opening and closing operations are performed using structuring elements of different sizes. The structuring elements are applied gradually from small to large to handle noise and holes of different sizes.

[0196] Noise removal: Opening operation is used, first erosion, then dilation.

[0197] Small hole filling: Use closed-loop operation, first expand, then etch.

[0198] Specifically, the erosion operation is: D(x,y)=min{D(x+i,y+j)+d(i,j)}, and the dilation operation is: D(x,y)=max{D(xi,yj)-d(i,j)}, where D is the distance graph and d is the distance from the center of the structuring element to (i,j).

[0199] Median filtering is used to smooth the boundaries, and connected component analysis is performed to remove regions with excessively small areas.

[0200] The processed mask is fused with the original sigmoid output to obtain a morphologically processed mask.

[0201] Specifically, in one embodiment of the present invention, step S4 includes:

[0202] S41 obtains the segmentation results of the rust, sets a threshold, binarizes the mask, and sets the value in the mask to 0 or 1;

[0203] S42 establishes a mapping relationship between pixels and actual size;

[0204] S43 calculates the number of values ​​of 1 in the mask, uses the mapping relationship to transform and calculate the corrosion area;

[0205] S44 obtains the total area of ​​the bolts, and calculates the percentage of corrosion based on the corrosion area and the total area of ​​the bolts.

[0206] S45 classifies the health level into health level 1, health level 2, health level 3, health level 4 and health level 5. Among them, health level 1 bolts have the lowest degree of corrosion, and health level 5 bolts have the highest degree of corrosion.

[0207] S46 divides the value range of the corrosion percentage into 5 value ranges, which correspond to health level 1, health level 2, health level 3, health level 4 and health level 5 in order from low to high.

[0208] S47 Based on the percentage of rust calculated in step S44, the health level of the bolt scan image is obtained.

[0209] Specifically, in one embodiment of the present invention, step S42 includes:

[0210] Obtain camera parameters, including focal length f and sensor size. width and Sensor height Image resolution width and Image height ;

[0211] Use a rangefinder to measure the shooting distance D from the camera lens to the inspection bolt;

[0212] Calculate the horizontal and vertical field of view (FOV):

[0213] FOV horizontal = 2*arctan(Sensor) width / (2*f));

[0214] FOV vertical = 2*arctan(Sensor) height / (2*f));

[0215] Calculate the actual field of view size:

[0216] Real width =2*D*tan(FOV) horizontal / 2);

[0217] Real height =2*D*tan(FOV) vertical / 2);

[0218] Calculate the pixel-to-millimeter mapping:

[0219] X scale =Real width / Image width ;

[0220] Y scale =Real height / Image height .

[0221] In another embodiment of the present invention, considering the issues of lens distortion and tilt angle, an alternative method is used to establish the mapping relationship between pixels and actual size:

[0222] Obtain camera parameters, including focal length f and sensor size. width and Sensor height Image resolution width and Image height The distortion coefficients k1, k2 (radial distortion) and p1, p2 (tangential distortion);

[0223] Use a rangefinder to measure the shooting distance D from the camera lens to the inspection bolt; obtain the camera tilt angle θ;

[0224] Calculate the horizontal and vertical field of view (FOV):

[0225] FOV horizontal = 2*arctan(Sensor) width / (2*f));

[0226] FOV vertical = 2*arctan(Sensor) height / (2*f));

[0227] Calculate the actual field of view size:

[0228] Real width =2*D*tan(FOV) horizontal / 2) / cosθ;

[0229] Real height =2*D*tan(FOV) vertical / 2) / cosθ;

[0230] Calculate the initial pixel-to-millimeter mapping:

[0231] X scale-initial=Real width / Image width ;

[0232] Y scale-initial =Real height / Image height .

[0233] For each pixel (x, y) in the image, perform the following correction:

[0234] Normalize pixel coordinates:

[0235] x normalized =(x-Image) width / 2) / Image width ;

[0236] y normalized =(y-Image) height / 2) / Image height ;

[0237] Calculate radial distortion:

[0238] r2=x normalized 2 +y normalized 2 ;

[0239] radial distortion =1+k1*r2+k2*r2 2 ;

[0240] Calculate tangential distortion:

[0241] tangential x =2*p1*x normalized *y normalized +p2*(r2+2*x normalized 2 )

[0242] tangential y =p1*(r2+2*y) normalized 2 )+2*p2*x normalized *y normalized ;

[0243] Application of distortion correction:

[0244] x corrected =x normalized *radial distortion +tangential x ;

[0245] ycorrected =y normalized *radial distortion +tangential y ;

[0246] Convert the corrected coordinates back to pixel coordinates:

[0247] x final =x corrected *Image width +Image width / 2;

[0248] y final =y corrected *Image height +Image height / 2;

[0249] Calculate the final pixel-to-millimeter mapping:

[0250] X scale-final =X scale-initial *(Image width / max(x final -min(x final )));

[0251] Y scale-final =Y scale-initial *(Image height / max(y final -min(y final ))).

[0252] After determining the mapping relationship, the rust area is calculated based on the number of pixels with a value of 1 in the mask. The total area of ​​the bolt is then obtained, allowing the calculation of the rust percentage, which is the percentage of the rusted area relative to the total bolt area.

[0253] The 0-100% rust percentage range is divided into 5 intervals: 0%: health level 1; 1-10%: health level 2; 10-25%: health level 3; 25-40%: health level 4; and above 40%: health level 5.

[0254] The corresponding health level can be determined based on the calculated percentage of rust.

[0255] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for evaluating the health of bolts in shield tunnels based on images, characterized in that, include: S1 acquires bolt scan images inside the tunnel and preprocesses the bolt scan images; S2 constructs deep learning models, including backbone networks and segmentation networks; The backbone network includes a feature extraction module and a region generation module, and the segmentation network includes a first network branch and a second network branch, wherein: The feature extraction module includes a 7*7 convolutional layer, a 3*3 max pooling layer, 4 residual blocks, and a feature pyramid. Each residual block contains multiple residual units, and each residual unit includes two 1*1 convolutional layers and one 3*3 convolutional layer. The feature pyramid adopts a top-down path. The region generation module includes a shared convolutional layer, a classification layer, and a bounding box regression layer; The first network branch includes two fully connected layers; The second network branch includes a 3x3 convolutional layer and a transposed convolutional layer; S3 inputs the preprocessed bolt scan image into a deep learning model for rust detection, uses the backbone network to extract feature maps, sets regions of interest (ROIs) for the feature maps, filters the ROIs, and inputs the filtered ROIs into a segmentation network to obtain the rust detection results of the bolt scan image, including rust location information, category information, and segmentation information. Step S3 includes: S31 inputs the preprocessed bolt scan image into the feature extraction module, uses a 7*7 convolutional layer to extract the first feature map C1, and then inputs 4 residual blocks in sequence to extract 4 feature maps, namely the second feature map C2, the third feature map C3, the fourth feature map C4, and the fifth feature map C5. The scale of each feature map is different, and the scale of the feature map increases sequentially according to the data flow. S32 constructs the pyramid using a top-down path, resulting in a five-layer pyramid feature map; S33 uses a fixed-size sliding window on a five-layer pyramid feature map, generates multiple anchor boxes of preset size and aspect ratio at the center of the sliding window, applies a 3*3 shared convolutional layer to each sliding window position, uses a classification layer for binary classification, predicts the probability of each anchor box being foreground or background, uses a bounding box regression layer for bounding box regression, predicts the adjustment parameters of each anchor box, selects the N anchor boxes with the highest scores based on the classification scores, applies NMS to the N anchor boxes, and uses the NMS-reduced anchor boxes as the filtered regions of interest. S34 applies RoI Align to each filtered region of interest, mapping regions of interest of different sizes to a feature map of a fixed size; S35 inputs the feature map of the region of interest after RoI Align into the first network branch, uses the first fully connected layer to output multi-class classification, i.e. the category information of corrosion, and uses the second fully connected layer to output the bounding box result, i.e. the location information of corrosion. S36 inputs the feature map of the region of interest after RoI Align into the second network branch, uses a 3*3 convolutional layer to enhance the feature map, uses a transposed convolutional layer to upsample the feature map, and generates a mask prediction for each category, i.e. the segmentation information of rust. S4 calculates the corrosion area based on the corrosion detection results, and evaluates the bolt based on the corrosion area according to the health evaluation rules to obtain the health evaluation results of the bolt scan image.

2. The image-based method for evaluating the health of shield tunnel bolts as described in claim 1, characterized in that, Step S1 includes: S11 converts the bolt scan image from the original RGB color space to the HSV color space to obtain an HSV image; S12 separates the H, S, and V channels of an HSV image; S13 performs exponential operations on the S and V channels respectively, where the value of the exponential factor ranges from 0.25 to 4; S14 Keep the H channel unchanged, and recombine the S and V channels processed in step S13 with the original H channel; S15 Convert the enhanced HSV image obtained in step S14 back to RGB space to obtain the preprocessed bolt scan image.

3. The image-based method for evaluating the health of shield tunnel bolts as described in claim 1, characterized in that, Step S32 includes: After performing a 1*1 convolution on C5, a second-layer pyramid feature map P5 is constructed: P5=Conv1*1(C5); P5 is used to construct the highest-level pyramid feature map P6 through 3*3 convolution and max pooling: P6=Maxpool2*2(Conv3*3(P5)); Upsample P5 to the size of C4, and add the result of 1*1 convolution between the upsampled P5 and C4 to construct the third pyramid feature map P4: P4=Conv3*3(Upsample(P5)+Conv1*1(C4)); Upsample P4 to the size of C3, and add the result of 1*1 convolution between the upsampled P4 and C3 to construct the fourth pyramid feature map P3: P3=Conv3*3(Upsample(P4)+Conv1*1(C3)); Upsample P3 to the size of C2, and add the result of 1*1 convolution between the upsampled P3 and C2 to construct the fifth pyramid feature map P2: P2=Conv3*3(Upsample(P3)+Conv1*1(C2)); The final five-layer pyramid feature map includes P2, P3, P4, P5, and P6, with each layer having the same number of channels.

4. The image-based method for evaluating the health of shield tunnel bolts as described in claim 1, characterized in that, Step S34 includes: Obtain the coordinates of the filtered region of interest on the original image; Set the target feature map size to N*N; Map the coordinates of each region of interest from the original image scale to the scale of the target feature map; Each mapped region of interest is uniformly divided into a grid the size of the target feature map, with each grid serving as a sub-window; Within each sub-window, select sampling points, and for each sampling point, use an interpolation algorithm to calculate its feature value; The feature values ​​of the sampling points within each sub-window are averaged to generate a single feature value. The feature values ​​of all sub-windows are combined to form the final feature map of the region of interest after RoI Align, which has the same size as the target feature map.

5. The image-based method for evaluating the health of shield tunnel bolts as described in claim 1, characterized in that, Step S35 includes: Flatten the feature map of the region of interest after RoI Align into a one-dimensional vector; The one-dimensional vector is input into the first fully connected layer, and after passing through the activation function, the original class score is output. The softmax function is applied to the output of the first fully connected layer to convert the original class scores into a probability distribution, thus obtaining the probability of each class. The flattened one-dimensional vector is input into the second fully connected layer to predict four bounding box adjustment values ​​(Δx, Δy, Δw, Δh) for each category; Fine-tune the position and size of the region of interest based on the bounding box adjustment value corresponding to the highest probability value in each category; The final output provides information on the type and location of the rust.

6. The image-based method for evaluating the health of shield tunnel bolts as described in claim 1, characterized in that, Step S36 includes: The feature map of the region of interest after RoI Align is input into the second network branch, and two 3*3 convolutional layers are used to extract and enhance the features to obtain the enhanced feature map. Upsampling of the enhanced feature map is performed using a transposed convolutional layer; The upsampled feature map is input into the final convolutional layer to generate mask predictions for each category. The sigmoid activation function is applied to the output of the final convolutional layer to compress the value of each pixel to between 0 and 1, representing the probability that the pixel belongs to the rusted region. Align the activated mask with the original region of interest and adjust it to the size of the original region of interest; Output the final mask, which is the segmentation information of the rust.

7. The image-based method for evaluating the health of shield tunnel bolts as described in claim 1, characterized in that, Step S4 includes: S41 obtains the segmentation results of the rust, sets the threshold, binarizes the mask, and sets the value in the mask to 0 or 1; S42 establishes a mapping relationship between pixels and actual dimensions; S43 calculates the number of values ​​of 1 in the mask, uses the mapping relationship to transform and calculate the corrosion area; S44 obtains the total area of ​​the bolts and calculates the percentage of corrosion based on the corrosion area and the total area of ​​the bolts. S45 divides the health level into health level 1, health level 2, health level 3, health level 4 and health level 5. Among them, the health level 1 bolt has the lowest degree of corrosion and the health level 5 bolt has the highest degree of corrosion. S46 divides the value range of the corrosion percentage into 5 value ranges, which correspond to health level 1, health level 2, health level 3, health level 4 and health level 5 in order from low to high. S47 Based on the percentage of rust calculated in step S44, the health level of the bolt scan image is obtained.

8. The image-based method for evaluating the health of shield tunnel bolts as described in claim 7, characterized in that, Step S42 includes: Obtain camera parameters, including focal length f and sensor size. width and Sensor height Image resolution width and Image height ; Use a rangefinder to measure the shooting distance D from the camera lens to the inspection bolt; Calculate the horizontal and vertical field of view (FOV): FOV horizontal =2*arctan(Sensor width / (2*f)); FOV vertical =2*arctan(Sensor height / (2*f)); Calculate the actual field of view size: Real width =2*D*tan(FOV horizontal / 2); Real height =2*D*tan(FOV vertical / 2); Calculate the pixel-to-millimeter mapping: X scale =Real width / Image width ; Y scale =Real height / Image height 。

Images