Tunneling rock ballast real-time identification method based on U-Net-SAM coupling driving

Abstract

The invention provides a tunneling rock slag real-time identification method based on U-Net-SAM coupling driving, and the method achieves the high-precision quantization of the size and morphological parameters of rock slag through the fusion of a semantic segmentation network (U-net) and a visual basis model (Segment Anything Model, SAM). The method comprises the following steps: firstly, reasoning an input rock slag image by using a U-Net network to generate a rough rock slag identification result; then, obtaining the center-of-mass coordinate of each rock ballast from the rough result as the automatic prompt input of an SAM model, and extracting an initial mask in combination with the zero sample segmentation capability of the SAM; and carrying out post-processing optimization on the initial mask by adopting a morphological noise filtering method and an intersection-to-union ratio threshold optimization overlapped region processing strategy. According to the method, automatic segmentation and size and shape parameter extraction of the tunnel conveyor belt rock ballast image are realized, the boundary identification problem of a traditional method in a dense particle scene is effectively solved, and real-time and reliable rock machine interaction data support is provided for realizing intelligent tunneling parameter regulation and control.

Description

Technical Field

[0001] The present invention relates to a tunnel engineering intelligent construction and geological perception technology, and in particular to a real-time identification method for tunnel excavation rock debris based on U-Net-SAM coupling drive. Background Art

[0002] In tunnel boring machine (TBM) construction, the dynamic coupling relationship between rock slag morphology, rock mass mechanical properties, and excavation parameters is the core characteristic of the rock-machine feedback mechanism. The TBM cutterhead causes shear fracture of the rock mass through the extrusion of the cutter blades. This fragmentation pattern is directly reflected in the geometric characteristics of the rock slag: intact rock mass has a strong homogeneity and a single crack propagation path, forming regular flaky rock slag; whereas rock mass with developed joints is controlled by structural planes, with cracks extending along weak planes, resulting in large-scale irregular blocks. By real-time analysis of rock slag characteristics in conveyor belt muck images and combining them with time-series data of excavation parameters (such as thrust, torque, and penetration), dynamic feedback on the rock slag fragmentation mechanism and equipment operating status can be obtained, thereby guiding the adjustment of cutterhead parameters and the optimization of tool layout, avoiding engineering risks such as machine jams and landslides.

[0003] Currently, the main methods for obtaining rock slag size and morphological parameters are manual measurement and traditional computer vision-based methods. Manual screening and statistics suffer from cumbersome processes and long detection cycles, while traditional computer vision-based methods exhibit significant limitations in complex engineering scenarios. These limitations include motion blur caused by conveyor belt operation, texture loss due to mud and water covering the rock slag surface, and boundary adhesion caused by dense accumulation, which severely restrict feature extraction accuracy. Existing algorithms often suffer from both over-segmentation and under-segmentation, particularly under conditions with diverse rock slag morphologies and fluctuating lighting conditions. This makes it difficult to meet the requirements of real-time geological perception and dynamic control of tunneling parameters. Therefore, a method that can combine dynamic images to identify rock slag features in real time is urgently needed. This method would provide reliable technical support for the construction of a closed-loop control system that combines dynamic coupling analysis of rock slag morphology with tunneling parameters and integrates rock-machine interaction, promoting intelligent decision-making and adaptive control in tunneling, and achieving the coordinated optimization of engineering safety and efficiency. Summary of the Invention

[0004] The present invention provides a real-time identification method for rock slag during tunneling, based on U-Net-SAM coupling. By integrating an improved U-Net network with the zero-shot segmentation capabilities of SAM to create a dual-model collaborative architecture, semantic segmentation masks for rock slag are extracted in real time based on conveyor belt muck images. Multidimensional morphological parameters such as equivalent circle diameter, aspect ratio, and roundness ratio are calculated using a geometric feature quantization algorithm. The method also dynamically inverts rock fragmentation mechanisms using synchronously correlated tunneling parameter time series data. This method's advantage lies in leveraging the zero-shot generalization capabilities of the SAM model to overcome the traditional segmentation algorithm's reliance on labeled data. By optimizing mask generation accuracy through semantic segmentation-guided prompting, it achieves non-contact, real-time identification of rock slag features, addressing the issues of low timeliness and insufficient data standardization associated with manual measurement. This method provides reliable technical support for the construction of a closed-loop control system for dynamic coupled analysis of rock slag morphology and tunneling parameters, including rock-machine interaction.

[0005] The real-time identification method for tunneling rock debris based on U-Net-SAM coupling includes the following steps (the code is developed in Python, and the main libraries used include: pytorch, torchvision, rasterio, Scikit-Image, scipy, etc.):

[0006] The first step is to obtain images of the soil on the conveyor belt during tunnel excavation, specifically:

[0007] The camera was fixed 0.8m above the conveyor belt. A 10cm square wooden board was placed on the conveyor belt for photo calibration. The calculated conversion factor between pixel size and actual size was approximately 0.4mm / pixel.

[0008] Take photos of the dirt on the conveyor belt. The dirt images are saved with the extension .jpg. The image resolution is 2160×2880 pixels and the bit depth is 24.

[0009] The second step is to build a U-Net network and implement rock chip image segmentation, specifically:

[0010] Build the U-net encoder-decoder network architecture and define the input layer as a 512×512×3 RGB image;

[0011] The encoder extracts features through a 4-level downsampling module (each level contains a double convolution layer, a batch normalization layer, and a maximum pooling layer), and the number of channels is doubled to 256 at each level;

[0012] The decoder restores the spatial resolution through a 4-level upsampling module (including a transposed convolution layer, a feature concatenation operation, and a double convolution layer);

[0013] Finally, the 3-channel softmax classification result is output;

[0014] Design the weighted cross entropy loss function. The specific calculation formula is: Where N is the number of samples, C is the number of categories, in this paper C = 3 (the values ​​of rock slag, boundary and background are 1, 2, and 0 respectively), y ture is the one-hot encoding of the actual category label, with a shape of (N, C), y pred is the logits output by the model, and the shape is also (N, C), w i is the weight of category i, and the category weight matrix is ​​set to [0.6, 1.0, 5.0], corresponding to the classification processing of background, rock debris, and boundary;

[0015] The third step is the block prediction and result combination of the image, specifically:

[0016] Read the .h5 model file containing the U-net network convolution kernel parameters and batch normalization parameters, and initialize the weights of each layer of the network;

[0017] Read the captured image as the input image and check whether the length and width of the image are multiples of 512. If not, add several blank rows and columns (all with 0 values) on the bottom and right sides of the image to make them multiples of 512.

[0018] The input image is cropped into several 512×512 pixel blocks in the form of a sliding window, with a stride of 256;

[0019] A double-loop structure is constructed to complete horizontal and vertical block-by-block reasoning. The output shape of each image block is (512, 512, 3), where channel 0 is the background probability, channel 1 is the rock debris probability, and channel 2 is the boundary probability, all with the shape of (512, 512, 1).

[0020] The prediction values ​​of the overlapping areas are weighted by the Hanning window weights. The Hanning vector is defined as: Where N = 512, is the length of the Hanning vector, n is the index, and the Hanning window weight matrix can be obtained by performing the outer product of two Hanning vectors;

[0021] The output results of the image blocks are multiplied by the Hanning window weight matrix one by one, and then spliced ​​together to obtain the segmentation results of the complete image, providing basic data for subsequent centroid calculation.

[0022] The fourth step is to generate the centroid coordinates of the rock slag based on the U-Net segmentation results and optimize the prompt point set, specifically:

[0023] Channel 1 (rock ballast probability) of the complete image segmentation result is binarized with a threshold of 0.5. The connected component labeling algorithm of the scikit-image library is used to obtain independent rock ballast regions. The area and centroid coordinate properties of each region are extracted through regionprops_table to generate the initial centroid coordinate set.

[0024] The initial centroids are screened based on channel 0 (background probability) of the complete image segmentation result, and coordinate points with background probability values ​​higher than 0.3 are removed to reduce the false detection rate.

[0025] Channel 2 (boundary probability) of the complete image segmentation result was extracted and binarized with a threshold of 0.5. The rock fragment boundary was reconstructed using the distance transform and watershed algorithm. The local maximum points were located to generate secondary centroid coordinates. The dense centroids were spatially clustered using DBSCAN clustering (neighborhood radius dbs_max_dist = 1.0), and redundant detection points were eliminated through mean coordinate fusion.

[0026] The primary centroid set and the optimized secondary centroid set are coordinately superimposed to construct a composite prompt point matrix covering the actual rock debris location, providing a high-confidence spatial prior for the SAM model.

[0027] The fifth step is high-precision rock ballast segmentation driven by the SAM model, specifically:

[0028] Load the pre-trained SAM model (sam_vit_h_4b8939.pth) to build a segmenter and a SamPredictor instance;

[0029] Encode the input image into feature embeddings to support subsequent fast mask prediction;

[0030] Use the centroid coordinates of the rock slag output by U-Net as a point prompt, and construct the point prompt input_point and input label (input_label=1, the label of the rock slag);

[0031] Call predictor.predict() to generate three sets of candidate masks and their confidence scores. The parameter multimask_output = True enables multi-scale output.

[0032] Select the mask with the highest confidence score and filter out masks whose area exceeds 10% of the total image area;

[0033] The sixth step is mask post-processing and geometry optimization, specifically:

[0034] Use the measure_label() function to mark the connected areas of each mask, retaining only the largest connected areas to eliminate fragmentation noise;

[0035] Use measure.find_contours() function to extract the mask boundary point set. If the mask position is at the edge of the image, zero padding with a pixel width of 1 is performed at the edge position;

[0036] Recalculate the contour based on the filled mask and convert the mask into a Polygon object;

[0037] Build an R-tree index for all polygon objects, quickly filter polygons ploy2 that may overlap with the current polygon poly1, and save polygons that do not overlap;

[0038] Use NetworkX to model overlapping polygon relationships as an undirected graph and identify overlapping polygon sets using the nx.connected_components(g) function.

[0039] Calculate the average intersection over union (mIoU) of each overlapping polygon with other polygons in turn, and select the polygon with the largest mIoU as the reference polygon. The calculation formula of mIoU is: Among them, A is polygon A, B i are polygons other than polygon A, and N is the number of overlapping polygons;

[0040] Calculate the difference between each polygon and the reference polygon, rasterize the difference area, and use the rock slag probability (channel 1) in step 4 as a reference to verify whether the difference area contains a valid rock slag area. The verification strategy is: use the difference area to retrieve the rock slag probability of channel 1. If the mean value of all probability points in the area exceeds 0.5, it is considered valid; otherwise, it is invalid.

[0041] Divide the valid overlapping area into polygons with larger areas, save the two resulting polygons, and delete the invalid overlapping area and its corresponding two polygons;

[0042] Set the erosion and dilation radius to 5 pixels, perform erosion and dilation operations on all polygons, and delete all polygons with an area less than 300 pixels to eliminate jagged edges and small noise.

[0043] The seventh step is the calculation and visualization of rock slag parameters, specifically:

[0044] Use rasterize_grains() to rasterize the polygons, use rasterio.transform to define the coordinate transformation, and convert the polygon vector data into integer labels with a label value of 1 and the remaining area (background) marked as 0;

[0045] Create a 2-pixel-wide buffer for each polygon, mark it as 2 after rasterization, and finally form a three-valued logic (0 represents background, 1 represents rock slag, and 2 represents rock slag boundary);

[0046] The regionprops_table() function is called to extract the geometric parameters of each rock ballast pixel, such as area, perimeter, major axis length, minor axis length, and orientation angle. The actual geometric parameters of the rock ballast are calculated based on the calibrated ratio of 0.4 mm / pixel. The results are saved as an Excel file, and the cumulative particle size curve is drawn based on the major axis length data.

[0047] The elongation index (EI) and circularity index (CI) of the rock ballast are calculated based on the geometric parameters to describe the shape of the chip. EI and CI are defined as the ratio of the longest axis L to the shortest axis S of the minimum enclosing boundary and the ratio of the contour area to the area of ​​a circle with the same circumference, respectively. The calculation results are also saved as an Excel file.

[0048] Use matplotlib to draw a dual-axis visualization view, using a random color scheme to distinguish independent rock fragments;

[0049] Dynamically generate structured data tables containing size-morphology characteristics to enable real-time interactive analysis of rock debris characteristics during excavation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the overall process of a real-time identification method for rock debris in tunneling based on U-Net-SAM coupling drive provided by an embodiment of the present invention;

[0051] Figure 2 The original input image of a real-time identification method for tunnel excavation rock debris based on U-Net-SAM coupling drive provided by an embodiment of the present invention;

[0052] Figure 3 A U-Net preliminary prediction diagram of a real-time identification method for rock slag during tunneling based on U-Net-SAM coupling drive provided by an embodiment of the present invention;

[0053] Figure 4 A three-channel display diagram of a preliminary prediction of a real-time identification method for rock debris in tunneling based on U-Net-SAM coupling drive provided by an embodiment of the present invention;

[0054] Figure 5 A diagram showing prompt points for a real-time identification method for rock debris in tunneling based on U-Net-SAM coupling drive, provided in an embodiment of the present invention;

[0055] Figure 6This is a diagram showing the error of SAM segmentation in a real-time identification method for rock ballast in tunneling based on U-Net-SAM coupling drive provided by an embodiment of the present invention;

[0056] Figure 7 This is an output diagram after post-processing optimization of a real-time identification method for rock slag in tunneling based on U-Net-SAM coupling drive provided by an embodiment of the present invention;

[0057] Figure 8 A graph showing a cumulative curve of rock slag particle size for a real-time identification method for rock slag during tunneling based on U-Net-SAM coupling drive provided by an embodiment of the present invention;

[0058] Figure 9 A schematic diagram of the result storage of a real-time identification method for rock slag in tunneling based on U-Net-SAM coupling drive provided by an embodiment of the present invention; DETAILED DESCRIPTION

[0059] To simplify the operational process, the present invention utilizes a highly integrated code. Specifically, by transferring a captured image of the slag as input to the model, the size and morphology of the slag can be obtained. The following description provides numerous specific details to facilitate a thorough understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0060] See also Figures 1 to 9 The embodiment of the present invention provides a real-time identification method for rock debris in tunneling based on U-Net-SAM coupling drive, comprising the following steps:

[0061] First, transfer the slag image, U-net's .h5 model file, and SAM's pre-trained .pth file to the model. The original slag image is as follows: Figure 2 As shown; Subsequently, the U-net network makes a preliminary prediction on the image, and the prediction results are as follows Figure 3 As shown; the prediction results are split into 3 channels, such as Figure 4 As shown, the three channels store the prediction information of rock slag, boundary and background respectively.

[0062] After completing the preliminary prediction of the rock slag image, channel 1 and channel 2 are binarized with a threshold of 0.5. The connected domain labeling algorithm of the scikit-image library is used to obtain the independent rock slag area of ​​channel 1. The area and centroid coordinate attributes of each region are extracted through regionprops_table to generate an initial centroid coordinate set. Subsequently, the initial centroids are screened in combination with channel 0, and the coordinate points with background probability values ​​higher than 0.3 are removed to reduce the false detection rate. The distance transform and watershed algorithm are used to reconstruct the rock slag boundary of channel 2, and the local maximum points are located to generate secondary centroid coordinates. The dense centroids are spatially clustered using DBSCAN clustering (neighborhood radius dbs_max_dist = 1.0), and redundant detection points are eliminated by mean coordinate fusion. Finally, the primary centroid set and the optimized secondary centroid set are coordinate superimposed to construct a composite prompt point matrix covering the actual rock slag position. All prompt points are shown as follows Figure 5 shown.

[0063] Then the prompt point matrix and input image are transferred to the SAM model, and the image is segmented twice using the zero-sample segmentation capability of SAM. The segmentation error is shown as follows: Figure 6 As shown. Figure 6 In (a), there are some boundary noise and background noise. Figure 6 (b) shows the number of mask overlaps in the form of a heat map. In order to obtain high-quality segmentation results, further post-processing optimization is required. The optimized results are as follows: Figure 7 Finally, perform geometric calculations on the output results, draw the cumulative curve of the rock ballast particle size, and export the results as an Excel file, as shown in the following example: Figure 8 and Figure 9 shown.

[0064] The above-described embodiment merely represents one embodiment of the present invention. While the description is relatively specific and detailed, it should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A real-time identification method for tunnel excavation rock debris based on U-Net-SAM coupling drive, characterized in that: The following steps are involved: In the first step, soil photos were collected from the TBM conveyor belt during tunnel excavation to serve as model input. The second step is to build a U-Net network, confirm the model's input and output formats, and select an appropriate loss function. The third step is to use block prediction and result combination methods to make a preliminary prediction of the output image. The fourth step is to generate the centroid coordinates of the rock slag based on the U-Net segmentation results and optimize the prompt point set. In the fifth step, the zero-shot segmentation capability of the SAM model is used to generate an initial mask. In the sixth step, the mask is post-processed and geometrically optimized to generate a high-quality rock ballast mask. The seventh step is to visualize the extracted size and shape parameters of the rock slag. The equivalent circle diameter is calculated through the optimized rock slag mask contour to quantify the rock slag size, and the morphological parameters of the crushed stone are calculated by combining the aspect ratio of the minimum circumscribed rectangle and the contour perimeter roundness ratio.

2. The real-time identification method for tunneling rock debris based on U-Net-SAM coupling drive according to step 2 of claim 1 is characterized by: The constructed U-net network uses a weighted cross entropy loss function, which is calculated as follows: Where N is the number of samples, C is the number of categories, in this paper C = 3 (the values ​​corresponding to rock slag, boundary and background are 1, 2, and 0 respectively), y ture is the one-hot encoding of the actual category label, with a shape of (N, C), y pred is the logits output by the model, and the shape is also (N, C), w i is the weight of category i, and the category weight matrix is ​​set to [0.6, 1.0, 5.0], corresponding to the classification processing of background, rock debris, and boundary.

3. The real-time identification method for tunneling rock debris based on U-Net-SAM coupling drive according to step 3 of claim 1 is characterized by: Read the captured image as the input image and check whether the length and width of the image are multiples of 512. If not, add several blank rows and columns on the bottom and right sides of the image to make them multiples of 512. Crop the input image into several 512×512 pixel image blocks in the form of a sliding window with a stride of 256. Construct a double loop structure to complete horizontal and vertical block-by-block reasoning. The output shape of each image block is (512, 512, 3). The prediction value of the overlapping area is weighted by the Hanning window weight. The Hanning vector is defined as: Where N = 512 is the length of the Hanning vector, n is the index, and the Hanning window weight matrix can be obtained by performing the outer product of two Hanning vectors. The output results of the image blocks are multiplied by the Hanning window weight matrix one by one, then spliced ​​and the blank rows and columns filled in are cropped to obtain the segmentation result of the complete image.

4. The method for real-time identification of rock debris during tunneling based on U-Net-SAM coupling drive according to step 4 of claim 1 is characterized in that: Channel 1 of the complete image segmentation result was binarized with a threshold of 0.

5. The connected component labeling algorithm of the scikit-image library was used to obtain independent rock slag regions. The area and centroid coordinate properties of each region were extracted using regionprops_table to generate an initial centroid coordinate set. Channel 0 of the complete image segmentation result was combined to screen the initial centroids, and coordinate points with background probability values ​​higher than 0.3 were removed to reduce the false detection rate. Channel 2 of the complete image segmentation result was extracted and binarized with a threshold of 0.

5. The rock slag boundary was reconstructed using distance transformation and watershed algorithm. Local maximum points were located to generate secondary centroid coordinates. DBSCAN clustering was used with a neighborhood radius of dbs_max_dist=1.0 to spatially cluster dense centroids. Redundant detection points were eliminated through mean coordinate fusion. The primary centroid set and the optimized secondary centroid set were coordinate-superimposed to construct a composite prompt point matrix covering the actual rock slag location.

5. The method for real-time identification of rock debris during tunneling based on U-Net-SAM coupling drive according to step 6 of claim 1 is characterized in that: The connected areas of each mask are marked using the measure_label() function, retaining only the largest connected areas and eliminating fragmented noise. The measure.find_contours() function is used to extract the mask boundary point set. If the mask position is at the edge of the image, the edge position is zero-filled with a pixel width of 1. The contour is recalculated based on the filled mask and the mask is converted into a polygon (Polygon) object. An R-tree index is constructed for all polygon objects to quickly filter polygons ploy2 that may overlap with the current polygon poly1 and save polygons that do not overlap. The overlapping polygon relationship is modeled as an undirected graph using NetworkX, and the overlapping polygon set is identified using the nx.connected_components(g) function. Calculate the average intersection over union (mIoU) of each polygon with other polygons in the overlapping polygon set in turn, and select the polygon with the largest mIoU as the reference polygon. The calculation formula of mIoU is: Among them, A is polygon A, B i are overlapping polygons except polygon A, and N is the number of overlapping polygons; Calculate the difference between each polygon and the reference polygon, rasterize the difference area, and use channel 1 in the fourth step as a reference to verify whether the area contains a valid rock slag area. The verification strategy is: use the difference area to retrieve the rock slag probability of channel 1. If the mean value of all probability points in the area exceeds 0.5, it is considered valid, otherwise it is invalid; divide the valid overlapping area into polygons with larger areas, save the two obtained polygons, and delete the invalid overlapping area and its corresponding two polygons; set the erosion and expansion radius to 5 pixels, perform erosion and expansion operations on all polygons, delete all polygons with an area less than 300 pixels, and eliminate jagged edges and small noise.

6. The method for real-time identification of rock debris during tunneling based on U-Net-SAM coupling drive according to step 7 of claim 1, characterized in that: Use rasterize_grains() to rasterize the polygons, use rasterio.transform to define the coordinate changes, convert the polygon vector data into integer labels, the label value is 1, and the rest of the area is marked as 0; create a buffer of 2 pixels wide for each polygon, and mark it as 2 after rasterization; call the regionprops_table() function to extract the area, perimeter, major axis length, minor axis length, direction angle and other geometric parameters of each rock ballast pixel, and convert the actual geometric parameters of the rock ballast according to the calibrated ratio of 0.4mm / pixel. The result is saved as an Excel file, and the cumulative curve of grain size is drawn with the data of major axis length; calculate the elongation index (EI) and circularity index (CI) of the rock ballast based on the geometric parameters to describe the shape of the chip. EI and CI are defined as the ratio of the longest axis L to the shortest axis S of the minimum enclosing boundary, and the ratio of the contour area to the area of ​​a circle with the same perimeter, respectively. The calculation results are also saved as an Excel file; A dual-axis visualization is plotted using matplotlib, using a random color scheme to distinguish independent rock fragments.

Images