A method for identifying fractures in roadway surrounding rock based on semantic segmentation and joint loss function
By using a method for identifying rock fissures in roadways based on semantic segmentation and joint loss function, and utilizing borehole inspection data and an improved Deeplabv3+ model, the problem of high manpower and computational resource consumption and low identification accuracy of traditional methods is solved, and efficient and automated identification and quantification of rock fissures are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNIV OF MINING & TECH
- Filing Date
- 2023-12-12
- Publication Date
- 2026-05-26
AI Technical Summary
Traditional methods for observing and analyzing borehole fractures require a lot of manpower and time, and have low identification accuracy. Deep learning models have a large computational load and little room for parameter optimization, making it difficult to quickly and accurately extract the characteristics of surrounding rock fractures.
A method for identifying rock fissures in roadways based on semantic segmentation and joint loss function is adopted. Video data is collected through a borehole peephole, and data augmentation and annotation are performed. The Deeplabv3+ model is improved by introducing the CBAM attention mechanism and joint loss function, and pixel-level segmentation and quantization are achieved by combining two-stage transfer learning.
It achieves high-precision automated identification of borehole fractures in surrounding rock, saving manpower and time, reducing computational resource requirements, improving the model's noise resistance and generalization ability, and providing a reliable basis for evaluating the stability of surrounding rock.
Smart Images
Figure CN117809177B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of semantic segmentation technology for borehole fractures in roadways based on deep learning, specifically to a method for identifying roadway fractures based on semantic segmentation and a joint loss function. Background Technology
[0002] The analysis and evaluation of borehole fractures in roadway surrounding rock has become an important research direction in the field of intelligent mining engineering. Pathological analysis and classification of fracture development are crucial for on-site evaluation of surrounding rock stability. With roadway excavation, the three major stresses within the surrounding rock are redistributed, disrupting the original equilibrium state. This leads to the deformation, development, and evolution of fractures within the coal body, forming loosened zones, fractured development areas, and other pathological fracture zones. This poses a significant threat to the construction of safe, green, and transparent mining projects.
[0003] Traditional methods for observing and analyzing borehole fractures require significant manpower and time, and the accuracy of the results is difficult to guarantee. Furthermore, the low accuracy of traditional image detection algorithms is hampered by factors such as uneven lighting and coal dust interference from borehole inspection equipment. With the widespread application of computer vision in mining engineering, pixel-level semantic segmentation algorithms have provided a powerful and feasible method for the rapid and accurate classification of fracture pathological development. Image-based fracture pathology segmentation and recognition demonstrates significant advantages over traditional observation methods, borehole imaging methods, and digital core columnar sections.
[0004] Commonly used deep learning-based fracture extraction algorithms are computationally intensive, have limited optimization space for model parameter operations, require manual resizing of the dataset before model training, have high image resolution requirements, low classification and recognition efficiency, suffer significant loss in extracting fracture features from small target edges, and lack detailed explanations of fracture parameter quantification methods. Therefore, there is an urgent need for a tunnel surrounding rock fracture identification method based on semantic segmentation and joint loss functions to address the above problems. Summary of the Invention
[0005] To address the aforementioned technical shortcomings, the present invention aims to provide a method for identifying fractures in roadway surrounding rock based on semantic segmentation and a joint loss function. This method primarily optimizes the modular structure of existing semantic segmentation detection models and introduces a joint loss function. While reducing the model size, it ensures accurate and rapid extraction of depth feature information from the rock strata for annular fractures, longitudinal fractures, and fractured zones, and obtains microscopic fracture quantification indicators. This achieves automated identification of borehole fractures in the surrounding rock, providing a more reliable reference for evaluating roadway surrounding rock instability.
[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:
[0007] This invention provides a method for identifying fractures in roadway surrounding rock based on semantic segmentation and joint loss function, comprising the following steps:
[0008] S100: A large amount of video data on the internal condition of the surrounding rock is collected using a borehole sight. A database of borehole fractures in the surrounding rock is established by randomly extracting frames from the video and then finely annotating them to obtain three label types: annular fractures, longitudinal fractures, and fractured zones.
[0009] S200: Data augmentation is performed by calling the Augmentor module method in Python software to expand the surrounding rock borehole fracture dataset, and pixel binary equalization preprocessing is performed.
[0010] S300: The Deeplabv3+ semantic segmentation algorithm model, improved by introducing the CBAM attention mechanism and joint loss function, performs pixel-level segmentation on the enhanced borehole dataset;
[0011] S400: Introduces a two-stage transfer learning training model based on the "Cityscapes dataset - self-made laboratory rock fracturing dataset";
[0012] S500: The improved Deeplabv3+ model trained above is used to classify and identify the borehole fracture test set, and the segmentation effect is evaluated using the intersection-over-union ratio and average pixel accuracy.
[0013] S600: Import the extracted fracture prediction and identification feature results into ImageJ software for quantitative extraction of surrounding rock fracture orientation, maximum length, maximum width, area, and perimeter.
[0014] Preferably, the method for constructing the surrounding rock borehole fracture sample dataset and the preprocessing method in step S100 are as follows:
[0015] S1001. Record fracture pathology videos using a borehole inspection instrument, and randomly sample the exported videos to obtain a dataset.
[0016] S1002. Based on the characteristics of crack development, the cracks are divided into four label types: background pixels, annular cracks, longitudinal cracks, and fractured areas. Labelme software is used to finely annotate the crack contours, unlike target detection tasks which only provide detection bounding boxes, thereby achieving pixel-level segmentation and recognition.
[0017] S1003. Given that the model integrates a module to reset the image pixel size, the image size of 512Pixel×512Pixel will no longer be divided.
[0018] Preferably, the data augmentation and preprocessing steps of the borehole fracture dataset in step S200 include:
[0019] S2001. Data augmentation is performed using the Augmentor module in Python software. The methods include mirroring, scaling, random rotation, adding salt and pepper noise, and convolution processing to expand the borehole fracture dataset by a specified amount, further deepening the learning of fracture features. The dataset is divided into training set and validation set ratio of 9:1.
[0020] S2002. To address the issue of concentrated pixel distribution, MATLAB is used to perform binary equalization on the collected dataset to balance the pixel distribution and perform nonlinear stretching at the grayscale pixel level, thus avoiding the gradual loss of crack edge features during model training.
[0021] Preferably, the optimization model step in step S300 specifically includes:
[0022] In the S3001 network encoder stage, the lightweight MobileNetv2 is used to replace the traditional Xception backbone network; the CBAM module is split and inserted, the SA module is inserted into the bottom layer of the MobileNetv2 backbone to extract clear low-dimensional channel boundary information of the gaps, and the CA module and ASPP module are inserted in parallel to superimpose deep feature information. In the network decoder stage, the CBAM is inserted as a whole before the 4x bilinear interpolation upsampling to enhance the pixel prominence features.
[0023] S3002, in general, involves extracting two effective feature layers from the initial layer of the optimized Deeplabv3+ network and processing them differently. On one hand, shallow feature information is obtained through dimensionality reduction convolution using the MobileNetv2 backbone network, with the SA module paying more attention to spatial location features, resulting in a larger feature layer size and higher resolution. On the other hand, different dilated convolutions and layer-by-layer pooling are performed through the ASPP module in parallel with the CA module, resulting in a smaller semantic feature layer size. This requires 4x bilinear interpolation upsampling to restore the original size and fusing it with low-dimensional features to complete semantic segmentation and recognition.
[0024] S3003 introduces a joint loss function, namely the weighted cross-entropy loss function and the Dice-Loss function, to avoid the imbalance in recognition between the segmented background and different types of label pixels.
[0025] Preferably, in step S300, a two-stage transfer learning training model is introduced using the "Cityscapes dataset - self-made laboratory rock fracturing dataset". The model training is divided into two stages: freezing and thawing. By fine-tuning the pre-trained weight parameters of the fully connected layer, the pre-trained weight ratio of the borehole fracture test set is identified and segmented to predict, thereby accelerating the model convergence speed and enhancing the extracted prominent features of the surrounding rock fractures.
[0026] The beneficial effects of this invention are as follows:
[0027] 1. This invention precisely quantifies relevant indicators such as the maximum length, maximum width, area, and perimeter of borehole fractures in surrounding rock at the pixel level, providing a feasible method for achieving high-precision, automated identification and parameter output.
[0028] 2. This invention classifies the target of surrounding rock borehole fracture segmentation into four types of labels: background, annular fracture, longitudinal fracture, and fractured zone, and performs pixel-level automated identification, saving a lot of human identification time.
[0029] 3. An optimized lightweight training backbone network is adopted. The application of the former inverse residual block depthwise separable convolution significantly reduces the complexity of model computation parameters and memory size. The Deeplabv3+ semantic segmentation model is improved by adding an attention mechanism, which improves the model feature learning speed and greatly reduces hardware resource consumption. Compared with traditional semantic segmentation algorithms, it has higher noise resistance and model generalization ability.
[0030] 4. This invention provides important ideas and methods for the detection of pathological development of fractures in surrounding rocks under complex backgrounds. Attached Figure Description
[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0032] Figure 1 A flowchart of a method for identifying roadway surrounding rock fissures based on semantic segmentation and joint loss function provided in an embodiment of the present invention;
[0033] Figure 2 A schematic diagram of the structure of a Deeplabv+3 model with an added attention mechanism provided in an embodiment of the present invention;
[0034] Figure 3 The images shown are randomly selected frames from the surrounding rock borehole fracture images, fracture zone type labels, fracture segmentation diagrams, and quantitative characterization parameters of the fracture parts (area, perimeter, maximum and minimum pixel values, skewness, etc., unit: cm) obtained in the embodiments of the present invention. Figure 2 Figure 3 To be mentioned in the specific embodiments below Detailed Implementation
[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0036] As attached Figure 1 As shown, the process of a method for identifying roadway surrounding rock fractures based on semantic segmentation and joint loss function includes:
[0037] S100. A large amount of video data on the internal condition of the surrounding rock is collected using a borehole sight. A database of borehole fractures in the surrounding rock is established by randomly extracting frames from the video and then finely annotated to obtain three label types: annular fractures, longitudinal fractures, and fractured zones.
[0038] S200. Data augmentation is performed by calling the Augmentor module method in Python software to expand the surrounding rock borehole fracture dataset, and pixel binary equalization preprocessing is performed.
[0039] S300: The Deeplabv3+ semantic segmentation algorithm model, which improves upon the introduction of the CBAM attention mechanism and joint loss function, performs pixel-level segmentation on the enhanced borehole dataset.
[0040] S400, Introducing a two-stage transfer learning training model based on the "Cityscapes dataset - self-made laboratory rock fracturing dataset";
[0041] S500. The improved Deeplabv3+ model trained above is used to classify and identify the borehole fracture test set, and the segmentation effect is evaluated using MIoU (Intersection over Union) and mPA (Mean Pixel Precision).
[0042] S600. After prediction, import the different types of label gap segmentation images into ImageJ software in batches, set the pixel legend ruler, and then you can get the actual length parameters.
[0043] The specific steps for constructing and labeling the three types of labeled surrounding rock borehole fracture datasets in step S100 are as follows:
[0044] A borehole inspection instrument was used to inspect the 5109 and 5017 working faces of Yushupo Coal Mine. The image size was 1920 pixels × 1080 pixels. 605 images of surrounding rock borehole fractures were obtained by randomly extracting frames from the video using Python programming software.
[0045] The surrounding rock borehole fracture images were manually annotated according to their contours using Labelme annotation software, serving as a positive example dataset. The borehole fracture dataset was then divided into four label types: background, annular fractures, longitudinal fractures, and fractured zones.
[0046] In step S200, the specific steps of data augmentation and pixel binarization equalization preprocessing performed by the Augmentor module are as follows:
[0047] Python programming software was used to mirror, scale, randomly rotate, add salt and pepper noise, and perform convolution processing on the images to expand the dataset of borehole fractures in the surrounding rock. MATLAB software was used to perform pixel binary equalization preprocessing on the images to balance the pixel distribution and perform grayscale pixel-level nonlinear stretching. The dataset was divided into training set: validation set = 9:1.
[0048] In step S300, as shown in the appendix Figure 2 The Deeplabv3+ semantic segmentation algorithm model, which incorporates the CBAM attention mechanism and joint loss function to improve the pixel-level segmentation process of the enhanced borehole dataset, is shown in the following steps:
[0049] In the network encoder stage, the lightweight MobileNetv2 is used instead of the traditional Xception backbone network; the CBAM module is split and inserted, the SA module is inserted into the bottom layer of the MobileNetv2 backbone to extract clear low-dimensional channel boundary information of the gaps, and the CA module and ASPP module are inserted in parallel to superimpose deep feature information. In the network decoder stage, the CBAM is inserted as a whole before the 4x bilinear interpolation upsampling to enhance the pixel prominence features.
[0050] A joint loss function (weighted cross-entropy loss function and Dice-Loss function) is introduced to avoid the imbalance in recognition between the background segmentation and the pixels of different types of labels.
[0051] In step S400, the two-stage transfer learning training model process of introducing the "Cityscapes dataset - self-made laboratory rock fracturing dataset" is as follows:
[0052] During the compression, tensile, and shear tests on coal and rock samples from a working face at Yushupo Coal Mine, a laboratory training model dataset was collected. Using a Motion-BLITZ-miniIMAG high-speed digital camera, more than 500 images were collected under various conditions, including close-range, high-speed, high-resolution, and balanced and sufficient illumination. Data augmentation methods were then used to expand the dataset to 2,000 images.
[0053] Download the Cityscapes dataset from the GitHub website, and place the two-stage dataset label files and original image files in the ImageSets folder and JPEGImage folder under the VOCdevkit subfolder, respectively. Use the improved Deeplabv3+ model to train and obtain the two-stage pre-trained weights, and then train the dataset on the surrounding rock borehole fractures.
[0054] In step S500, a comparison of the predicted segmentation effect with traditional semantic segmentation models such as FCN and U-Net is added, and the curve is plotted based on the MIoU value recorded during the training process. The improved model of this invention achieves an MIoU value of 95.35%.
[0055] In step S600, as shown in the appendix Figure 3 As shown, the detailed operation for crack quantization in ImageJ software is as follows: File-Open-Open prediction result map-Analyze-Set Scale-Known distance-Unit of length (set to cm)-SetMeasurements-Get(Area,Angle,Max_length,Max_width,Perimeter)-OK.
[0056] This invention improves the speed and accuracy of fracture feature extraction by introducing a lightweight MobileNetv2 network backbone to reduce network model resource overhead, realizes automated segmentation of fractures with different types of labels, addresses the huge subjective error, and provides pixel-level quantitative index data export. It provides a new feasible solution for the task of identifying the instability of surrounding rock caused by fracture development pathology in roadway.
[0057] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A method for identifying fractures in roadway surrounding rock based on semantic segmentation and joint loss function, characterized in that, Includes the following steps: S100: A large amount of video data on the internal condition of the surrounding rock is collected using a borehole sight. A database of borehole fractures in the surrounding rock is established by randomly extracting frames from the video and then finely annotating them to obtain three label types: annular fractures, longitudinal fractures, and fractured zones. S200: Data augmentation is performed by calling the Augmentor module method in Python software to expand the surrounding rock borehole fracture dataset, and pixel binary equalization preprocessing is performed. S300: The Deeplabv3+ semantic segmentation algorithm model, improved by introducing the CBAM attention mechanism and joint loss function, performs pixel-level segmentation on the enhanced borehole dataset; S400: Introduces a two-stage transfer learning training model based on the "Cityscapes dataset - self-made laboratory rock fracturing dataset"; S500: The improved Deeplabv3+ model trained above is used to classify and identify the borehole fracture test set, and the segmentation effect is evaluated using the intersection-over-union ratio and average pixel accuracy. S600: Import the extracted fracture prediction and identification feature results into ImageJ software for quantitative extraction of surrounding rock fracture orientation, maximum length, maximum width, area, and perimeter indicators; The optimization process in step S300 specifically includes: S3001, the network encoder stage, where the lightweight MobileNetv2 is used instead of the traditional Xception backbone network; The CBAM module is split and inserted, the SA module is inserted into the bottom layer of the MobileNetv2 backbone to extract clear low-dimensional channel boundary information of the gaps, and the CA module and ASPP module are inserted in parallel to superimpose deep feature information. In the network Decoder stage, the CBAM module is inserted as a whole before the 4x bilinear interpolation upsampling to enhance the pixel prominence features. S3002. After the two effective feature layers of the initial layer extracted from the optimized Deeplabv3+ network are processed through different procedures, on the one hand, shallow feature information is obtained by dimensionality reduction convolution through the MobileNetv2 backbone network. The SA module pays more attention to spatial location features, and its feature layer size is large and the resolution is high. On the other hand, different dilated convolutions and layer-by-layer pooling are performed through the ASPP module in parallel with the CA module. Its semantic feature layer size is small. It needs to be restored to its original size by 4 times bilinear interpolation upsampling and fused with low-dimensional features to complete semantic segmentation and recognition. S3003 introduces a joint loss function, namely the weighted cross-entropy loss function and the Dice-Loss function, to avoid the imbalance in the recognition between the background segmentation and the pixels of different types of labels. In step S300, a two-stage transfer learning training model is introduced using the "Cityscapes dataset - self-made laboratory rock fracturing dataset". The model training is divided into two stages: freezing and thawing. By fine-tuning the pre-trained weight parameters of the fully connected layer, the pre-trained weight ratio of the borehole fracture test set is identified and segmented to predict, thereby accelerating the model convergence speed and enhancing the extracted prominent features of the surrounding rock fractures.
2. The method for identifying roadway surrounding rock fractures based on semantic segmentation and joint loss function as described in claim 1, characterized in that, The specific steps for constructing the surrounding rock borehole fracture sample dataset and preprocessing method in step S100 are as follows: S1001, record fracture pathology videos using a borehole inspection camera, and randomly sample frames from the exported videos to create a dataset. S1002. Based on the characteristics of crack development, the cracks are divided into four label types: background pixels, annular cracks, longitudinal cracks, and fractured areas. Labelme software is used to finely annotate the crack contours, unlike target detection tasks which only provide detection bounding boxes, thereby achieving pixel-level segmentation and recognition. S1003. Given that the model integrates a module to reset the image pixel size, the image size of 512Pixel×512Pixel will no longer be divided.
3. The method for identifying roadway surrounding rock fractures based on semantic segmentation and joint loss function as described in claim 2, characterized in that, The data augmentation and preprocessing methods for the borehole fracture dataset in step S200 include: S2001, using the Augmentor module in Python software to perform data augmentation, including mirroring, scaling, random rotation, adding salt and pepper noise, and convolution processing to expand the borehole fracture dataset by a specified amount, further deepening the learning of fracture features. The dataset is divided into training set: validation set = 9:
1. S2002. To address the issue of concentrated pixel distribution, MATLAB is used to perform binary equalization on the collected dataset to balance the pixel distribution and perform nonlinear stretching at the grayscale pixel level, thus avoiding the gradual loss of crack edge features during model training.