Coal rock fracture intelligent extraction method based on improved U-Net

Abstract

The invention discloses a coal rock fracture intelligent extraction method based on improved U-Net. The method comprises the following steps: S1, constructing a coal rock fracture CT image data set; s2, constructing an improved U-Net segmentation model, specifically comprising the following steps: S2.1, taking VGG16 as a backbone network, and introducing a depth separable convolution module; s2.2, a PPA attention module is added after each layer of depth separable convolution of the decoder, the PPA attention module is introduced after each up-sampling stage of the decoder, and the output of the PPA attention module is subjected to batch normalization and Dropout layer processing; s2.3, defining a composite loss function; s3, training and optimizing a segmentation model, wherein the specific steps comprise: S3.1, setting hyper-parameters; and S3.2, training the model by using the training set, adjusting hyper-parameters by using the verification set, and evaluating the performance by using the test set, wherein the evaluation indexes comprise MIoU, MAcc and FWIoU. According to the method, the problems of difficult identification of small fractures, large model calculation amount, poor multi-scale information fusion and class imbalance in the coal rock fracture image can be solved, and the robustness, segmentation precision and practicability of the model are improved.

Description

Technical Field

[0001] The present invention belongs to the cross-technical field of artificial intelligence and mine image processing, and particularly relates to an intelligent coal and rock fracture extraction method based on an improved U-Net. Background Art

[0002] CT image segmentation is widely used in coal rock fracture research, enabling in-depth exploration of the internal structure of coal rocks and is crucial for coal resource extraction and coal mine safety. Traditional image segmentation methods include gray-level co-occurrence matrices and support vector machines, as well as gradient and morphological operations such as Sobel and Canny. These methods can achieve segmentation of coal rock fractures to a certain extent. However, these methods have the following shortcomings: noise and uneven illumination in coal rock images cause significant errors in the results of methods such as threshold segmentation and edge detection. Furthermore, their multi-scale feature representation capabilities are limited, making it difficult to balance microcrack pixel details with macrocrack network structures, leading to scale discontinuities during 3D reconstruction. Furthermore, these traditional methods lack intelligence, rely heavily on human experience, and lack adaptive learning capabilities.

[0003] With the rise and continuous development of deep learning, convolutional neural network-based segmentation algorithms have continuously emerged. Classic semantic segmentation algorithms such as FCN, U-Net, DeeplabV3+, PSPNet, and HRNet have all contributed to improving segmentation accuracy and efficiency. However, the following challenges remain. Coal mining equipment suffers from limited computing resources, and traditional deep learning models are computationally intensive. These models respond too slowly under complex operating conditions, failing to meet the real-time response requirements of underground mines. Existing models suffer from incomplete edge extraction and misjudgment of intersection points for coal rock fractures with unclear boundaries and complex scales of fracture intersections, such as fragmented, overlapping, and intersecting fractures. The cross-scale distribution of coal rock fracture CT images, ranging from micropores to macrofractures, requires models that can fuse multi-scale features for segmentation. However, existing models are not effective at integrating fracture features at different scales, and their ability to segment the coexistence of micropores and macrofractures is insufficient. Furthermore, faced with the "foreground-background" class imbalance in coal rock images, segmentation models are less effective and general-purpose models are difficult to apply. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent coal rock fracture extraction method based on an improved U-Net, which can solve the problems of difficult identification of small cracks in coal rock fracture images, large model calculation amount, poor multi-scale information fusion and category imbalance, and improve the robustness, segmentation accuracy and practicality of the model.

[0005] To achieve the above objectives, the present invention provides an intelligent coal and rock fracture extraction method based on an improved U-Net, comprising the following steps: S1. Constructing coal rock fracture CT image dataset; S2. Build an improved U-Net segmentation model, which includes the following steps: S2.1, using VGG16 as the backbone network, introduces the depthwise separable convolutional module; S2.2. Add a PPA attention module after each depth-wise separable convolution in the decoder. The PPA attention module is introduced after each upsampling stage of the decoder, and its output is processed by batch normalization and dropout layers. S2.3. Define the composite loss function; S3. Separate model training and optimization. The specific steps include: S3.1, Hyperparameter settings; S3.2. Use the training set to train the model, the validation set to adjust the hyperparameters, and the test set to evaluate the performance. The evaluation indicators include MIoU, MAcc, and FWIoU.

[0006] As a further solution of the present invention: the specific steps in S1 include: S1.1. Collect industrial CT scan images of coal sample fracturing experiments, perform Gaussian filtering on the original images to reduce noise, and use the Labelme tool to perform pixel-level annotation of the fracture areas; S1.2, use data augmentation operations to expand the dataset; S1.3. Divide the training set, validation set, and test set into a ratio of 7:1:2.

[0007] As a further solution of the present invention: the specific steps in S2 include: S2.1. The depthwise separable convolution module consists of a series of depthwise convolution and pointwise convolution. The depthwise convolution uses a 3×3 convolution kernel to independently process the spatial features of each input channel, and the pointwise convolution uses a 1×1 convolution kernel to achieve cross-channel feature fusion. S2.2, the PPA attention module includes a multi-scale feature extraction layer and an attention enhancement layer. The multi-scale feature extraction layer consists of a multi-branch path composed of Skip connection and three 3×3 convolutions, and fuses features through residual addition. The attention enhancement layer contains two parallel LocalGlobalAttention submodules, which divide the feature map into 2×2 and 4×4 blocks to generate multi-scale weights. It combines the ECA channel attention mechanism and the spatial attention mechanism. The ECA channel attention calculates the channel weight through global average pooling combined with one-dimensional convolution. The spatial attention generates a spatial weight map through 7×7 convolution through the maximum pooling and average pooling results, and finally undergoes batch normalization and dropout processing. S2.3. The composite loss function consists of Dice loss and Focal loss. The Dice loss function formula is as follows: ; In formula (1), is the predicted segmentation result of the segmentation model, is the true segmentation label, is a constant; The Focal loss function formula is as follows: ; In formula (2), is the predicted probability, To adjust the category weights, To control the degree of attention paid to difficult and easy samples; The specific expression of the composite loss function is as follows: ; In formula (3), is the Dice loss function, is the Focal loss function, is the weight coefficient.

[0008] As a further solution of the present invention: In S3, MAcc is the average category accuracy. The accuracy is calculated for each category separately, and then the average value of all categories is taken to equally evaluate the classification ability of the model in each category. The formula is: ; In formula (4), TP is the number of pixels correctly predicted as a certain category, and FN is the number of pixels of a certain category that are incorrectly predicted as other categories.

[0009] As a further solution of the present invention: In S3, MIoU is the mean intersection over union ratio, and the IoU of each category is calculated as the ratio of the number of pixels in that category to the total number of pixels in all categories. MIoU is the average of all category IoUs, and the formula is: ; In formula (5), TP is the number of pixels correctly predicted as a certain category, FN is the number of pixels of a certain category that are incorrectly predicted as other categories, and FP is the number of pixels incorrectly predicted as a certain category.

[0010] As a further solution of the present invention: In S3, FWIoU is the frequency-weighted intersection-over-union (IoU), which is improved on the basis of MIoU. The class pixel frequency is introduced as the weight, and the number of pixels of each class in the true label (TP+FN) is used as the weight to perform weighted average of the IoU of each class. The formula is: ; In formula (6), TP is the number of pixels correctly predicted as a certain category, FN is the number of pixels of a certain category that are incorrectly predicted as other categories, and FP is the number of pixels incorrectly predicted as a certain category.

[0011] Compared with the prior art, the present invention has the following beneficial effects: By introducing a depthwise separable convolution module, the number of parameters and computational overhead of the segmentation model are reduced; by introducing a PPA attention module, the segmentation model's ability to recognize tiny cracks, blurred edge areas, and crack cross-structures is enhanced; by implementing a Focal+Dice composite loss function, the segmentation model's recognition accuracy for images with an imbalanced ratio of positive and negative samples is improved; through experiments on coal and rock crack image data, the crack recognition accuracy of the segmentation model is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flow chart of the steps of the present invention.

[0013] Figure 2 It is a schematic diagram of the overall structure of the segmentation model.

[0014] Figure 3 It is a structural diagram of the depth-wise separable convolution module.

[0015] Figure 4 It is a structural diagram of the PPA attention module.

[0016] In the figure: 1. 3×3 convolution, batch normalization, rectified linear unit, 2. Depthwise separable convolution, 3. 2×2 max pooling, 4. Bilinear interpolation, 5. Concatenation, 6. PPA, 7. 1×1 convolution, 8. Depthwise convolution, 9. Point-by-point convolution, 10. Attention mechanism, 11. Random dropout (0,1) + batch normalization + rectified linear unit. DETAILED DESCRIPTION

[0017] The present invention will be further described below with reference to the accompanying drawings.

[0018] like Figure 1 As shown, an intelligent coal rock fracture extraction method based on improved U-Net includes the following steps: S1. Constructing coal rock fracture CT image dataset; Furthermore, the specific steps in S1 include: S1.1. Collect industrial CT scan images of coal sample fracturing experiments, perform Gaussian filtering to reduce noise on 550 original images, and use the Labelme tool to annotate the fracture areas at the pixel level. S1.2. Data augmentation operations were used to expand the dataset, including random translation, mirror flipping, dynamic brightness adjustment, and the addition of Gaussian noise and random scattered noise, ultimately expanding the original 550 images to 5500. S1.3. Divide the training set, validation set, and test set into a ratio of 7:1:2.

[0019] S2. Build an improved U-Net segmentation model. The overall structure diagram of the segmentation model is as follows: Figure 2 As shown, the specific steps include: S2.1. Using VGG16 as the backbone network, we introduce a depthwise separable convolution module. The depthwise separable convolution module consists of depthwise convolution and pointwise convolution in series. The structure diagram of the depthwise separable convolution module is shown in the figure. Figure 3 As shown in the figure, the depth convolution uses a 3×3 convolution kernel to independently process the spatial features of each input channel, and the point-by-point convolution uses a 1×1 convolution kernel to achieve cross-channel feature fusion; Among them, the number of parameters of the depthwise separable convolution module is about 1 / 8 of that of the traditional convolution, and the convolution kernel size is fixed at 3×3.

[0020] S2.2. Add a PPA attention module after each layer of depth-separable convolution in the decoder. The structural diagram of the PPA attention module is as follows: Figure 4 As shown in the figure, in order to more fully extract the contextual feature information of the image, the PPA attention module is introduced after each upsampling stage of the decoder. The PPA attention module contains a multi-scale feature extraction layer and an attention enhancement layer. The multi-scale feature extraction layer consists of a multi-branch path composed of Skip connection and three 3×3 convolutions, and the features are fused by residual addition. The attention enhancement layer contains two parallel LocalGlobalAttention sub-modules, which divide the feature map into 2×2 and 4×4 blocks to generate multi-scale weights, combined with the ECA channel attention mechanism and the spatial attention mechanism. The ECA channel attention calculates the channel weight through global average pooling combined with one-dimensional convolution, and the spatial attention generates a spatial weight map through 7×7 convolution through the maximum pooling and average pooling results, and finally undergoes batch normalization and Dropout processing; S2.3. Define the composite loss function. The composite loss function consists of Dice loss and Focal loss. The Dice loss function formula is as follows: ; In formula (1), is the predicted segmentation result of the segmentation model, is the true segmentation label, is a constant, preferably a smaller constant, to prevent the denominator from being zero; The Focal loss function formula is as follows: ; In formula (2), is the predicted probability, To adjust the category weights, In order to control the degree of attention of difficult and easy samples, The larger the value, the more the model focuses on difficult-to-classify samples, effectively improving the segmentation ability of pixels that are easily misclassified as cracks. In order to focus on difficult-to-classify samples, The value is set to 2, for crack pixels The value is set to 0.8, for background pixels Set the value to 0.2.

[0021] The specific expression of the composite loss function is as follows: ; In formula (3), is the Dice loss function, is the Focal loss function, is the weight coefficient. The weight coefficient is used to balance the effects of the two loss functions.

[0022] In order to alleviate the problem of imbalance between positive and negative samples, The value is set to 0.5, so that the advantage of the Dice loss function in focusing on the overlap of the target area and the characteristic of the Focal loss function in focusing on difficult samples complement each other.

[0023] S3. Segmentation model training and optimization. The specific steps include: S3.1. Hyperparameter settings: preferably, 100 training rounds, 8 batch size, 0.0001 initial learning rate, AdamW optimizer, 1e-4 weight decay, and CosineAnnealingLR learning rate scheduler. S3.2. Use the training set to train the model, the validation set to adjust the hyperparameters, and the test set to evaluate the performance. The evaluation indicators include MIoU, MAcc, and FWIoU.

[0024] Furthermore, in S3, MAcc is the average category accuracy, which is created to address the problem of Acc deviation. The accuracy is calculated for each category separately (the proportion of TP of a certain category to the total number of real pixels of that category), and then the average of all categories is taken to equally evaluate the classification ability of the model in each category. The formula is: ; In formula (4), TP is the number of pixels correctly predicted as a certain category, and FN is the number of pixels of a certain category that are incorrectly predicted as other categories.

[0025] Furthermore, in S3, MIoU is the mean intersection over union (IoU), which is the most commonly used indicator for semantic segmentation. It measures the overlap accuracy between the predicted area and the true area. The IoU of each category is calculated as the ratio of the number of pixels in that category to the total number of pixels in all categories. MIoU is the average of the IoUs of all categories, and the formula is: ; In formula (5), TP is the number of pixels correctly predicted as a certain category, FN is the number of pixels of a certain category that are incorrectly predicted as other categories, and FP is the number of pixels incorrectly predicted as a certain category.

[0026] Furthermore, in S3, FWIoU is the frequency-weighted intersection over union (IoU), which is improved on the basis of MIoU. The category pixel frequency is introduced as the weight to highlight the importance of high-frequency categories. The number of pixels of each category in the true label (TP+FN) is used as the weight to perform weighted average of the IoU of each category. The formula is: ; In formula (6), TP is the number of pixels correctly predicted as a certain category, FN is the number of pixels of a certain category that are incorrectly predicted as other categories, and FP is the number of pixels incorrectly predicted as a certain category.

[0027] In terms of network architecture, this paper introduces a depthwise separable convolution module into the VGG16 backbone network. This module decomposes traditional convolution into two independent operations: depthwise convolution and pointwise convolution. The depthwise convolution uses a 3×3 convolution kernel to process the spatial features of each input channel independently, while the pointwise convolution uses a 1×1 convolution kernel to achieve cross-channel feature fusion. This design significantly reduces the number of model parameters, thereby improving model inference speed, while retaining the first-layer standard convolution to maintain the ability to extract shallow texture features. Regarding the feature extraction mechanism, a Parallelized Patch-aware Attention (PPA) module is embedded after each upsampling stage of the U-Net decoder. This module first captures local details through a multi-scale feature extraction layer. Skip connections preserve the original features, and three 3×3 convolution paths extract multi-level semantic information. Feature reuse is achieved through residual addition. The attention enhancement layer uses two parallel LocalGlobalAttention submodules to divide the feature map into 2×2 and 4×4 local patches, respectively, simulating pyramid pooling to construct a multi-scale perception system. Finally, the ECA channel attention and spatial attention mechanisms are connected in series. The ECA channel attention enhances the crack texture response by combining global average pooling with one-dimensional convolution. The spatial attention generates a spatial weight map by fusing the maximum pooling and average pooling results through 7×7 convolution to focus on the edge area. The output layer is processed by batch normalization and Dropout to improve the generalization ability of the model. To address the problem of sample imbalance, the Dice-Focal composite loss function is designed. The Dice loss enhances the overall grasp of the crack area by calculating the overlap between the predicted area and the real area, and the Focal loss focuses on the crack pixels that are difficult to segment. The two are separated by weight coefficients. Balanced, improving the recall rate of small fractures. On a test set of 5,500 coal and rock CT images, this method achieved a MIoU of 88.47%, a 1.75% improvement over the baseline U-Net, improving the model's accuracy in intelligent extraction of coal and rock fractures. The GPU inference time for a single image was 1.2 seconds, reducing the time required for intelligent extraction of coal and rock fractures.

Claims

1. An intelligent coal rock fracture extraction method based on improved U-Net, characterized in that: The following steps are involved: S1. Constructing coal rock fracture CT image dataset; S2. Build an improved U-Net segmentation model, which includes the following steps: S2.1, using VGG16 as the backbone network, introduces the depthwise separable convolutional module; S2.

2. Add a PPA attention module after each depth-wise separable convolution in the decoder. The PPA attention module is introduced after each upsampling stage of the decoder, and its output is processed by batch normalization and dropout layers. S2.

3. Define the composite loss function; S3. Segmentation model training and optimization. The specific steps include: S3.1, Hyperparameter settings; S3.

2. Use the training set to train the model, the validation set to adjust the hyperparameters, and the test set to evaluate the performance. The evaluation indicators include MIoU, MAcc, and FWIoU.

2. The method for intelligent extraction of coal and rock fractures based on improved U-Net according to claim 1, characterized in that: The specific steps in S1 include: S1.

1. Collect industrial CT scan images of coal sample fracturing experiments, perform Gaussian filtering on the original images to reduce noise, and use the Labelme tool to perform pixel-level annotation of the fracture areas; S1.2, use data augmentation operations to expand the dataset; S1.

3. Divide the training set, validation set, and test set into a ratio of 7:1:

2.

3. The method for intelligent extraction of coal and rock fractures based on improved U-Net according to claim 1 or 2, characterized in that: The specific steps in S2 include: S2.

1. The depthwise separable convolution module consists of a series of depthwise convolution and pointwise convolution. The depthwise convolution uses a 3×3 convolution kernel to independently process the spatial features of each input channel, and the pointwise convolution uses a 1×1 convolution kernel to achieve cross-channel feature fusion. S2.2, the PPA attention module includes a multi-scale feature extraction layer and an attention enhancement layer. The multi-scale feature extraction layer consists of a multi-branch path composed of Skip connection and three 3×3 convolutions, and fuses features through residual addition. The attention enhancement layer contains two parallel LocalGlobalAttention submodules, which divide the feature map into 2×2 and 4×4 blocks to generate multi-scale weights. It combines the ECA channel attention mechanism and the spatial attention mechanism. The ECA channel attention calculates the channel weight through global average pooling combined with one-dimensional convolution. The spatial attention generates a spatial weight map through 7×7 convolution through the maximum pooling and average pooling results, and finally undergoes batch normalization and dropout processing. S2.

3. The composite loss function consists of Dice loss and Focal loss. The Dice loss function formula is as follows: ； In formula (1), is the predicted segmentation result of the segmentation model, is the true segmentation label, is a constant; The Focal loss function formula is as follows: ; In formula (2), is the predicted probability, To adjust the category weights, To control the degree of attention paid to difficult and easy samples; The specific expression of the composite loss function is as follows: ; In formula (3), is the Dice loss function, is the Focal loss function, is the weight coefficient.

4. The method for intelligent extraction of coal and rock fractures based on improved U-Net according to claim 3, characterized in that: In S3, MAcc is the average category accuracy. The accuracy is calculated for each category separately, and then the average of all categories is taken to equally evaluate the classification ability of the model in each category. The formula is: ； In formula (4), TP is the number of pixels correctly predicted as a certain category, and FN is the number of pixels of a certain category that are incorrectly predicted as other categories.

5. The method for intelligent extraction of coal and rock fractures based on improved U-Net according to claim 3, characterized in that: In S3, MIoU is the mean intersection over union (IoU). The IoU of each category is calculated as the ratio of the number of pixels in that category to the total number of pixels in all categories. MIoU is the average of all categories of IoU. The formula is: ; In formula (5), TP is the number of pixels correctly predicted as a certain category, FN is the number of pixels of a certain category that are incorrectly predicted as other categories, and FP is the number of pixels incorrectly predicted as a certain category.

6. The method for intelligent extraction of coal and rock fractures based on improved U-Net according to claim 5, characterized in that: In S3, FWIoU is the frequency-weighted intersection over union (IoU), which is an improvement on MIoU. It introduces the category pixel frequency as the weight, and uses the number of pixels of each category in the true label (TP+FN) as the weight to perform weighted average of the IoU of each category. The formula is: ； In formula (6), TP is the number of pixels correctly predicted as a certain category, FN is the number of pixels of a certain category that are incorrectly predicted as other categories, and FP is the number of pixels incorrectly predicted as a certain category.

Images