A residual and multi-scale feature based U-Net image segmentation network
By using the U-Net image segmentation network based on residuals and multi-scale features, the problems of automated identification and accurate segmentation in shale fracture segmentation are solved, achieving efficient and accurate shale fracture segmentation and improving the robustness and generalization ability of the segmentation algorithm.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHWEST UNIV
- Filing Date
- 2023-06-16
- Publication Date
- 2026-07-03
AI Technical Summary
Existing 3D visualization software cannot effectively and autonomously identify and segment shale fractures. Traditional digital image processing methods require extensive preprocessing and algorithm design when segmenting rock fractures, have poor generalization ability, and cannot actively learn fracture characteristics.
A U-Net image segmentation network based on residual and multi-scale features is adopted. The semantic features of crack images at different scales are learned through 5 levels of residual convolution. Combined with a multi-scale feature fusion module and Focal Loss function, the crack segmentation accuracy is improved.
It achieves efficient and accurate segmentation of shale and mudstone fractures, improves the robustness of the segmentation algorithm and its ability to identify minute fractures, and reduces manual intervention and time consumption.
Smart Images
Figure CN122336293A_ABST
Abstract
Description
[0001] This application is the following application.
[0002] The application number is: 202310720965.7
[0003] Application date: June 16, 2023
[0004] The application is titled: Divisional Application for a U-Net Fracture Segmentation Method for Shale Based on Residuals and Multi-Scale Features. Technical Field
[0005] This invention relates to the field of image recognition, and more specifically to a U-Net image segmentation network based on residuals and multi-scale features. Background Technology
[0006] Shale and mudstone fractures are formed under abnormally high pressure, tectonic stress, and fluid erosion. Well-developed fractures facilitate the storage and migration of shale gas, and their evolutionary characteristics are a key reference indicator in shale gas development. Computed tomography (CT) is a reliable non-destructive testing method, initially used in medical imaging research and subsequently widely applied in earth sciences. CT can clearly observe the three-dimensional morphology of shale and mudstone, accurately reflecting the original topological structure of fractures, providing important guidance for shale gas development technologies such as wellbore stability analysis, reservoir fracturing, and microfracture sealing. However, most existing 3D visualization software characterizes materials based on traditional threshold segmentation methods and requires extensive manual processing and parameter adjustments of CT images by professionals based on experience. This makes it impractical to effectively and autonomously identify and segment shale and mudstone fractures, which is time-consuming and inaccurate. Therefore, efficient segmentation of fractures in shale and mudstone CT images is an important research topic.
[0007] In the study of rock microstructure, extracting fracture information is an indispensable step, and many scholars have proposed various image processing techniques for rock CT image segmentation. However, traditional digital image processing methods require extensive image preprocessing and algorithm design for rock fracture segmentation, have poor generalization ability, and cannot actively learn fracture features. Therefore, more robust segmentation algorithms are needed.
[0008] Therefore, how to address the problems existing in current technologies, given the complex nature of shale fractures and the difficulty in separating micro-fractures, is a crucial area of research that those skilled in the art urgently need to study. Summary of the Invention
[0009] In view of this, the present invention provides a U-Net image segmentation network based on residuals and multi-scale features to solve the problems existing in the background technology.
[0010] To achieve the above objectives, the present invention adopts the following technical solution: A U-Net image segmentation network based on residual and multi-scale features includes three parts: feature extraction, feature fusion, and output prediction. The feature extraction part uses 5 levels of residual convolution to process the feature map, with the number of residual modules being 3, 4, 6, 3, and 3, respectively, to learn the semantic features of crack images at different scales; The feature fusion part includes a multi-scale feature fusion module and four upsampling residual convolution operations. The feature fusion module is used to capture multi-scale feature information. The deconvolution upsampling reconstructs the compressed spatial information layer by layer, and the skip connections supplement the information lost during the encoding process to enable feature aggregation, so that the resolution of the predicted image output by the network is consistent with that of the original input image. The output prediction part uses 1×1 convolution to adjust the prediction map into a single channel, and then uses the Sigmoid function to make the pixel prediction probability value return to 0-1. Pixels with a probability value greater than a set threshold are identified as crack pixels, and those with a probability value less than the set threshold are background pixels. Finally, an accurate binary segmentation map of mudstone and shale cracks is output.
[0011] Optionally, the method also includes using the Focal Loss function to improve the balance of the shale fracture segmentation network based on the segmentation results and the true values. The Focal Loss function is defined as follows: ; ; In the formula: This is an adjustable coefficient. To predict the probability magnitude, This is the tag value.
[0012] Optionally, the segmentation accuracy can also be evaluated using a confusion matrix.
[0013] Optionally, the multi-scale fusion module includes 1×1 convolution, multi-scale convolution, and global mean pooling; multi-scale convolution uses depth-separable convolution with kernel sizes of 3, 5, and 7 to perform convolution operations; global mean pooling encodes image feature information from a global perspective; the feature maps output from each part of global mean pooling are concatenated by channels, and finally, 1×1 convolution is used to adjust the number of channels to obtain a feature map that aggregates multi-scale semantic information.
[0014] Optionally, it also includes nonlocal mean filtering for noise reduction of the region to be segmented.
[0015] Optionally, acquire grayscale CT images of mudstone and shale, and use a mask to crop out the edge information of the grayscale CT images as the region to be segmented.
[0016] As can be seen from the above technical solution, compared with the prior art, the present invention provides a U-Net shale fracture segmentation method based on residual and multi-scale features. First, the conventional convolution in the U-Net network structure is replaced with a residual module, increasing the network depth to fully learn fracture features. Second, a multi-scale feature fusion (MSFF) module is added to alleviate the problem of inaccurate segmentation of tiny fractures in shale CT images, thereby improving fracture segmentation accuracy. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0018] Figure 1 This is a diagram of the overall network structure of the present invention; Figure 2 This is a structural diagram of the residual module of the present invention; Figure 3 This is a structural diagram of the multi-scale feature fusion module of the present invention. Detailed Implementation
[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] See Figure 1 This invention discloses a U-Net method for segmenting shale fractures based on residuals and multi-scale features, comprising the following steps: S1: Acquire CT grayscale images of mudstone and shale, and use a mask to cut off the edge information of the CT grayscale image as the region to be segmented; S2: Construct a shale fracture segmentation network, input the region to be segmented into the shale fracture segmentation network, and output the segmentation result; S3: The loss function measures the deviation between the predicted result and the true value, which is crucial for improving the network's segmentation performance. The ordinary cross-entropy function is prone to getting stuck in local optima in imbalanced tasks, thus reducing segmentation performance. Focal Loss, compared to the binary cross-entropy function, has an additional adjustment factor that adaptively adjusts the learning weights during training. This means reducing the weight of simple samples and increasing the weight of inaccurately classified samples, allowing the network to focus on the difficult-to-classify samples, thus improving the overall model performance. This invention selects Focal Loss as the loss function to address the imbalance of positive and negative samples in shale fracture segmentation, focusing the learning weights more on a smaller number of fracture samples. The Focal Loss function is defined as follows: ; ; In the formula: This is an adjustable coefficient. To predict the probability magnitude, This is the tag value.
[0021] The network of this invention consists of three parts: feature extraction, feature fusion, and output prediction. The overall structure is as follows: Figure 1 As shown, each cuboid represents a multi-channel feature map. The number of image channels is labeled above the feature map, and the image resolution is labeled in the lower left corner. Arrows represent operations such as convolution, pooling, or skip connections. The feature extraction part uses 5 levels of residual convolution to process the feature map, with the number of residual modules being 3, 4, 6, 3, and 3, respectively, to fully learn the semantic features of the crack image at different scales. The feature fusion part includes a multi-scale feature fusion module and 4 upsampling residual convolution operations. The feature fusion module is used to capture multi-scale feature information. Deconvolution upsampling reconstructs the compressed spatial information layer by layer, and skip connections are used to supplement the information lost during the encoding process to achieve feature aggregation, so that the resolution of the network output prediction image is consistent with that of the original input image. The output prediction part first uses a 1×1 convolution to adjust the prediction map to a single channel, and then uses the Sigmoid function to normalize the pixel prediction probability value to 0-1. Pixels with a probability value greater than a set threshold are identified as crack pixels, and those less than the set threshold are background pixels. Finally, an accurate binary segmentation map of mudstone and shale cracks is output.
[0022] Furthermore, replacing the conventional convolutional layers in U-Net with residual blocks (Res-Blocks) in the encoding and decoding paths can enhance the network's feature learning ability. Residual blocks, such as... Figure 2 As shown. The Parameterized Corrected Linear Unit (PReLU) function is a variant of the classic Corrected Linear Unit Activation Function (ReLU). PReLU provides an adaptively learned linear component to the negative input of the neuron. To address the issue of ReLU negative input failure, replacing the ReLU in the residual module with PReLU can make the network more robust and increase its resistance to noise interference. The expression is as follows: ; The core formula of the residual module is shown below: ; in It is an identity mapping of the input, expressed as Figure 2 The short-circuit connection on the right side adjusts the channel dimension through a 1×1 convolution, enabling feature information to be directly transferred between low and high layers, thus ensuring the stability of the training gradient. The residual mapping part consists of two 3×3 convolutional layers. After the convolutional layers, a batch normalization (BN) layer and a PReLU activation function are added. BN can make the data distribution of each batch consistent and speed up the convergence of the network. It is a numerical addition operation, which allows the network to learn new features only based on the original input during the learning process, that is, to learn the residual. The residual features are easier to learn than the original features. Finally, the result of the residual mapping is passed through the PReLU activation function to obtain a non-linear output. When the residual is 0, only the identity mapping is performed, and the network skips the current layer and directly passes the training weights to the input of the next layer, which not only prevents gradient vanishing but also avoids additional computation.
[0023] Furthermore, such as Figure 3 As shown, the Multi-scale Feature Fusion Block (MSFF) is located at the bottom of the shale segmentation network and consists of three parts: 1×1 convolution, multi-scale convolution, and global average pooling (GAP). The 1×1 convolution extracts small target feature information pixel-by-pixel; the multi-scale convolution uses depthwise separable convolutions with kernel sizes of 3, 5, and 7. Depthwise separable convolutions reduce the number of parameters to lighten the network and utilize short-circuit connections to reduce gradient vanishing; global average pooling encodes image feature information from a global perspective, thereby improving the network's segmentation performance. The feature maps output from each part are concatenated, and finally, the number of channels is adjusted using a 1×1 convolution to obtain a feature map that aggregates multi-scale semantic information.
[0024] Specifically, in this embodiment, the sample used was mudstone and shale from the Ningming Basin. The sample was processed into a cylindrical shape with a diameter of 60 mm and a height of 40 mm. It was scanned using a Phoenix V | Tome | XM micron CT scanner. A layered scanning method was used to obtain 1426 CT sequence images with a resolution of 1612×1686. The mudstone and shale fractures showed obvious differences in quantity and morphology in the longitudinal direction of the rock mass. The original mudstone and shale CT grayscale images were sampled at intervals of 150 frames.
[0025] Because the internal fissures of the Ningming mudstone and shale are connected to the external environment, and fragments fall from the edges of the samples after weathering and fracturing, the network may misclassify the area between the fragments and the sample as fissures during segmentation. A circular mask is used to crop the image edge information as a new region to be segmented. While traditional filtering algorithms such as median filtering and mean filtering suppress noise to some extent, they blur the edge details of the image. The nonlocal mean algorithm proposed by Buades et al. can effectively reduce Gaussian noise in core CT images while preserving the detailed features of the core and the spatial correlation between materials. Therefore, this invention uses nonlocal mean filtering to denoise the data. To alleviate GPU memory shortage, all CT images in the dataset were cropped to 256 pixels × 256 pixels. A binary label was created for every 20 frames to prevent overfitting due to sample similarity. Pixels in the shale fracture area were labeled in white, and the rest were labeled in black. Manually labeling shale fracture images is time-consuming and laborious. Data augmentation operations such as flipping, rotating, and affine transformation were performed on the labeled dataset to expand the dataset. Finally, a dataset consisting of 4160 labeled shale fracture CT images was obtained and named MudshaleCrack. The preprocessed dataset was divided into training, validation, and test sets in a ratio of 8:1:1.
[0026] To evaluate the segmentation performance of the network, a confusion matrix is introduced to represent the number of correct and incorrect identifications for each category, as shown in Table 1.
[0027] Table 1 Confusion Matrix
[0028] Where TP represents the number of white pixels correctly predicted as cracks, TN represents the number of black pixels correctly predicted as background, FP represents the number of white pixels that were incorrectly predicted as cracks, and FN represents the number of black pixels that were incorrectly predicted as background.
[0029] Five evaluation metrics were selected to quantify the consistency between the segmentation results and the crack regions in the actual annotations: precision (P), recall (R), pixel accuracy (PA), F1 score (F1_score), and intersection-over-union (IoU). The definitions of each metric are as follows: ; ; ; ; ; Precision (P) represents the proportion of pixels predicted as cracks in the predicted results, measuring whether there are false positives. Recall (R) represents the proportion of pixels predicted as cracks in the true values, measuring whether there are false negatives. The F1 score is a criterion that integrates precision and recall; a higher F1 score indicates fewer missed crack pixels and fewer false positives. PA represents the percentage of correctly predicted pixels out of all pixels. IoU represents the quotient of the intersection and union of manually labeled cracks and network-predicted cracks.
[0030] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0031] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A residual and multi-scale feature based U-Net image segmentation network, characterized in that, include: The process consists of three parts: feature extraction, feature fusion, and output prediction. The feature extraction part uses 5 levels of residual convolution to process the feature map, with the number of residual modules being 3, 4, 6, 3, and 3, respectively, to learn the semantic features of crack images at different scales; The feature fusion part includes a multi-scale feature fusion module and four upsampling residual convolution operations. The feature fusion module is used to capture multi-scale feature information. The deconvolution upsampling reconstructs the compressed spatial information layer by layer, and the skip connections supplement the information lost during the encoding process to enable feature aggregation, so that the resolution of the predicted image output by the network is consistent with that of the original input image. The output prediction part uses 1×1 convolution to adjust the prediction map into a single channel, and then uses the Sigmoid function to make the pixel prediction probability value return to 0-1. Pixels with a probability value greater than a set threshold are identified as crack pixels, and those with a probability value less than the set threshold are background pixels. Finally, an accurate binary segmentation map of mudstone and shale cracks is output. 2.The U-Net image segmentation network based on residual and multi-scale features of claim 1, characterized in that, It also includes using the Focal Loss function to improve the balance of the shale fracture segmentation network based on the segmentation results and the true values. The Focal Loss function is defined as follows: ； ； In the formula: is an adjustable coefficient, is a predicted probability size, is a label value. 3.The U-Net image segmentation network based on residual and multi-scale features of claim 1, characterized in that, It also includes using a confusion matrix to evaluate segmentation accuracy.
4. The U-Net image segmentation network based on residuals and multi-scale features according to claim 1, characterized in that, The multi-scale fusion module includes 1×1 convolution, multi-scale convolution, and global mean pooling. Multi-scale convolution uses depth-separable convolution with kernel sizes of 3, 5, and 7 to perform convolution operations. Global mean pooling encodes image feature information from a global perspective. The feature maps output from each part of global mean pooling are concatenated by channels, and finally, 1×1 convolution is used to adjust the number of channels to obtain a feature map that aggregates multi-scale semantic information.
5. The U-Net image segmentation network based on residuals and multi-scale features according to claim 1, characterized in that, It also includes nonlocal mean filtering for noise reduction of the region to be segmented.
6. The U-Net image segmentation network based on residuals and multi-scale features according to claim 1, characterized in that, Acquire grayscale CT images of mudstone and shale, and use a mask to crop out the edge information of the grayscale CT images as the region to be segmented.