Rock thin section image granularity recognition method based on transunet, electronic device and storage medium
By combining TransUNet with Transformer and U-Net, the problem of long-distance modeling in rock thin section image recognition is solved, achieving high-precision granularity recognition and automated analysis, thus improving recognition efficiency and accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NORTHEAST GASOLINEEUM UNIV
- Filing Date
- 2023-04-23
- Publication Date
- 2026-05-19
AI Technical Summary
Existing convolution-based rock thin section image recognition methods cannot effectively perform long-distance modeling, making it difficult to identify high-precision granularity. Traditional methods require professionals to spend a lot of time on observation and recognition.
A rock thin section image recognition method based on TransUNet is adopted, which combines the advantages of Transformer and U-Net. ResNet50 is used as the encoder for global context modeling and feature fusion. Long-distance dependencies are captured through self-attention mechanism and multi-head self-attention. Indistinct grain boundaries are processed through image erosion and dilation.
It improves the accuracy of grain size identification and analysis efficiency of rock thin section images, can automatically adjust the receptive field to capture features at different scales, adapts to detailed information at multiple scales, and simplifies the complex rock thin section analysis process.
Smart Images
Figure CN116543256B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of basic geological research, and more specifically, to a method for identifying the grain size of rock thin section images, applicable to processing rock thin section images with complex grain sizes. Background Technology
[0002] Thin section analysis is one of the most important tools in geological research, including rock structure and composition. Grain size analysis of thin sections provides a direct visual representation of petrological characteristics, offering technical support for the study of sedimentary systems and environments. However, traditional artificial thin section grain size identification methods have limitations in terms of efficiency, repeatability, and accuracy.
[0003] To address this critical issue, several scholars have employed deep learning networks to study the grain size segmentation of core images, using methods such as UNet and ResUNet.
[0004] For example, Chinese patent document CN114387328A discloses an RQD calculation method based on a deep learning model and borehole core images. The method includes: establishing a borehole core image dataset; extracting image features from the dataset and inputting these features into a UNet deep network for training to obtain an EUNet model; using the EUNet segmentation model to perform semantic segmentation on the borehole core images to obtain core regions and background regions, identifying single-row borehole core images; extracting the contours of all cores in the single-row borehole core images and counting the number of pixels in the core contours to create a pixel waveform image; determining the core type based on the pixel waveform image; determining the number and location of complete cores from the pixel waveform image to define the study area; further determining the core boundaries; and calculating the core length according to the core boundaries to obtain the RQD. This method achieves intelligent quantitative analysis of RQD, improving the efficiency of geological exploration.
[0005] Chinese patent document CN115688590A discloses a method for predicting the permeability of multi-mineral cores based on deep learning, comprising: (a) constructing a three-dimensional digital core and randomly generating pore structures, and obtaining multiple multi-mineral digital core images by image segmentation of the three-dimensional digital core; (b) using multi-physics simulation software to simulate and calculate the permeability corresponding to each multi-mineral digital core image, and constructing a multi-mineral digital core dataset based on each multi-mineral digital core image and its corresponding permeability; (c) constructing an SE-ResNet18 convolutional neural network, and training the SE-ResNet18 convolutional neural network using the multi-mineral digital core dataset to calculate the permeability corresponding to the multi-mineral digital core image; (d) inputting the image of the multi-mineral core to be predicted into the trained SE-ResNet18 convolutional neural network to determine the permeability of the multi-mineral core.
[0006] However, these existing research methods are limited by the inherent locality of convolution. Convolution-based methods cannot effectively carry out long-distance modeling and cannot achieve high-precision granularity recognition of rock thin section images. Summary of the Invention
[0007] This disclosure proposes a rock thin section image grain size recognition method, electronic device and storage medium based on TransUNet, which can overcome the shortcomings of the prior art.
[0008] The present disclosure discloses a rock thin section image grain size recognition method, electronic device, and storage medium based on TransUNet. Basic Scheme 1:
[0009] A method for grain size recognition of rock thin section images based on TransUNet, the first step of which is to acquire rock core thin section images,
[0010] Obtaining microscopic thin-section images of rocks showing rock grains and pores; characterized in that the method further includes:
[0011] The second step is data preprocessing, which includes extracting N (N>150) 256*256 images at different locations from the microscopic rock thin section images and dividing these images into two datasets, namely DataA and DataB.
[0012] Dataset A is used to train the TransUNet image segmentation model; Dataset B is used to verify the model recognition results and output image granularity recognition results; both Dataset A and Dataset B include original images and labeled images.
[0013] The third step is feature extraction, which includes using a convolutional neural network ResNet50 as the encoder of TransUNet. The DataA dataset obtained in the second step is input into the ResNet50 encoder to form the original rock thin section image space. The original rock thin section image space is then converted into a feature map S1, which contains multiple channels. Each channel captures at least one different feature in the input image. The output of the ResNet50 encoder is represented as a three-dimensional tensor of shape (H, W, C), where H is the height of the feature map S1, W is the width of the feature map S1, and C is the number of channels in the feature map S1.
[0014] The fourth step is feature mapping, which involves flattening the feature map S1 in TransUNet, adding positional encoding, and performing linear projection to map the features to the sequence form required by the Transformer, resulting in the feature map S2 in the Transformer sequence form.
[0015] The fifth step is global context modeling, which includes performing global context modeling in the TransUNet decoder to generate enhanced feature maps S3.
[0016] The feature map S2 obtained via the fourth step is processed using a Transformer-based structure to capture the global long-range dependencies of the rock thin section images in the feature map S2; the processing includes processing and segmenting the rock thin section images in the feature map S2 through self-attention mechanism, multi-head self-attention, layer normalization, residual connection and feedforward neural network;
[0017] Step 6, pixel-by-pixel classification; including mapping the enhanced feature map S3 generated in step 5 through inverse linear projection, removal of position encoding, reshaping operation, upsampling and fusion, and convolutional layer operation to obtain pixel-level output feature map S4 back to the original rock thin section image space to obtain a pixel matrix, and classifying each pixel in the pixel matrix pixel-by-pixel to obtain a grain size category probability image S5.
[0018] Step 7: Calculate the loss; use the loss function to calculate the loss between the granular category probability image S5 output from Step 6 and the label images in the DataA dataset; the loss function is shown in formulas (1) and (2):
[0019] Formula (1) Cross-entropy loss:
[0020] CE(p,q)=-Σ[y*log(y_hat)+(1-y)*log(1-y_hat)];where y is the true label (0 or 1), and y_hat is the probability value predicted by the model;
[0021] Formula (2) Dice loss:
[0022] Dice_Loss = 1 - Dice_Coefficient; where Dice_Coefficient = (2 * |predicted region ∩ true region|) / (|predicted region| + |true region|);
[0023] The goal of the loss function is to measure the difference between the model output and the true label;
[0024] Step 8, iterative training; repeat steps 3 to 7 until the loss function stops decreasing, then stop training to obtain the transUnet training model for rock thin section image grain size recognition.
[0025] In the ninth step, the transUnet training model for rock thin section image grain size recognition obtained in the eighth step is used to perform grain size recognition on the images in the DataB dataset, resulting in a grain size characterization image of the rock thin section.
[0026] Based on the basic scheme 1, further optimization yields scheme 2: In the second step, the DataA dataset is augmented, and the data augmentation method includes preferred random image rotation and random flipping to enhance the generalization ability of the training model.
[0027] By improving scheme 2, we can obtain scheme 3:
[0028] In the fifth step, the processing order of self-attention mechanism and multi-head self-attention is before layer normalization and residual connection;
[0029] The self-attention mechanism is used to calculate the association weights between each element in feature map S2 and other elements to capture long-distance dependencies.
[0030] The multi-head self-attention is used to enhance the expressive power of the self-attention mechanism by dividing the self-attention into multiple "heads", each of which independently calculates the attention weight;
[0031] The layer normalization and residual connections are used to maintain gradient stability and accelerate training; the layer normalization is used to reduce the scale of the feature map S2 to prevent gradient vanishing and gradient explosion; the residual connections are used to add the input and output of each layer in the feature map S2 for information flow transmission.
[0032] The feedforward neural network is used to process the feature map S2 after layer normalization and residual connection processing position by position to obtain the enhanced feature map S3; the feedforward neural network adopts a simple multilayer perceptron to further extract features and enhance the representation ability of the model.
[0033] Based on Scheme 3, Scheme 4 is obtained through further optimization: In the sixth step,
[0034] The inverse linear projection involves applying a fully connected layer to map a two-dimensional tensor of shape (N,E) back to a two-dimensional tensor of shape (N,C+D), where E represents the dimension of the decoder output, and C and D represent the number of channels and the dimension of the position encoding of the feature map, respectively.
[0035] The removal of position codes includes removing the position codes from the enhanced feature map S3 after inverse linear projection, resulting in a two-dimensional tensor of shape (N,C);
[0036] The reshaping operation includes reshaping a two-dimensional tensor of shape (N,C) back into a three-dimensional tensor of shape (H,W,C) for upsampling and fusion operations; the reshaping operation preserves the height H and width W of the feature map S1;
[0037] The upsampling and fusion are used to restore the enhanced feature map S3 to the same spatial resolution as the image in the original rock section image space; the upsampling includes methods using bilinear interpolation and transposed convolution; the fusion includes fusing shallow features from the ResNet50 encoder with features from the TransUNet decoder;
[0038] The convolutional layer is used after the upsampling and fusion operations to convert the enhanced feature map S3 into a pixel-level segmentation result, resulting in a pixel-level output feature map S4. The convolutional layer is a 1x1 convolutional layer used to transform the number of channels at each pixel position into a number of channels equal to the number of granularity categories. The pixel-level output feature map S4 can represent the probability distribution of each pixel belonging to different granularity categories.
[0039] An activation function and pixel assignment are applied to convert the pixel-level output feature map S4 into a granular category assignment. The application of the activation function and category assignment includes applying the Softmax activation function to normalize the probabilities. The output of the activation function represents the probability of each pixel belonging to each granular category. The sum of the probabilities of each pixel belonging to each granular category is 1. The category with the highest probability is selected to assign a granular category to each pixel, resulting in a granular category probability image S5.
[0040] Based on Scheme 4, Scheme 5 is further optimized: In the seventh step, the gradient of the loss function with respect to the model parameters is calculated by the backpropagation algorithm, and the model parameters are updated using an optimizer to minimize the loss function. The optimizer includes the Adam optimizer.
[0041] Based on Scheme 5, Scheme 6 is further optimized: the identification method further includes separating the particles using image erosion techniques, addressing the phenomenon of particle adhesion between particles in the rock thin section particle characterization image; the specific path is as follows:
[0042] The image erosion is a morphological operation that reduces the size of an image region. A structuring element circle is convolved with the image. During the convolution process, the structuring element slides along each pixel of the image that represents the rock slab grains. If all pixels within the structuring element perfectly match the pixels of the corresponding region in the image, the value of the center pixel is preserved; otherwise, the center pixel is set to 0.
[0043] Based on Scheme 6, Scheme 7 is further optimized: the identification method further includes filling the voids in the particles of the rock thin section particle characterization image using image dilation; the specific path is as follows:
[0044] The image dilation is a morphological operation that increases the size of an image region. During convolution, the structuring element slides along each pixel of the image that represents the rock slab grains. If any pixel within the structuring element matches a pixel in the corresponding region of the image, the center pixel is set to 1; otherwise, the value of the center pixel is preserved.
[0045] This invention also has two other applications:
[0046] An electronic device includes a processor and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method described in any one of schemes 1 to 7.
[0047] A computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method described in any one of schemes 1 to 7.
[0048] The above-described at least one technical solution adopted in one or more embodiments of this specification can achieve the following beneficial effects:
[0049] The clastic components in rock thin sections exhibit diverse structures, with significant variations in grain size, sorting, and rounding. They also possess characteristics of diverse shapes and sizes, and the grains exhibit various relationships, including point contact, line contact, convex-concave contact, and suture contact, leading to unclear boundaries. The clastic components in thin sections can be composed of multiple rock-forming minerals or various rock fragments, which may have similar or distinct microscopic features. This complicates grain identification and segmentation, and traditional rock thin section analysis methods typically require specialized researchers to spend considerable time on observation, identification, and statistical analysis. The identification method presented in this disclosure, compared to traditional CNNs, combines the advantages of Transformer and U-Net for rock thin section image recognition. By replacing the encoder with ResNet50, it can more effectively capture local and global information in the image, contributing to improved segmentation accuracy in grain size recognition tasks. The recognition method disclosed herein utilizes the self-attention mechanism of a transformer for global context modeling, automatically adjusting the receptive field size to capture features at different scales. Furthermore, it leverages a U-Net structure to achieve multi-level feature fusion, which helps the model better distinguish different structures and organizations in granularity recognition tasks. Simultaneously, this provides rich information for subsequent segmentation tasks. The resulting trained model performs significantly better in handling granularity recognition tasks with unclear boundaries, adapting to detailed information at multiple scales and greatly improving analysis efficiency. This lays a solid foundation for the scientific and accurate characterization of the particle size, sorting, and rounding features of flank debris.
[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.
[0051] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description
[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.
[0053] Figure 1 This is a flowchart of the rock thin section image grain size recognition method of the present invention.
[0054] Figure 2 These are partial rock thin section images used in the grain size identification method described in this invention.
[0055] Figure 3 This is a schematic diagram of the preprocessed images during the data set creation process described in this invention.
[0056] Figure 4 This is a flowchart of the feature extraction process of the TransUnet network model described in this invention.
[0057] Figure 5 This is a flowchart of the feature mapping process for the TransUnet network model described in this invention.
[0058] Figure 6 This is a flowchart illustrating the global context modeling process of the TransUnet network model described in this invention.
[0059] Figure 7 This is a flowchart of the pixel-by-pixel classification process of the TransUnet network model described in this invention.
[0060] Figure 8 This is a schematic diagram illustrating the granularity identification of the TransUnet network model described in this invention. Detailed Implementation
[0061] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.
[0062] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art should understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure. A specific embodiment is given below, intended to further describe the technical solution provided in this disclosure in conjunction with the accompanying drawings and embodiments. It should be noted that the following embodiments are intended to facilitate understanding of the invention and do not constitute any limitation thereof.
[0063] like Figure 1 As shown, a specific embodiment of the rock thin section image grain size recognition method based on TransUNet described in this disclosure is performed according to the following steps:
[0064] The first step is to acquire core thin section images to obtain microscopic rock thin section images that show rock particles and pores. The core is then prepared into thin sections of different sizes according to different needs. The prepared thin sections are then scanned in a wide field of view using instruments such as core scanners and electron microscopes to obtain microscopic rock thin section images of rock particles and pores.
[0065] In practice, the rock core is prepared into thin sections measuring 10mm x 10mm. These sections are then scanned using a Zeiss EVO 15 scanning electron microscope to obtain microscopic images of the rock particles and pores. For example... Figure 2 The image shown is a microscopic image of a densely grained rock section.
[0066] The second step is data preprocessing, which includes extracting N (N>150) 256*256 images at different locations from the microscopic rock thin section images and dividing these images into two datasets, namely DataA and DataB.
[0067] Dataset A is used to train the TransUNet image segmentation model; Dataset B is used to verify the model recognition results and output image granularity recognition results; both Dataset A and Dataset B include original images and labeled images.
[0068] In the second step, a preferred implementation is to augment the DataA dataset. The augmentation method includes, preferably, random image rotation and random flipping, to enhance the generalization ability of the training model.
[0069] In practical implementation, the following example can be used as a reference: Randomly extract 200 256*256 images of rock thin sections at different locations from the microscopic rock thin section images described in the first step. Divide these images into two datasets in a 7:3 ratio: a training set and a test set. To enhance the generalization ability of the training model, random image rotation and random image flipping are used to augment the training set, such as... Figure 3 As shown, a is the original image; b, c, and d represent image rotations of 90 degrees, 180 degrees, and 270 degrees, respectively; d and e represent horizontal and vertical flips of the image, respectively. Through random image rotation and flipping, the training set was finally expanded to 521 rock thin section images.
[0070] The third step is feature extraction. A ResNet50 convolutional neural network is used as the encoder for TransUNet. The DataA dataset obtained in the second step is input into the ResNet50 encoder to form the original rock thin-section image space. This original rock thin-section image space is then converted into a feature map S1. The feature map contains multiple channels, each capturing at least one distinct feature from the input image. The encoder output is represented as a three-dimensional tensor of shape (H, W, C), where H is the height of the feature map, W is the width of the feature map, and C is the number of channels.
[0071] In practical implementation, the PyTorch deep learning framework can be used, replacing the TransUnet encoder with a ResNet50 convolutional neural network. The training set obtained in the second step is placed in the train folder of the specified training path in TransUnet. Then, the TransUnet ResNet50 encoder reads the training set in the train folder of the specified training path, forms the original rock thin section image space through convolution, and then converts the original rock thin section image space into feature map S1 through 5 convolutions of ResNet-50. The process is as follows. Figure 4 As shown, the feature map S1 is obtained as a three-dimensional tensor of (8,8,2048). Generating the feature map S1 through a 5-fold convolution of ResNet-50 helps the model better capture details and useful features in the image and reduces computational complexity.
[0072] The fourth step is feature mapping, which involves flattening the feature map S1 in TransUNet, adding positional encoding, and performing linear projection to map the features to the sequence form required by the Transformer, resulting in the feature map S2 in the Transformer sequence form.
[0073] In the specific implementation, each channel of the feature map S1 obtained in the third step is traversed. For each channel, starting from the top left corner, each pixel is traversed row by row. For each pixel, it is added to the flattened sequence. The flattened sequences of all channels are concatenated sequentially to obtain a one-dimensional sequence of length (8*8*2048). The feature map is reshaped into a sequence of size (64, 2048), where 64 = 8*8 and 2048 is the number of channels. Then, positional encoding is added to the one-dimensional sequence. First, the dimension of the positional encoding is determined; the dimension of the positional encoding should be the same as the number of channels in the feature sequence, i.e., 2048. Then, positional encoding is generated: for each sequence index i (0-63), its corresponding positional encoding vector is calculated. Each element of the encoding vector is calculated using sine and cosine functions, as follows:
[0074] For even-numbered indices j (i.e., 0, 2, 4, ...) in the position-coded vector, calculate:
[0075] PE[i,j]=sin(i / 10000^(2*j / d))
[0076] Where d is the dimension of the position encoding, which is 2048.
[0077] For odd-numbered indices j (i.e., 1, 3, 5, ...) in the positional encoding vector, calculate:
[0078] PE[i,j]=cos(i / 10000^(2*j / d-1))
[0079] Where d is the dimension of the position encoding, which is 2048.
[0080] Adding positional encoding to the one-dimensional sequence: The calculated 64 positional encoding vectors are added to the reshaped one-dimensional sequence. This gives the one-dimensional sequence positional information, allowing the model to distinguish different elements within the sequence. Specifically, the i-th positional encoding vector is added to the i-th element of the feature sequence. This results in a sequence of shape (64, 2048) that contains positional information.
[0081] Next, linear projection is performed. First, the weight matrix and bias vector are defined. The linear projection layer consists of a weight matrix (W) and a bias vector (b). The weight matrix W has a shape of (256, 2048), representing the mapping from the input dimension to the output dimension. The bias vector b has a shape of (2048). Then, a matrix of shape (64, 2048) is input, where 64 is the number of samples in a batch of data, and 2048 is the number of features for each sample. A linear transformation is performed: the input data matrix is multiplied by the weight matrix, and then the bias vector is added. This operation can be represented as:
[0082] output = input * W^T + b
[0083] Here, * denotes matrix multiplication, and W^T is the transpose of the weight matrix W. The output matrix has a shape of (64, 256), which is the feature map S. 2， The process is as follows Figure 5 As shown.
[0084] The fifth step is global context modeling, which includes performing global context modeling in the TransUNet decoder to generate enhanced feature maps S3.
[0085] The feature map S2 obtained via the fourth step is processed using a Transformer-based structure to capture the global long-range dependencies of the rock thin section images in the feature map S2; the processing includes processing and segmenting the rock thin section images in the feature map S2 through self-attention mechanism, multi-head self-attention, residual connection, layer normalization and feedforward neural network.
[0086] In practice, the processing order of self-attention mechanism and multi-head self-attention is before layer normalization and residual connection;
[0087] In this example, the self-attention mechanism is used to calculate the association weights between each element in the feature map S2 and other elements to capture long-distance dependencies. The self-attention mechanism calculates self-attention weights for the input feature map S2. First, the input feature map S2 is used as a query vector, key vector, and value vector. Then, the query vector and key vector are multiplied by a matrix to obtain a similarity matrix of shape (64, 64). Next, each element in the similarity matrix is divided by the square root of the dimension of each vector to scale the similarity. A softmax operation is performed on the scaled similarity matrix to obtain the attention weight matrix. The attention weight matrix is multiplied by the value vector matrix to obtain a weighted sum. Finally, an output matrix of shape (64, 256) is obtained. The feature map S2 is updated using the output matrix.
[0088] The multi-head self-attention mechanism enhances the expressive power of the self-attention mechanism by dividing the self-attention into multiple "heads," each independently calculating attention weights. In this mechanism, the input feature map is projected into 16 subspaces, and self-attention weights are calculated in each subspace. This produces 16 attention matrices and 16 output matrices. The 16 output matrices are concatenated: in each subspace, an output matrix of shape (64, 256 / 16) is obtained. These matrices are then concatenated along the last dimension (256 / 16) to obtain a matrix of shape (64, 256). The concatenated matrix is multiplied by the attention weight matrix: this results in a matrix of shape (64, 256), which is the recombined output feature map S. 21.
[0089] The layer normalization and residual connections are used to maintain gradient stability and accelerate training; the layer normalization is used to reduce the feature map S. 21 The scale is adjusted to prevent gradient vanishing and gradient explosion; the residual connection is used to connect the feature map S. 21 The input and output of each layer are added together for information flow transmission; the residual connection adds the output (64, 256) of the multi-head self-attention to the original input feature map S2 (64, 256), and its calculation formula is as follows:
[0090] S 22 =S2+MultiHeadAttention(S 21 )
[0091] Here, MultiHeadAttention represents the output matrix of the multi-head self-attention mechanism. This formula means adding the output of the multi-head self-attention mechanism to the original input feature map to obtain a richer feature representation.
[0092] Layer normalization normalizes the feature map after residual connections. The formula for layer normalization is:
[0093] S 23 =LayerNorm(S 22 )
[0094] Here, LayerNorm represents the layer normalization operation, and its input is the feature map S. 22 The output is the normalized feature map S. 23 .
[0095] The feedforward neural network is used to process the feature map S after layer normalization and residual connection. 23 Position-by-position processing is performed to obtain the enhanced feature map S3; the feedforward neural network employs a simple multilayer perceptron to further extract features and enhance the model's representational power. For the input feature map S... 23 A fully connected neural network is used to process it position by position, and the calculation formula is as follows:
[0096] MLP(S3) = ReLU(S 23 W1+b1)W2+b2
[0097] In this diagram, W1 and W2 represent the weight matrices of the first and second layers, respectively; b1 and b2 represent the bias vectors of the first and second layers, respectively; and ReLU represents the ReLU activation function. The weight matrices represent the connection strength between neural network layers. They control the degree of correlation between input and output features. The bias vectors are used to adjust the output of the neural network layers, thereby enabling the activation function to perform better.
[0098] Specifically, for the input feature map S3, it is first multiplied by the weight matrix W1 of the first layer, and then the bias vector b1 is added to obtain the output of the first layer. Then, the ReLU activation function is applied to the output of the first layer to obtain a non-linear intermediate output. Finally, the intermediate output is multiplied by the weight matrix W2 of the second layer, and the bias vector b2 is added to obtain the final position-wise processed output. The output enhanced feature map S3 will have the same shape (64, 256) as the input feature map S2, and the process is as follows: Figure 6 As shown, after global context modeling through the TransUNet decoder, the enhanced feature map S3 contains a synthesis of local and global information, which helps the model better understand the input image.
[0099] Step 6, pixel-by-pixel classification; including mapping the enhanced feature map S3 generated in step 5 through inverse linear projection, removal of position encoding, reshaping operation, upsampling and fusion, and convolutional layer operation to obtain pixel-level output feature map S4 back to the original rock thin section image space to obtain a pixel matrix, and classifying each pixel in the pixel matrix pixel-by-pixel to obtain a grain size category probability image S5.
[0100] In this example, the preferred implementation involves mapping the 2D tensor of the enhanced feature map S3 (64, 256) obtained in step 5 back to a 2D tensor of shape (64, 2304) using inverse linear projection. After inverse linear projection, the positional encoding is removed from the enhanced feature map S3, resulting in a 2D tensor of shape (64, 2048), which is then reshaped back to a 3D tensor of shape (8, 8, 2048). Upsampling and fusion restore the enhanced feature map S3 to the same spatial resolution (256*256) as the image in the original rock section image space. The enhanced feature map S3 is converted into pixel-level segmentation results through a 1x1 convolutional layer, resulting in a pixel-level output feature map S4. The Softmax activation function is applied to the output feature map S4, which converts the original scores in the feature map S4 into a probability distribution. The process is as follows: Figure 7 As shown, at each pixel location, the category with the highest probability is selected as the grain size category of that pixel, resulting in a grain size category probability image S5. The grain size category probability image S5 can reflect the grain size category of each region in the rock thin section image.
[0101] Step 7: Calculate the loss; use the loss function to calculate the loss between the granular category probability image S5 output from Step 6 and the label images in the DataA dataset; the loss function is shown in formulas (1) and (2):
[0102] Formula (1) Cross-entropy loss:
[0103] CE_Loss(p,q)=-Σ[y*log(y_hat)+(1-y)*log(1-y_hat)]. Where y is the true label (0 or 1), and y_hat is the probability value predicted by the model.
[0104] Formula (2) Dice loss:
[0105] Dice_Loss = 1 - Dice_Coefficient. Where Dice_Coefficient = (2 * |Predicted Region ∩ True Region|) / (|Predicted Region| + |True Region|)
[0106] The goal of the loss function is to measure the difference between the model output and the true label;
[0107] In this example, the cross-entropy loss and Dice loss are mixed at a ratio of 50%.
[0108] Loss=0.5*Dice_Loss+0.5*CE_Loss
[0109] The loss between the granular class probability image S5 output from step 6 and the label images in the training set is calculated using the Loss algorithm. The gradient of the loss function with respect to the model parameters is calculated using the backpropagation algorithm, and the model parameters are updated using the Adam optimizer to minimize the loss function.
[0110] Step 8, iterative training; repeat steps 3 to 7 until the effect of the loss function decreases or stops decreasing, then stop training to obtain the transUnet training model for rock thin section image grain size recognition.
[0111] In practice, the training hyperparameter `batch` is adjusted to 16, with the actual batch size depending on the hardware. The initial learning rate is set to 0.01, and it decays every 20 epochs with a decay factor of 0.1. The number of epochs is set to 250, depending on the size of the training set. Steps three through seven are iterated to obtain the trained TransUNet segmentation model. Training stops when the loss function's effectiveness decreases or ceases. The test set is then used for TransUNet segmentation model recognition, outputting image granularity recognition results.
[0112] In the ninth step, the transUnet training model for rock thin section image grain size recognition obtained in the eighth step is used to perform grain size recognition on the images in the DataB dataset, resulting in a grain size characterization image of the rock thin section.
[0113] To address the phenomenon of particle adhesion between particles observed in rock thin section grain characterization images, image erosion is employed to separate the particles; the specific path is as follows:
[0114] The image erosion is a morphological operation that reduces the size of an image region. A structuring element circle is convolved with the image. During the convolution process, the structuring element slides along each pixel of the image that represents the rock slab grains. If all pixels within the structuring element perfectly match the pixels of the corresponding region in the image, the value of the center pixel is preserved; otherwise, the center pixel is set to 0.
[0115] To address the issue of voids in grains appearing in rock thin section grain characterization images, image dilation is used to fill these voids; the specific path is as follows:
[0116] The image dilation is a morphological operation that increases the size of an image region. During convolution, the structuring element slides along each pixel of the image that represents the rock slab grains. If any pixel within the structuring element matches a pixel in the corresponding region of the image, the center pixel is set to 1; otherwise, the value of the center pixel is preserved.
[0117] In this example, the test set is input into the transUnet training model for rock thin section image grain size recognition obtained in step eight. The model will classify the pixels of each image in the test set into the same category with the highest probability, thereby achieving grain size recognition and outputting a rock thin section grain characterization image, such as... Figure 8 As shown.
[0118] Figure 8 The three images shown are all thin-section images of rocks from the test set, with the red lines encompassing the identified particles. The human operator judges whether there is contact between the red lines in the output rock thin-section particle characterization image. If there is contact, it indicates particle adhesion, and image erosion is used for post-processing. If irregular red lines also exist within the area encompassed by the red lines, it indicates the presence of voids in the particles, and image dilation is used for post-processing. The final output rock thin-section particle characterization image shows more distinct particles, making it easier to count the number of particles and extract their morphology.
[0119] For the specific embodiments described above, their practicality can be evaluated using the recall and precision rates shown in Table 1 below. Recall represents the accuracy rate, which is the ratio of the number of correctly predicted positive samples to the total number of actual positive samples. A higher recall rate indicates greater accuracy in capturing actual positive samples.
[0120] Recall = TP / (TP + FN), where TP is the number of true negatives and FN is the number of false negatives.
[0121] Precision represents the accuracy of the prediction. Precision is the ratio of the number of correctly predicted positive samples to the total number of samples predicted as positive. A higher precision indicates that the model is more accurate in predicting positive samples.
[0122] Precision = TP / (TP + FP), where TP is the number of true positives and FP is the number of false positives;
[0123] F1 score: The F1 score is the harmonic mean of precision and recall, ranging from 0 to 1. The F1 score combines precision and recall and can reflect the overall performance of the model to some extent.
[0124] F1-score=2*(Precision*Recall) / (Precision+Recall)
[0125] Numbers Recall Precision F1 score 1 0.960 0.889 0.923 2 0.962 0.921 0.941 3 0.951 0.793 0.864 4 0.960 0.842 0.897 5 0.951 0.864 0.905 6 0.953 0.827 0.885
[0126] Table 1
[0127] As the examples above illustrate, the diverse shapes, sizes, and orientations of particles in rock thin sections, coupled with close contact or overlap, result in unclear boundaries, complicating particle identification and segmentation. TransUNet combines the advantages of Transformer and U-Net, replacing its encoder with ResNet50. This allows TransUNet to more effectively capture local and global information in images, improving segmentation accuracy in particle size recognition tasks. TransUNet leverages the self-attention mechanism of Transformer for global context modeling, automatically adjusting the receptive field size to capture features at different scales. Furthermore, it utilizes the U-Net structure to achieve multi-level feature fusion, making the model more effective in handling particle size recognition tasks with unclear boundaries.
[0128] The embodiments of this disclosure have been described above and are exemplary, not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles of the embodiments, their practical application, or improvements to technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A method for grain size recognition of rock thin section images based on TransUNet, the first step of which is to acquire rock core thin section images. Obtaining microscopic thin-section images of rocks showing rock grains and pores; characterized in that, The method further includes: The second step is data preprocessing, which includes extracting N 256*256 images at different locations from the microscopic rock thin section images, where N>150, and dividing these images into two datasets, namely DataA and DataB. Dataset A is used to train the TransUNet image segmentation model; Dataset B is used to verify the model recognition results and output image granularity recognition results; both Dataset A and Dataset B include original images and labeled images. The third step is feature extraction, which includes using a convolutional neural network ResNet50 as the encoder of TransUNet. The DataA dataset obtained in the second step is input into the ResNet50 encoder to form the original rock thin section image space. The original rock thin section image space is then converted into a feature map S1, which contains multiple channels. Each channel captures at least one different feature in the input image. The output of the ResNet50 encoder is represented as a three-dimensional tensor of shape (H, W, C), where H is the height of the feature map S1, W is the width of the feature map S1, and C is the number of channels in the feature map S1. The fourth step is feature mapping, which involves flattening the feature map S1 in TransUNet, adding positional encoding, and performing linear projection to map the features to the sequence form required by the Transformer, resulting in the feature map S2 in the Transformer sequence form. The fifth step is global context modeling, which includes performing global context modeling in the TransUNet decoder to generate enhanced feature maps S3. The feature map S2 obtained via the fourth step is processed using a Transformer-based structure to capture the global long-range dependencies of the rock thin section images in the feature map S2; the processing includes processing and segmenting the rock thin section images in the feature map S2 through self-attention mechanism, multi-head self-attention, layer normalization, residual connection and feedforward neural network; Step 6, pixel-by-pixel classification; including mapping the enhanced feature map S3 generated in step 5 through inverse linear projection, removal of position encoding, reshaping operation, upsampling and fusion, and convolutional layer operation to obtain pixel-level output feature map S4 back to the original rock thin section image space to obtain a pixel matrix, and classifying each pixel in the pixel matrix pixel-by-pixel to obtain a grain size category probability image S5. Step 7: Calculate the loss; use the loss function to calculate the loss between the granular category probability image S5 output from Step 6 and the label images in the DataA dataset; the loss function is shown in formulas (1) and (2): Formula (1) Cross-entropy loss: CE(p,q)=-Σ[y*log(y_hat)+(1-y)*log(1-y_hat)];where y is the true label, which is 0 or 1, and y_hat is the probability value predicted by the model; Formula (2) Dice loss: Dice_Loss = 1 - Dice_Coefficient; where Dice_Coefficient = (2 * |predicted region ∩ true region|) / (|predicted region| + |true region|); The goal of the loss function is to measure the difference between the model output and the true label; Step 8, iterative training; repeat steps 3 to 7 until the loss function stops decreasing, then stop training to obtain the transUnet training model for rock thin section image grain size recognition. In the ninth step, the transUnet training model for rock thin section image grain size recognition obtained in the eighth step is used to perform grain size recognition on the images in the DataB dataset, resulting in a grain size characterization image of the rock thin section.
2. The method for grain size recognition of rock thin section images based on TransUNet according to claim 1, characterized in that: In the second step, the DataA dataset is augmented using methods such as preferred random image rotation and random flipping to enhance the generalization ability of the trained model.
3. The method for grain size recognition of rock thin section images based on TransUNet according to claim 2, characterized in that: In the fifth step, the processing order of self-attention mechanism and multi-head self-attention is before layer normalization and residual connection; The self-attention mechanism is used to calculate the association weights between each element in feature map S2 and other elements to capture long-distance dependencies. The multi-head self-attention is used to enhance the expressive power of the self-attention mechanism by dividing the self-attention into multiple "heads", each of which independently calculates the attention weight; The layer normalization and residual connections are used to maintain gradient stability and accelerate training; the layer normalization is used to reduce the scale of the feature map S2 to prevent gradient vanishing and gradient explosion; the residual connections are used to add the input and output of each layer in the feature map S2 for information flow transmission. The feedforward neural network is used to process the feature map S2 after layer normalization and residual connection processing position by position to obtain the enhanced feature map S3; the feedforward neural network adopts a simple multilayer perceptron to further extract features and enhance the representation ability of the model.
4. The rock thin section image grain size recognition method based on TransUNet according to claim 3, characterized in that: In the sixth step, The inverse linear projection involves applying a fully connected layer to map a two-dimensional tensor of shape (N,E) back to a two-dimensional tensor of shape (N,C+D), where E represents the dimension of the decoder output, and C and D represent the number of channels in feature map S1 and the dimension of the position encoding in feature map S2, respectively. The removal of position codes includes removing the position codes from the enhanced feature map S3 after inverse linear projection, resulting in a two-dimensional tensor of shape (N,C); The reshaping operation includes reshaping a two-dimensional tensor of shape (N,C) back into a three-dimensional tensor of shape (H,W,C) for upsampling and fusion operations; the reshaping operation preserves the height H and width W of the feature map S1; The upsampling and fusion are used to restore the enhanced feature map S3 to the same spatial resolution as the image in the original rock section image space; the upsampling includes methods using bilinear interpolation and transposed convolution; the fusion includes fusing shallow features from the ResNet50 encoder with features from the TransUNet decoder; The convolutional layer is used after the upsampling and fusion operations to convert the enhanced feature map S3 into a pixel-level segmentation result, resulting in a pixel-level output feature map S4. The convolutional layer is a 1x1 convolutional layer used to transform the number of channels at each pixel position into a number of channels equal to the number of granularity categories. The pixel-level output feature map S4 can represent the probability distribution of each pixel belonging to different granularity categories. An activation function and pixel assignment are applied to convert the pixel-level output feature map S4 into a granular category assignment. The application of the activation function and category assignment includes applying the Softmax activation function to normalize the probabilities. The output of the activation function represents the probability of each pixel belonging to each granular category. The sum of the probabilities of each pixel belonging to each granular category is 1. The category with the highest probability is selected to assign a granular category to each pixel, resulting in a granular category probability image S5.
5. The method for grain size recognition of rock thin section images based on TransUNet according to claim 4, characterized in that: In the seventh step, the gradient of the loss function with respect to the model parameters is calculated using the backpropagation algorithm, and the model parameters are updated using an optimizer, including the Adam optimizer, to minimize the loss function.
6. The method for grain size recognition of rock thin section images based on TransUNet according to claim 5, characterized in that: The identification method further includes using image erosion to separate particles from each other, addressing the adhesion between particles observed in rock thin section particle characterization images; the specific path is as follows: The image erosion is a morphological operation that reduces the size of an image region. A structuring element circle is convolved with the image. During the convolution process, the structuring element slides along each pixel of the image that represents the rock slab grains. If all pixels within the structuring element perfectly match the pixels of the corresponding region in the image, the value of the center pixel is preserved; otherwise, the center pixel is set to 0.
7. The method for grain size recognition of rock thin section images based on TransUNet according to claim 6, characterized in that: The identification method further includes filling voids in the particles of rock thin section particle characterization images using image dilation; the specific path is as follows: The image dilation is a morphological operation that increases the size of an image region. During convolution, the structuring element slides along each pixel of the image that represents the rock slab grains. If any pixel within the structuring element matches a pixel in the corresponding region of the image, the center pixel is set to 1; otherwise, the value of the center pixel is preserved.
8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.
9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.