Particle segmentation method and system for sandstone microsection image

Abstract

The invention discloses a particle segmentation method and system for a sandstone microsection image, and the method comprises the steps: configuring a first mark and a second mark which are used for representing the particle boundary of sandstone debris and the category of the sandstone debris for a historical sandstone microsection image, and forming a segmentation label; constructing a preset machine learning model, and training the preset machine learning model by using the historical image carrying the segmentation label to obtain a particle segmentation model, the preset machine learning model generates a first feature map combination through a residual network with multiple attention mechanisms, generates a second feature map combination through a feature pyramid network adopting cavity convolution, determines an area where particles are located, and identifies particle boundaries and types to which the particles belong, so as to realize particle segmentation; and inputting the to-be-segmented sandstone microsection image into the particle segmentation model to obtain a particle segmentation result. According to the method, irregular or long and narrow particle boundaries are effectively identified, and the particle segmentation accuracy of the sandstone microsection image is improved.

Description

Technical Field

[0001] The invention belongs to the field of petroleum geological exploration, and in particular relates to a particle segmentation method and system for sandstone microscopic thin-section images. Background Art

[0002] Sandstone thin section image segmentation is the process of processing sandstone thin section microscopic images according to grayscale, color, texture shape, contour and other information, and forming independent sub-regions at the sandstone particles. Traditional sandstone thin section image segmentation usually uses the following methods:

[0003] (1) Threshold-based image segmentation. In this method, first, the grayscale histogram of the sandstone image is calculated and classified by setting a threshold. Then, based on the assumption that the threshold interval of the grayscale value of the same particle pixel in the image should be the same, the sandstone particles are segmented;

[0004] (2) Edge-based image segmentation. In this method, boundary information is generated based on the RGB value of the image, and then the boundary information and region information are combined to identify and segment sandstone particles;

[0005] (3) Region-based image segmentation. In this method, first, by extracting the color, texture, contrast and other features of the sandstone thin section image, the genetic algorithm is used to identify particles, merge regions and segment boundaries. Alternatively, pre-segmentation is performed through superpixel segmentation, followed by feature extraction and region merging. Subsequently, similar target particles are merged to extract independent and complete particles, thereby achieving the segmentation of sandstone particles.

[0006] Sandstone particles are of various types, sizes, and unclear particle boundaries. Image segmentation based on traditional methods has problems such as under-segmentation and over-segmentation. In addition, the segmentation boundaries are not obvious and segmentation is missed. It cannot meet the needs of sandstone thin section microscopic image segmentation. Therefore, how to improve the segmentation accuracy of sandstone thin section images has become a technical problem that needs to be solved urgently.

[0007] With the rapid development of deep learning and computer vision technology, compared with traditional image segmentation algorithms, image segmentation based on deep learning can better meet the current needs for image segmentation. In the process of realizing the present invention, the inventors found that the current conventional image segmentation based on the Mask R-CNN network can achieve relatively good segmentation effects in the sandstone microscopic image segmentation task. However, in practical applications, due to the characteristics of sandstone images of different sizes, irregular shapes, and dense distribution, it is still very difficult to accurately complete this fine-grained dense segmentation task. Summary of the invention

[0008] In order to solve the above problems, an embodiment of the present invention provides a particle segmentation method for sandstone microscopic thin section images, including: configuring a first mark and a second mark for representing the particle boundaries and categories of sandstone debris for historical sandstone microscopic thin section images, respectively, to form segmentation labels; constructing a preset machine learning model, and using historical images carrying the segmentation labels to train the preset machine learning model to obtain a particle segmentation model, wherein the preset machine learning model generates a first feature map combination through a residual network with a multiple attention mechanism, and generates a second feature map combination through a feature pyramid network using a dilated convolution, determines the area where the particles are located and identifies the particle boundaries and categories therein, so as to achieve particle segmentation; inputs the sandstone microscopic thin section image to be segmented into the particle segmentation model to obtain a particle segmentation result.

[0009] Preferably, the multiple attention mechanisms include but are not limited to: channel attention mechanism, coordinate attention mechanism and spatial attention mechanism, wherein the multiple attention mechanisms first compress the downsampled original feature map of the sandstone microscopic thin section image based on average pooling, then reduce the dimension and activate the compressed feature map, and after completing the dimension reduction and activation processing, restore the dimension of the current compressed feature map to obtain weight data representing the importance of the scale of the attention object; using the weight data corresponding to different attention mechanisms, the downsampled original feature map containing weighted features with different scales of the attention object is obtained, which is recorded as the first feature map combination.

[0010] Preferably, in the process of determining the region where the particles are located and identifying the particle boundaries and categories therein, the process includes: using the feature pyramid network to obtain an upsampled original feature map of the sandstone microscopic thin section image, recorded as the second feature map combination, and fusing the first feature map combination with the second feature map combination, so as to obtain the region where the particles are located and the corresponding particle boundaries and categories according to the information fusion result.

[0011] Preferably, the process of fusing information of the first feature map combination with the second feature map combination includes: using the bottom-up process of the feature pyramid to perform unidirectional fusing of the first feature map combination, and then using the top-down process, combined with the first feature map combination that has completed unidirectional fusion, to perform unidirectional fusing of the second feature map combination, and then using the lateral connection process between the corresponding levels of the bottom-up process and the top-down process to complete the information fusion.

[0012] Preferably, the process of determining the region where the particles are located and identifying the boundaries and categories of the particles therein further includes:

[0013] The preset machine learning model also generates a candidate detection frame about the completed information fusion feature map through a candidate region extraction network, and maps the candidate detection frame to the corresponding feature map based on a regional feature aggregation algorithm, thereby obtaining the region where the particle is located.

[0014] Preferably, in the process of generating candidate detection frames for completing the information fusion feature map, it includes: configuring multiple groups of anchor points with different aspect ratios for each pixel point on the completed information fusion feature map, and determining the position of the anchor points after regression; performing confidence scoring on each anchor point, and then extracting a specified number of anchor points in descending order of confidence scores, and performing non-maximum suppression on the extracted anchor points, and then extracting a specified number of anchor points from the remaining anchor points that have completed non-maximum suppression in descending order of confidence scores as the candidate detection frame.

[0015] Preferably, in the process of determining the area where the particles are located and identifying the particle boundaries and categories therein, it also includes: the preset machine learning model also obtains the category of the area where the particles are located through a target detection branch, and identifies the particle boundaries within the area where the particles are located through a mask branch, and based on this, obtains the particle segmentation result.

[0016] Preferably, before inputting the sandstone microscopic thin section image to be segmented into the grain segmentation model to obtain the grain segmentation result, the grain segmentation method also includes: dividing the historical image into a training set and a test set, so as to obtain error data of the current grain segmentation model by using the historical images in the test set, so as to perform error correction on it, wherein the error data includes but is not limited to: a first error for determining the region where the grain is located, a second error for identifying the category to which it belongs, and a third error for identifying the grain boundary, wherein the third error is obtained by calculating the similarity between the actual grain boundary features and the actual category of the historical image in the test set input into the grain segmentation model and the output grain segmentation result, and combining the loss function.

[0017] Preferably, before using historical images carrying segmentation labels to train the preset machine learning model to obtain a particle segmentation model, the particle segmentation method also includes: using multiple image processing methods to preprocess the historical images to improve the clarity of the particle boundaries on the image, wherein the multiple image processing methods include but are not limited to: brightness transformation, contrast transformation, cropping, scaling and rotation.

[0018] On the other hand, the present invention also provides a particle segmentation system for sandstone microscopic thin section images, and the particle segmentation system includes the following modules: an image processing module, which is used to configure a first mark and a second mark for representing the grain boundaries and categories of sandstone debris for historical sandstone microscopic thin section images, respectively, to form a segmentation label; a model training module, which is used to construct a preset machine learning model, and use historical images carrying the segmentation labels to train the preset machine learning model to obtain a particle segmentation model, wherein the preset machine learning model generates a first feature map combination through a residual network with a multiple attention mechanism, and generates a second feature map combination through a feature pyramid network using a void convolution, determines the area where the particles are located and identifies the grain boundaries and categories therein, so as to achieve particle segmentation; a segmentation result generation module, which is used to input the sandstone microscopic thin section image to be segmented into the particle segmentation model to obtain a particle segmentation result.

[0019] Compared with the prior art, one or more embodiments of the above scheme may have the following advantages or beneficial effects:

[0020] The present invention proposes a particle segmentation method and system for sandstone microscopic thin section images. The method first uses the Mask R-CNN algorithm framework as the backbone network to construct a residual network with a multi-attention mechanism and a machine learning model of a feature pyramid network using a dilated convolution; then, the machine learning model is trained using historical sandstone microscopic thin section images to obtain a particle segmentation model; finally, the sandstone microscopic thin section image to be segmented is input into the particle segmentation model to obtain a particle segmentation result. Based on the improvement and optimization of the traditional Mask R-CNN network, the present invention enhances the performance of the obtained particle segmentation model in the sandstone microscopic thin section image segmentation task, effectively identifies irregular or narrow grain boundaries, and improves the accuracy of particle segmentation of sandstone microscopic thin section images.

[0021] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0023] Figure 1 This is a step diagram of a particle segmentation method for sandstone microscopic thin section images according to an embodiment of the present application.

[0024] Figure 2This is a diagram of the overall network structure of a preset machine learning model for a particle segmentation method for sandstone microscopic thin section images according to an embodiment of the present application.

[0025] Figure 3 This is a flowchart of the operation of the channel attention mechanism of the particle segmentation method for sandstone microscopic thin section images according to an embodiment of the present application.

[0026] Figure 4 This is a flowchart of the operation of the coordinate attention mechanism and the spatial attention mechanism of the particle segmentation method for sandstone microscopic thin section images according to an embodiment of the present application.

[0027] Figure 5 This is an example diagram of the dilated convolution of the particle segmentation method for sandstone microscopic thin section images according to an embodiment of the present application.

[0028] Figure 6 It is a module block diagram of a particle segmentation system for sandstone microscopic thin section images according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will describe the implementation methods of the present invention in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly. It should be noted that as long as there is no conflict, the various embodiments of the present invention and the various features in the embodiments can be combined with each other, and the technical solutions formed are all within the protection scope of the present invention.

[0030] In addition, the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be performed in an order different from that shown here.

[0031] Sandstone particles are of various types, sizes, and unclear particle boundaries. Image segmentation based on traditional methods has problems such as under-segmentation and over-segmentation. In addition, the segmentation boundaries are not obvious enough and segmentation is missed. It cannot meet the needs of sandstone thin section microscopic image segmentation. Therefore, how to improve the segmentation accuracy of sandstone thin section images has become a technical problem that needs to be solved urgently.

[0032] Therefore, in order to solve the above problems, an embodiment of the present invention proposes a particle segmentation method and system for sandstone microscopic thin section images. The method first uses the Mask R-CNN algorithm framework as the backbone network to construct a residual network with multiple attention mechanisms and a machine learning model of a feature pyramid network using a dilated convolution; then, the machine learning model is trained using historical sandstone microscopic thin section images to obtain a particle segmentation model; finally, the sandstone microscopic thin section image to be segmented is input into the particle segmentation model to obtain a particle segmentation result. Based on the improvement and optimization of the traditional Mask R-CNN network, the present invention enhances the performance of the obtained particle segmentation model in the sandstone microscopic thin section image segmentation task, effectively identifies irregular or narrow grain boundaries, and improves the accuracy of particle segmentation of sandstone microscopic thin section images.

[0033] Embodiment 1

[0034] Figure 1 This is a step diagram of the particle segmentation method for sandstone microscopic thin section images in the embodiment of the present application. Figure 1 To illustrate the various steps of this method.

[0035] like Figure 1 As shown, in step S110, the first mark and the second mark for indicating the grain boundary and the category of the sandstone debris are respectively configured for the historical sandstone microscopic thin section image to form a segmentation label. Specifically, the present embodiment first obtains the historical sandstone microscopic thin section image taken by a professional polarizing microscope, and obtains the historical image combination for training the particle segmentation model through preliminary screening. Then, the grain boundary and the category of the sandstone debris on each image in the historical image combination are determined, so as to mark the first mark and the second mark indicating the grain boundary and the category of the sandstone debris on the corresponding image, and use the combination of the first mark and the second mark contained in each image as the segmentation label, thereby using the historical image carrying the segmentation label as the input information of the preset machine learning model below to obtain a particle segmentation model capable of outputting particle segmentation results.

[0036] In the embodiment of the present application, each particle is surrounded by a polygonal frame, and the polygonal frame is used as the first mark. After completing the mark configuration, the formed segmentation label is saved as a json format file to train the preset machine learning model, wherein the first mark is stored in the form of coordinates.

[0037] In an embodiment of the present application, the images in the historical image combination have typical and comprehensive sandstone category features, and the historical image combination includes single polarized images and orthogonal polarized images. The particle segmentation model obtained by training based on the historical image combination of this embodiment has better generalization. Further, in a specific embodiment of the present application, the particle size of sandstone debris in the sandstone category feature covers a range from very fine particles to gravel, the rock debris types cover three types: igneous rock, sedimentary rock, and metamorphic rock, the sorting covers three types: good, medium, and poor, the roundness covers five types: angular, sub-angular, sub-circular, circular, and extremely circular, and the debris particles cover different contact types such as point contact, line contact, concave-convex contact, and suture contact.

[0038] Before using historical images carrying segmentation labels to train the preset machine learning model to obtain a particle segmentation model, this embodiment also uses a variety of image processing methods to pre-process the historical images to improve the clarity of the particle boundaries on the image. In practical applications, the sandstone sample slices used to obtain historical sandstone microscopic thin-section images show different characteristics under different angles of light. Therefore, in order to highlight the characteristics of the particles and reduce the learning difficulty of the model segmentation, this embodiment uses a variety of image processing methods to pre-process the historical images, so that the particles on the image are more clearly distinguished from the background, and the boundary contours of the particles are clearer, so as to achieve the purpose of improving the clarity of the particle boundaries on the image.

[0039] In a specific embodiment of the present application, a variety of image processing methods include but are not limited to: brightness transformation, contrast transformation, cropping, scaling and rotation. After preprocessing with brightness transformation and contrast transformation, the color difference between the particles and the background in the orthogonal polarized image in the historical image combination will be more obvious, and the outline of the particles in the single polarized image will be more eye-catching. On this basis, after further preprocessing with cropping, scaling and rotation, more obvious particle features (particle boundaries and categories) can be obtained. It can be seen that this embodiment effectively improves the recognition of particles on the image and provides technical support for obtaining accurate particle segmentation results.

[0040] Further, in step S120, a preset machine learning model is constructed, and the preset machine learning model is trained using historical images carrying segmentation labels to obtain a particle segmentation model, wherein the preset machine learning model generates a first feature map combination through a residual network with a multiple attention mechanism, and generates a second feature map combination through a feature pyramid network using a dilated convolution, determines the area where the particles are located and identifies the particle boundaries and categories therein, so as to achieve particle segmentation. Figure 2 This is a diagram of the overall network structure of the preset machine learning model for the particle segmentation method of sandstone microscopic thin section images in the embodiment of the present application. Figure 2The overall network structure of the preset machine learning model of this embodiment is described in detail.

[0041] After forming the segmentation labels, this embodiment uses the historical images carrying the segmentation labels as the input information of the preset machine learning model, substitutes them into the preset machine learning model, and starts training the preset machine learning model. After the training is completed, a particle segmentation model is obtained. The particle segmentation model can segment the particles on the sandstone microscopic thin section image to be segmented and generate the particle segmentation results of the sandstone microscopic thin section image to be segmented. Figure 2 As shown, this embodiment builds a preset machine learning model based on the Mask R-CNN algorithm framework, and extracts particle features (particle boundaries and categories) based on the residual network (ResNet) improved by multi-attention fusion and the feature pyramid network (FPN) using dilated convolution, thereby completing particle segmentation. Among them, the preset machine learning model of this embodiment generates a first feature map combination through a residual network with a multiple attention mechanism to extract the semantic information of the image particles, effectively suppressing the interference of the network gradient disappearance or explosion phenomenon on the particle segmentation result. At the same time, the preset machine learning model of this embodiment generates a second feature map combination by using a feature pyramid network with dilated convolution, and then based on the information fusion between the first feature map combination and the second feature map combination, effectively expands the receptive field of the obtained particle segmentation model, and realizes the fusion of multi-scale information. It can be seen that this embodiment ensures the accuracy of determining the area where the particles are located and identifying the particle boundaries and categories therein, and on this basis, accurate particle segmentation results can be obtained.

[0042] The multiple attention mechanisms include but are not limited to: channel attention mechanism, coordinate attention mechanism and spatial attention mechanism. First, the multiple attention mechanisms compress the downsampled original feature map of the sandstone microscopic thin section image based on average pooling, and then reduce the dimension and activate the compressed feature map. After completing the dimension reduction and activation processing, the current compressed feature map is restored to obtain the weight data representing the importance of the scale of the attention object; then, the weight data corresponding to different attention mechanisms are used to obtain the downsampled original feature map containing weighted features with different scales of the attention object, which is recorded as the first feature map combination.

[0043] In a specific embodiment of the present application, in order to improve the particle segmentation model's ability to extract particle features, while retaining the attention mechanism in the traditional Mask R-CNN algorithm, four additional branches for implementing the attention mechanism are introduced, thereby forming a residual network with multiple attention mechanisms that integrates the multi-attention improved CA+SP+SE-ResNet network, to improve the particle segmentation model's ability to focus on key information and the training effect. Among them, the four branches for implementing the attention mechanism are the channel attention mechanism (SE), the coordinate attention mechanism (CA) including the horizontal coordinate attention mechanism and the vertical coordinate attention mechanism, and the spatial attention mechanism (SP).

[0044] Figure 3 This is a flowchart of the operation of the channel attention mechanism of the particle segmentation method for sandstone microscopic thin section images according to an embodiment of the present application. Figure 4 This is a flowchart of the operation of the coordinate attention mechanism and the spatial attention mechanism of the particle segmentation method for sandstone microscopic thin section images in the embodiment of the present application. Figure 3 and Figure 4 The operation process of the multiple attention mechanism of this embodiment is described in detail.

[0045] First, refer to Figure 3, the preset machine learning model of this embodiment learns the importance of different particle features extracted by the convolution layer for particle segmentation during the training process through the channel attention mechanism (SE), combined with the feature recalibration of each channel, thereby enhancing the response of the particle segmentation model to useful features, suppressing unimportant features, and finally improving the feature representation ability of the model. In the channel attention mechanism (SE), the downsampled original feature map of the sandstone microscopic thin section image with a size of c×h×w (obtained after convolution and pooling) is first compressed by global average pooling, so that the two-dimensional h×w feature channel is compressed into a real number, and each real number is connected to form a one-dimensional vector with a length equal to c. Then, the compressed feature map is reduced in dimension and activated by the first fully connected layer, so that the corresponding feature dimension is reduced to 1 / r (for example: 1 / 4) before the dimension reduction, so as to reduce the complexity of the operation, and the ReLU activation function is used to activate the compressed feature map after the dimension reduction, so that the convergence speed of the particle segmentation model in the application process can be maintained in a stable state, and feature learning is easier, thereby effectively ensuring the segmentation efficiency and the accuracy of the obtained particle segmentation results. After completing the dimensionality reduction and activation processing, the second fully connected layer is used to restore the dimension, thereby realizing the restoration of the dimension of the current compressed feature map. Finally, the Sigmoid activation function is used to obtain the channel weights and the normalized feature weights, and finally the weight data representing the importance of the scale of the attention object as the channel scale is obtained. At this time, in a manner similar to the traditional machine learning model with channel attention mechanism, setting the scaling Scale operation can weight the weights by channel to the corresponding original feature map. Among them, in the compression step, the average value of the compressed feature channel is calculated using the following expression:

[0046]

[0047] Among them, z c represents the average value of the cth feature channel after compression, u c It represents the value of the c-th feature channel at the corresponding position, i and j represent the horizontal and vertical coordinates respectively, H represents the height of the feature map, and W represents the width of the feature map.

[0048] In a specific embodiment of the present application, a learnable parameter W0 is used to generate the weights of each feature channel, and then a linear layer W1 with a dimension of C / r×C is multiplied by z c , the scaling operation is performed through the fully connected layer to reduce the amount of calculation. At this time, W1z cThe dimension of the feature map obtained is still 1×1×C / r. After further passing through the ReLU layer, it is multiplied by the linear layer W2 with a dimension of C×C / r to obtain a feature map with a dimension of 1×1×C. Finally, after passing through the Sigmoid activation function, the channel weights s of the C feature maps are generated. The entire process of generating the feature map weights s is expressed by the following expression:

[0049]

[0050] Among them, s represents the channel weight, W0 represents the learnable parameter, σ and δ represent the Sigmoid activation function and ReLU activation function respectively, and W1 and W2 represent different linear layers respectively.

[0051] Next, refer to Figure 4 , the coordinate attention mechanism (CA) of this embodiment includes attention mechanisms in both horizontal and vertical directions, so two branches are required for implementation. The way in which the coordinate attention mechanism (CA) obtains weight data is similar to that of the channel attention mechanism (SE). However, since the coordinates are two-dimensional, after average pooling, they need to be cascaded to form an intermediate feature map of size C / r×(H+W) before activation, where r represents the downsampling ratio, and the effect is consistent with the dimensionality reduction operation in the channel attention mechanism (SE). After completing the dimensionality reduction and activation processing, the intermediate feature map is re-divided into two directions, and then the horizontal coordinate weight and the vertical coordinate weight are obtained respectively through the Sigmoid activation function, so as to achieve the purpose of obtaining weight data that characterizes the importance of the scale of the attention object as the coordinate scale. At this point, in a manner similar to that of a traditional machine learning model with a coordinate attention mechanism, setting the scaling Scale operation can weight the weights to the corresponding original feature map according to the coordinates.

[0052] Continue to refer to Figure 4 , the spatial attention mechanism (SP) of this embodiment compresses and generates a vector of 1×H×W, and obtains weight data representing the importance of the scale of the attention object as the spatial position scale in a manner similar to the channel attention mechanism (SE) and the coordinate attention mechanism (CA). At this time, in a manner similar to the traditional machine learning model with spatial attention mechanism, setting the scale operation can weight the weights according to the spatial distribution to the corresponding original feature map.

[0053] Finally, using the weight data corresponding to different attention mechanisms, the above four branches are weighted and summed to obtain a set of feature map combinations with different sizes, that is, the original feature map containing weighted features with different attention object (channel, coordinate and spatial distribution) scales, and this set of feature maps is used as the first feature map combination. In this way, this embodiment assigns weights to the channels of the feature map combination based on the channel attention mechanism, improves the perception ability of the particle segmentation model of particles of different sizes, helps the particle segmentation model to extract particle boundaries based on the coordinate attention mechanism, and helps the particle segmentation model to track more particles based on the spatial attention mechanism.

[0054] In the process of determining the region where the particles are located and identifying the boundaries and categories of the particles therein, the feature pyramid network is used to obtain the upsampled original feature map of the sandstone microscopic thin section image, which is recorded as the second feature map combination, and the first feature map combination is fused with the second feature map combination, so as to obtain the region where the particles are located and the corresponding particle boundaries and categories according to the information fusion result. In a specific embodiment of the present application, the following is adopted: Figure 5 The dilated convolution method shown in the figure is used to expand the receptive field of the particle segmentation model and realize the fusion of multi-scale information ( Figure 5 This is an example diagram of the dilated convolution of the particle segmentation method for sandstone microscopic thin-section images in the embodiment of the present application). First, using the feature pyramid network, based on the dilated convolution and upsampling processing, the upsampled original feature map of the sandstone microscopic thin-section image is obtained. The obtained upsampled original feature map is a set of feature map combinations with different sizes that have not been weighted summed in accordance with the attention mechanism. This set of feature maps is used as the second feature map combination, and the sizes of the feature maps in the second feature map combination are arranged in order from small to large. Based on the generation method of the first feature map, the first feature map combination contains different semantic information, and the sizes of the feature maps in the first feature map combination are arranged in order from large to small. The dilated convolution can also learn multi-scale global semantic information. At this time, by filling r-1 zeros in the adjacent positions of the convolution kernel before expansion, the first feature map combination and the second feature map combination are fused, so that the area where the particles are located and the corresponding particle boundaries and categories can be obtained according to the information fusion results, where r represents the expansion rate. That is to say, the atrous convolution of this embodiment increases the size of the receptive field by filling (r-1) elements of 0 in the ordinary convolution. In fact, only non-zero elements will play a role in the calculation.

[0055] In practical applications, dilated convolution can take into account both resolution and receptive field, making the model more sensitive to large sandstone particles. The spatial resolution of the image is maintained during dilated convolution, making it easier to accurately locate sandstone particle targets. Given different r, different expansion rates will be obtained, changing the size of the convolution kernel and the receptive field of the model, so that multi-scale global semantic information can be obtained. At the same time, dilated convolution can integrate information from a wider range around the particles to improve the resolution of small particles. The size of the convolution kernel after dilated convolution is expressed by the following expression:

[0056] K=k+(k-1)(r-1) (3)

[0057] Among them, K represents the size of the convolution kernel after expansion, k represents the size of the convolution kernel before expansion, and r represents the expansion rate.

[0058] In the process of information fusion of the first feature map combination and the second feature map combination, the first feature map combination is unidirectionally fused using the bottom-up process of the feature pyramid, and then the second feature map combination is unidirectionally fused using the top-down process, combined with the first feature map combination that completes the unidirectional fusion, and then the information fusion is completed using the lateral connection process between the corresponding levels of the bottom-up process and the top-down process. The feature pyramid is mainly composed of three parts: bottom-up, top-down, and lateral connection. In an embodiment of the present application, the bottom-up part appropriately reduces the feature map with the maximum resolution in the first feature map combination, and completes the unidirectional fusion of the first feature map combination. Then, the top-down part appropriately enlarges the feature map with the minimum resolution in the second feature map combination, and combines the first feature map combination that completes the fusion to complete the unidirectional fusion of the second feature map combination. That is to say, the fusion of each feature map in the second feature map combination is achieved by using the feature map in the first feature map combination that is at the same level as the feature map and completes the fusion, and the feature map in the second feature map combination that is at the upper level of the feature map and completes the fusion. After completing all unidirectional fusions, the fused feature maps (belonging to the first feature map combination and the second feature map combination, respectively) with the same size between the corresponding levels of the bottom-up process and the top-down process are further fused to complete the information fusion. Alternatively, in another embodiment of the present application, the bottom-up part appropriately reduces the feature map with the largest size in the first feature map combination, and completes the unidirectional fusion of the first feature map combination, and the top-down part appropriately enlarges the feature map with the smallest size in the second feature map combination, and combines the second feature map combination that completes the fusion to complete the unidirectional fusion of the second feature map combination. After completing all unidirectional fusions, the fused feature maps (belonging to the first feature map combination and the second feature map combination, respectively) with the same resolution between the corresponding levels of the bottom-up process and the top-down process are further fused using the lateral connection process to complete the information fusion.

[0059] Next, in the process of determining the region where the particle is located and identifying the particle boundaries and categories therein, the preset machine learning model also generates a candidate detection frame about the completed information fusion feature map through the candidate region extraction network, and maps the candidate detection frame to the corresponding feature map based on the regional feature aggregation algorithm, thereby obtaining the region where the particle is located. Specifically, the preset machine learning model also generates a candidate detection frame for framing the particle region on the completed information fusion feature map through the candidate region extraction network (RPN network). After generating the candidate detection frame, based on the regional feature aggregation algorithm (for example: ROIAlign), the generated candidate detection frame is mapped to the corresponding completed information fusion feature map, so that the size and position of the candidate detection frame match the feature map, thereby obtaining the region where the particle is located.

[0060] In the process of generating candidate detection frames for the completed information fusion feature map, first, multiple groups of anchor points with different aspect ratios are configured for each pixel point on the completed information fusion feature map, and the position of the anchor points after regression is determined; then, each anchor point is scored with a confidence score, and then a specified number of anchor points are extracted in descending order of confidence scores, and non-maximum suppression is performed on the extracted anchor points, and then a specified number of anchor points are extracted from the remaining anchor points that have completed non-maximum suppression in descending order of confidence scores as candidate detection frames.

[0061] In a specific embodiment of the present application, multiple groups of anchor points with different aspect ratios are configured for each pixel point on the information fusion feature map, and the position of the anchor point after regression is determined based on the generated candidate detection frame. Next, the confidence score is performed on each configured anchor point, and the anchor points are sorted from high to low according to the confidence score, and the first K anchor points are taken. Next, non-maximum suppression is performed on these K anchor points, and the suppressed results are re-sorted from high to low according to the confidence score. Finally, the first n boxes after sorting are taken as candidate detection frames. For example: to complete 3 anchor points with different aspect ratios for each pixel point on the information fusion feature map, the aspect ratios are {0.5, 1, 1.5}, and n is taken as 1000.

[0062] In the process of determining the region where the particle is located and identifying the particle boundary and category therein, the preset machine learning model also obtains the category of the particle in the region where the particle is located through the target detection branch, and identifies the particle boundary in the region where the particle is located through the mask branch, based on which, the particle segmentation result is obtained. In an embodiment of the present application, the particle segmentation result is obtained by performing target detection and mask generation on a feature map with a candidate detection frame. The preset machine learning model has two outputs, a target detection branch and a mask branch. Among them, the target detection branch is responsible for obtaining the category and border regression of the region where the particle is located through the fully connected layer, and the mask branch outputs the mask through convolution and upsampling, identifies the particle boundary in the region where the particle is located, and obtains the instance segmentation result (ie: particle segmentation result).

[0063] Furthermore, before inputting the sandstone microscopic thin section image to be segmented into the particle segmentation model to obtain the particle segmentation result, the particle segmentation method also includes: dividing the historical images into a training set and a test set, so as to obtain error data of the current particle segmentation model using the historical images in the test set, thereby performing error correction on it, wherein the error data includes but is not limited to: a first error for determining the area where the particle is located, a second error for identifying the category to which it belongs, and a third error for identifying the boundary of the particle.

[0064] Specifically, in order to improve the accuracy of the segmentation results of the particle segmentation model, this embodiment divides the historical images into a training set and a test set, and uses the historical images in the training set to obtain the particle segmentation model. After obtaining the particle segmentation model, the historical images in the test set are used to obtain the error data of the current particle segmentation model, so as to perform error correction. The error data includes but is not limited to: determining the first error L of the region where the particle is located; box (the error generated in the process of generating the candidate detection frame), and the second error L for identifying the category to which it belongs cls (the error generated in the process of generating the category to which the particle belongs), and the third error L for identifying the particle boundary mask (Error generated by the particle segmentation process) In the embodiment of the present application, the third error is obtained by calculating the similarity between the actual particle boundary features and the actual category of the historical images in the test set of the input particle segmentation model and the output particle segmentation result, and combining the loss function.

[0065] Next, the correction process of the third error of this embodiment is described in detail.

[0066] In a specific embodiment of the present application, by calculating the similarity between the actual particle boundary features and the actual category of the historical image input to the particle segmentation model and the output particle segmentation result, the predicted result of the instance segmentation is compared with the actual result for graphical similarity, and the average cross entropy loss representing the third error is calculated using the following expression:

[0067]

[0068] Among them, L mask represents the average cross entropy loss, s represents the number of instances used to calculate the average cross entropy loss, and y i represents the true value of the binary mask, Represents the predicted value of the binary mask.

[0069] After that, the parameters are updated in the direction of the gradient descent of the loss function, and the aforementioned average cross entropy loss calculation steps are repeated until the loss function converges, thereby completing the correction of the third error.

[0070] In addition, this embodiment also uses the method of using the Sigmod function to calculate the average relative entropy of all pixels in the target area where the particles are located to obtain the third error. In practical applications, the sandstone particles in the sandstone microscopic thin section image are often complex in shape and very dense. In order to generate a very accurate sandstone particle boundary, it is actually more necessary to compare the real result and the predicted result to ensure that their boundaries and contours fit perfectly. Therefore, in another specific embodiment of the present application, based on the loss function, the similarity coefficient Dice that characterizes the graphic similarity between the real segmentation result and the predicted segmentation result of the sandstone particles is used to obtain the similarity error Diceloss function. Further, combined with the non-convexity of the Diceloss function, problems such as gradient explosion may occur, resulting in training anomalies such as more unstable training and difficulty in training convergence. The Log-cosh function is used to smooth the Diceloss loss, thereby obtaining the third error. Accordingly, this embodiment replaces the traditional loss function used in model training segmentation with the Log-cosh Diceloss function, which effectively improves the accuracy of segmentation mask fitting. Among them, let the real result of the segmentation be |G|, the predicted result be |P|, and the third error is calculated using the following expression:

[0071]

[0072] Among them, L Ic-dice represents the third error, cosh represents the hyperbolic function, G represents the true segmentation result, and P represents the predicted segmentation result.

[0073] In the embodiment of the present application, |G∩P| is the dot product of the binary image of the predicted segmentation result and the true segmentation result, and |G| and |P| are directly added element by element.

[0074] Furthermore, this embodiment uses the following expression to calculate the similarity coefficient:

[0075]

[0076] In practical applications, since sandstone microscopic thin section image segmentation is a binary classification problem, in order to avoid the denominator of the fraction being 0 and reduce overfitting, this embodiment also adds smoothing (for example, adding 1) to the numerator and denominator of the similarity coefficient at the same time, which effectively avoids the overfitting problem and improves the reliability of the calculation results.

[0077] It can be seen that the third error obtained by using the Diceloss function based on the similarity error and smoothing the Diceloss loss through the Log-cosh function is used to correct the particle segmentation model, which improves the accuracy of the original uncorrected particle segmentation model in fitting the segmentation mask. The corrected particle segmentation model can simulate the boundaries of sandstone microscopic thin section images more finely, and effectively improve the segmentation effect of irregular sandstone particles, such as narrow and long particles and sandstone particles of different sizes, and the contour will be finer.

[0078] Further, in step S130, the sandstone microscopic thin section image to be segmented is input into the particle segmentation model to obtain a particle segmentation result. Specifically, in a manner similar to that of the historical image, a segmentation label of the sandstone microscopic thin section image to be segmented is formed, and then the image to be segmented carrying the segmentation label is used as input information of the particle segmentation model, and the corresponding particle segmentation result is used as output information of the particle segmentation model, thereby obtaining the particle segmentation result of the current sandstone microscopic thin section image to be segmented.

[0079] In a specific embodiment of the present application, the images from the SMISD segmentation dataset and the COCO dataset are segmented using a granular segmentation model based on a traditional segmentation network, and the performance data of the traditional segmentation network shown in Table 1 are obtained. It can be seen from the calculation that the segmentation accuracy of the improved Mask R-CNN network in this embodiment is significantly improved compared with the traditional Mask R-CNN, HTC, PointRend and other algorithms. Among them, AP, AP S 、AP M 、AP Lis the average accuracy of target segmentation objects of different sizes. At the same time, based on the added channel attention mechanism, this embodiment strengthens the modeling ability of the particle segmentation model for image color information (specifically, the number of separable particles detected in the orthogonal polarized light microscopy image has increased significantly), based on the added coordinate attention mechanism and spatial attention mechanism, and the adopted dilated convolution, the segmentation effect of the particle segmentation model on small particles is improved (specifically, the number of small particles detected has increased significantly). Furthermore, this embodiment corrects the error of the particle segmentation model based on the Log-cosh Diceloss function, which enhances the particle segmentation model's boundary fitting effect on particles, especially for irregular or narrow particle boundaries.

[0080] Table 1. Traditional segmentation network performance data

[0081]

[0082] Embodiment 2

[0083] Based on the particle segmentation method for sandstone microscopic thin section images described in the first embodiment, this embodiment further provides a particle segmentation system for sandstone microscopic thin section images. Figure 6 It is a module block diagram of a particle segmentation system for sandstone microscopic thin section images according to an embodiment of the present application.

[0084] like Figure 6 As shown, the particle segmentation system for sandstone microscopic thin section images in the embodiment of the present invention includes: an image processing module 61, a model training module 62 and a segmentation result generation module 63. Specifically, the image processing module 61 is implemented according to the method described in step S110 above, and is configured to respectively configure the first mark and the second mark for representing the grain boundary and the category of the sandstone debris for the historical sandstone microscopic thin section image to form a segmentation label; the model training module 62 is implemented according to the method described in step S120 above, and is configured to build a preset machine learning model, and use the historical image carrying the segmentation label to train the preset machine learning model to obtain a particle segmentation model, wherein the preset machine learning model generates a first feature map combination through a residual network with a multiple attention mechanism, and generates a second feature map combination through a feature pyramid network using a dilated convolution, determines the area where the particle is located and identifies the grain boundary and the category therein, so as to achieve particle segmentation; the segmentation result generation module 63 is implemented according to the method described in step S130 above, and is configured to input the sandstone microscopic thin section image to be segmented into the particle segmentation model to obtain a particle segmentation result.

[0085] The present invention proposes a particle segmentation method and system for sandstone microscopic thin section images. The method first uses the Mask R-CNN algorithm framework as the backbone network to construct a residual network with multiple attention mechanisms and a feature pyramid network using a dilated convolution. Then, the machine learning model is trained using historical sandstone microscopic thin section images to obtain a particle segmentation model. Finally, the sandstone microscopic thin section image to be segmented is input into the particle segmentation model to obtain a particle segmentation result. The present invention is based on the improvement and optimization of the traditional Mask R-CNN network, by integrating multiple attention mechanisms into the feature extraction network, and adding dilated convolution to the FPN network to obtain rich features, and further adopting the Log-cosh Diceloss function suitable for the task target to obtain the loss of the particle segmentation model, so that the particle segmentation model is more consistent with the sandstone thin section image segmentation scene, and the performance of the obtained particle segmentation model in the sandstone microscopic thin section image segmentation task is enhanced, and irregular or narrow grain boundaries are effectively identified, and the accuracy of particle segmentation of sandstone microscopic thin section images is improved.

[0086] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any person familiar with the technology within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0087] It should be understood that the embodiments disclosed in the present invention are not limited to the specific structures, processing steps or materials disclosed herein, but should be extended to equivalent substitutions of these features understood by ordinary technicians in the relevant field. It should also be understood that the terms used herein are only used for the purpose of describing specific embodiments and are not meant to be limiting.

[0088] The "one embodiment" or "embodiment" mentioned in the specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "embodiment" appearing in various places throughout the specification do not necessarily refer to the same embodiment.

[0089] Although the embodiments disclosed in the present invention are as above, the contents described are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.

Claims

1. A particle segmentation method for sandstone microscopic thin section images, characterized in that: include: The historical sandstone microscopic thin section image is respectively configured with a first mark and a second mark for indicating the grain boundary and the category to which the sandstone fragments belong, so as to form a segmentation label; Constructing a preset machine learning model, and using the historical images carrying the segmentation labels to train the preset machine learning model to obtain a particle segmentation model, wherein the preset machine learning model generates a first feature map combination through a residual network with a multiple attention mechanism, and generates a second feature map combination through a feature pyramid network using a dilated convolution, determines the area where the particles are located and identifies the particle boundaries and categories therein, so as to achieve particle segmentation; The microscopic thin-section image of the sandstone to be segmented is input into the particle segmentation model to obtain a particle segmentation result.

2. The particle segmentation method according to claim 1, characterized in that: The multiple attention mechanisms include but are not limited to: channel attention mechanism, coordinate attention mechanism and spatial attention mechanism, wherein: The multiple attention mechanisms first compress the downsampled original feature map of the sandstone microscopic thin section image based on average pooling, then perform dimensionality reduction and activation processing on the compressed feature map, and after completing the dimensionality reduction and activation processing, restore the dimension of the current compressed feature map to obtain weight data representing the importance of the scale of the attention object; By using the weight data corresponding to different attention mechanisms, a downsampled original feature map containing weighted features with different attention object scales is obtained, which is recorded as the first feature map combination.

3. The particle segmentation method according to claim 1 or 2, characterized in that: The process of determining the area where the particles are located and identifying the boundaries and categories of the particles therein includes: The feature pyramid network is used to obtain an upsampled original feature map of the sandstone microscopic thin section image, which is recorded as the second feature map combination. The first feature map combination and the second feature map combination are fused, so as to obtain the area where the particles are located and the corresponding particle boundaries and categories according to the information fusion result.

4. The particle segmentation method according to claim 3, characterized in that: In the process of fusing the first feature map combination with the second feature map combination, the method includes: The first feature map combination is unidirectionally fused using the bottom-up process of the feature pyramid, and then the second feature map combination is unidirectionally fused using the top-down process in combination with the first feature map combination that has completed the unidirectional fusion, and then the information fusion is completed using the lateral connection process between the corresponding levels of the bottom-up process and the top-down process.

5. The particle segmentation method according to claim 3 or 4, characterized in that: The process of determining the region where the particles are located and identifying the boundaries and categories of the particles therein also includes: The preset machine learning model also generates a candidate detection frame about the completed information fusion feature map through a candidate region extraction network, and maps the candidate detection frame to the corresponding feature map based on a regional feature aggregation algorithm, thereby obtaining the region where the particle is located.

6. The particle segmentation method according to claim 5, characterized in that: In the process of generating candidate detection boxes for completing the information fusion feature map, including: To complete the information fusion feature map, multiple groups of anchor points with different aspect ratios are configured for each pixel point, and the position of the anchor points after regression is determined; A confidence score is given to each anchor point, and then a specified number of anchor points are extracted in descending order of confidence scores, and non-maximum suppression is performed on the extracted anchor points. Then, a specified number of anchor points are extracted from the remaining anchor points that have completed non-maximum suppression in descending order of confidence scores as the candidate detection boxes.

7. The particle segmentation method according to any one of claims 1 to 6, characterized in that: The process of determining the region where the particles are located and identifying the boundaries and categories of the particles therein also includes: The preset machine learning model also obtains the category of the region where the particle is located through the target detection branch, and identifies the particle boundary within the region where the particle is located through the mask branch, based on which, the particle segmentation result is obtained.

8. The particle segmentation method according to any one of claims 1 to 7, characterized in that: Before inputting the sandstone microscopic thin section image to be segmented into the particle segmentation model to obtain the particle segmentation result, the particle segmentation method further includes: The historical images are divided into a training set and a test set, so as to obtain error data of the current particle segmentation model using the historical images in the test set, so as to perform error correction on the model, wherein the error data includes but is not limited to: a first error for determining the region where the particle is located, a second error for identifying the category to which it belongs, and a third error for identifying the boundary of the particle, wherein: The third error is obtained by calculating the similarity between the actual particle boundary features and the actual category of the historical image in the test set of the particle segmentation model input into the particle segmentation model and the output particle segmentation result, and combining the loss function.

9. The particle segmentation method according to any one of claims 1 to 8, characterized in that: Before using the historical images carrying the segmentation labels to train the preset machine learning model to obtain the particle segmentation model, the particle segmentation method further includes: The historical images are preprocessed by using a variety of image processing methods to improve the clarity of the particle boundaries on the images, wherein the various image processing methods include but are not limited to: brightness transformation, contrast transformation, cropping, scaling and rotation.

10. A particle segmentation system for sandstone microscopic thin section images, characterized in that: The particle segmentation system includes the following modules: An image processing module, which is used to configure a first mark and a second mark for representing the grain boundary and the category of the sandstone fragments for the historical sandstone microscopic thin section image, respectively, to form a segmentation label; A model training module, which is used to build a preset machine learning model, and use the historical images carrying the segmentation label to train the preset machine learning model to obtain a particle segmentation model, wherein the preset machine learning model generates a first feature map combination through a residual network with a multiple attention mechanism, and generates a second feature map combination through a feature pyramid network using a dilated convolution, determines the area where the particles are located and identifies the particle boundaries and categories therein, so as to achieve particle segmentation; The segmentation result generating module is used to input the sandstone microscopic thin section image to be segmented into the particle segmentation model to obtain the particle segmentation result.

Images