Strawberry-shaped pyrite segmentation quantification method and device based on scanning electron microscope image

Abstract

The invention discloses a strawberry-shaped pyrite segmentation quantification method and device based on a scanning electron microscope image, and the method comprises the steps: inputting an obtained to-be-recognized scanning electron microscope image into a Mask DINO-based strawberry-shaped pyrite segmentation model, and obtaining a strawberry-shaped pyrite segmentation result; inputting the to-be-recognized scanning electron microscope image into a scale recognition model to obtain a scale detection frame and a character detection frame; measuring based on the measuring scale detection frame to obtain the length of the measuring scale; performing character recognition based on the character detection box to obtain an actual distance represented by a corresponding scale length; based on the scale length and the actual distance represented by the scale length, calculating to obtain a single-pixel scale; and based on the strawberry-shaped pyrite segmentation result and the single-pixel scale, obtaining a parameter quantification result of each strawberry-shaped pyrite in the scanning electron microscope image to be identified.

Description

Technical Field

[0001] The present invention relates to the technical field of reservoir analysis, and in particular to a method and device for segmenting and quantifying strawberry pyrite based on scanning electron microscope images. Background Art

[0002] In shale oil and gas exploration, accurate identification of the environment is crucial for understanding the formation mechanism of organic-rich shales. Characteristics such as the crystal form and particle size of strawberry pyrite can accurately assess the redox environment of the water during the deposition of organic-rich shales. This helps geologists better assess the formation mechanism of organic-rich shales and is of great significance for reservoir analysis. Summary of the Invention

[0003] In order to obtain better strawberry pyrite parameter quantification results, an embodiment of the present invention provides a strawberry pyrite segmentation and quantification method and device based on scanning electron microscope images.

[0004] In a first aspect, an embodiment of the present invention provides a method for segmenting and quantifying strawberry pyrite based on a scanning electron microscope image, the method comprising:

[0005] The acquired scanning electron microscope image to be identified is input into the strawberry pyrite segmentation model based on Mask DINO to obtain the strawberry pyrite segmentation result;

[0006] Inputting the scanning electron microscope image to be identified into a scale recognition model to obtain a scale detection frame and a text detection frame;

[0007] Perform measurement based on the scale detection frame to obtain the scale length;

[0008] Performing text recognition based on the text detection frame to obtain the actual distance represented by the scale length;

[0009] Calculating a single-pixel scale based on the scale length and the actual distance represented by the scale length;

[0010] Based on the strawberry pyrite segmentation result and the single-pixel scale, a parameter quantification result of each strawberry pyrite in the scanning electron microscope image to be identified is obtained.

[0011] In one or some optional implementations of the embodiments of the present application, the strawberry pyrite segmentation model based on Mask DINO is obtained by the following method:

[0012] Obtain the strawberry pyrite dataset and the initial Mask DINO model;

[0013] The initial Mask DINO model is trained based on the strawberry pyrite dataset to obtain the Mask DINO-based strawberry pyrite segmentation model.

[0014] In one or some optional implementations of the embodiments of the present application, the initial Mask DINO model includes a backbone network, a Transformer encoder, a unified and enhanced query selection strategy, and a Transformer decoder;

[0015] The initial Mask DINO model is trained based on the strawberry pyrite dataset to obtain the Mask DINO-based strawberry pyrite segmentation model, comprising:

[0016] Input each sample image in the strawberry pyrite dataset into the backbone network to obtain corresponding feature maps of different scales;

[0017] Obtaining a position embedding feature according to the sample image;

[0018] Inputting the position embedding feature and the multiple multi-scale feature maps into the Transformer encoder to obtain enhanced features;

[0019] Applying the unified and enhanced query selection strategy to the enhanced features to obtain anchor box queries and content queries;

[0020] Inputting the enhanced features, the anchor box query, and the content query into the Transformer decoder to obtain a query embedding feature;

[0021] Fusing the largest feature map among the multiple multi-scale feature maps with the upsampled enhanced features to obtain a pixel embedding mapping feature;

[0022] Performing a dot product calculation on the query embedding feature and the pixel embedding mapping feature to obtain a predicted segmented image corresponding to the sample image;

[0023] The initial Mask DINO model is optimized according to the predicted segmentation image to obtain the Mask DINO-based strawberry pyrite segmentation model.

[0024] In one or some optional embodiments of the present application, the strawberry pyrite dataset is obtained by:

[0025] Acquire multiple scanning electron microscope images of strawberry pyrite;

[0026] Each strawberry-shaped pyrite in each strawberry-shaped pyrite scanning electron microscope image is outlined and annotated to obtain a strawberry-shaped pyrite data set.

[0027] In one or some optional implementations of the embodiment of the present application, the scale recognition model is obtained by:

[0028] Obtaining a scale extraction and recognition dataset and the pre-trained scale recognition model;

[0029] The pre-trained scale recognition model is trained based on the scale extraction and recognition data set to obtain the scale recognition model.

[0030] In one or some optional implementations of the embodiment of the present application, the scale extraction and recognition dataset is obtained by:

[0031] Acquire a plurality of strawberry-shaped pyrite scanning electron microscope images; the plurality of strawberry-shaped pyrite scanning electron microscope images include a plurality of different styles of scale bars;

[0032] A preset number of strawberry pyrite scanning electron microscope images containing each scale pattern are taken to obtain an unlabeled scale extraction and recognition dataset;

[0033] The scale detection frame and the text detection frame of each strawberry-shaped pyrite scanning electron microscope image in the unlabeled scale extraction and recognition data set are respectively marked to obtain a scale extraction and recognition data set.

[0034] In one or some optional implementations of the embodiments of the present application, the parameter quantification results of the strawberry-shaped pyrite include at least one of the major and minor axes, area, contour perimeter, equivalent circle diameter, and roundness;

[0035] The step of obtaining a parameter quantification result of each strawberry-shaped pyrite in the scanning electron microscope image to be identified based on the strawberry-shaped pyrite segmentation result and the single-pixel scale includes:

[0036] Extracting the outline boundary of the strawberry-shaped pyrite based on the strawberry-shaped pyrite segmentation result;

[0037] Determining a minimum circumscribed rectangle of the strawberry-shaped pyrite outline boundary;

[0038] Obtaining the major and minor axes of the strawberry-shaped pyrite according to the minimum circumscribed rectangle of the outline boundary of the strawberry-shaped pyrite and the single-pixel scale;

[0039] Obtaining the area and / or perimeter of the strawberry-shaped pyrite based on the outline boundary of the strawberry-shaped pyrite and the single-pixel scale;

[0040] Calculating the equivalent circle diameter of the strawberry-shaped pyrite according to the area of ​​the strawberry-shaped pyrite;

[0041] The roundness of the strawberry-shaped pyrite is calculated according to the area and the perimeter of the outline of the strawberry-shaped pyrite.

[0042] In a second aspect, an embodiment of the present invention provides a device for segmenting and quantifying strawberry pyrite based on a scanning electron microscope image, the device comprising:

[0043] A model segmentation module is used to input the acquired scanning electron microscope image to be identified into the strawberry pyrite segmentation model based on Mask DINO to obtain the strawberry pyrite segmentation result;

[0044] A detection frame recognition module is used to input the scanning electron microscope image to be recognized into a scale recognition model to obtain a scale detection frame and a text detection frame;

[0045] A length measurement module, configured to measure based on the scale detection frame to obtain the scale length;

[0046] a text recognition module, configured to perform text recognition based on the text detection frame to obtain the actual distance represented by the scale length;

[0047] A first calculation module is configured to calculate a single-pixel scale based on the scale length and the actual distance represented by the scale length;

[0048] A parameter quantification module is used to obtain a parameter quantification result of each strawberry pyrite in the scanning electron microscope image to be identified based on the strawberry pyrite segmentation result and the single-pixel scale.

[0049] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for segmenting and quantifying strawberry pyrite based on scanning electron microscope images.

[0050] In a fourth aspect, an embodiment of the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the strawberry pyrite segmentation and quantification method based on scanning electron microscope images as described above is implemented.

[0051] In a fifth aspect, an embodiment of the present invention provides a computer program product comprising instructions, which, when executed on a computer device, enables the computer device to execute the above-mentioned method for segmenting and quantifying strawberry pyrite based on scanning electron microscope images.

[0052] The beneficial effects of the above technical solutions provided by the embodiments of the present invention include at least:

[0053] The strawberry pyrite segmentation and quantification method based on scanning electron microscope images provided by the embodiment of the present invention, by using the deep learning model Mask DINO to build a segmentation model, realizes the intelligent segmentation of strawberry pyrite on the strawberry pyrite scanning electron microscope image, and utilizes the flexible and scalable characteristics of the strawberry pyrite segmentation model based on Mask DINO to improve the detection and segmentation effect of extremely small targets such as strawberry pyrite, optimize the performance and accuracy of segmentation, and realize intelligent recognition and segmentation of strawberry pyrite. At the same time, the detection frame in the strawberry pyrite scanning electron microscope image is obtained by the scale recognition model, and the strawberry pyrite segmentation result is combined with the intelligent analysis to realize the automatic extraction of important parameters such as major and minor axes, area, and roundness, which can greatly improve the accuracy and immediacy of the quantification of strawberry pyrite parameters, and lay the foundation for subsequent statistics and a deeper understanding and quantification of the structure and characteristics of pyrite.

[0054] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0055] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0057] Figure 1 A schematic diagram of the steps of a method for segmenting and quantifying strawberry pyrite based on scanning electron microscope images provided in an embodiment of the present invention;

[0058] Figure 2 Schematic diagram of the strawberry pyrite segmentation model framework based on Mask DINO provided in an embodiment of the present invention;

[0059] Figure 3 A schematic diagram of the scale recognition model provided by an embodiment of the present invention;

[0060] Figure 4 Schematic diagram of various scales on a scanning electron microscope image provided by an embodiment of the present invention;

[0061] Figure 5 A schematic diagram of the stages of scale recognition provided by an embodiment of the present invention;

[0062] Figure 6A schematic diagram of the segmentation results of strawberry-shaped pyrite provided in an embodiment of the present invention;

[0063] Figure 7 A schematic diagram of parameter quantification results of strawberry pyrite provided in an embodiment of the present invention;

[0064] Figure 8 A schematic diagram of parameter quantification results of a strawberry-shaped pyrite provided by an embodiment of the present invention;

[0065] Figure 9 A distribution diagram of the diameter and area ratio of strawberry-shaped pyrite provided in an embodiment of the present invention;

[0066] Figure 10 This is a diagram showing the diameter distribution of strawberry-shaped pyrite provided in the examples of this application;

[0067] Figure 11 Schematic diagram of the structure of the strawberry pyrite segmentation and quantification device based on scanning electron microscope images provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0069] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0070] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0071] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0072] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0073] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0074] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0075] In order to illustrate the technical solution of the present application, specific embodiments are provided below.

[0076] The inventors have found that in the prior art, the industry currently uses manual identification and cross-crossing method to analyze strawberry pyrite, which is inefficient and highly dependent on experience. Traditional image processing methods are mainly based on pixel values ​​for processing, and there are problems such as inability to process in batches, prone to over-extraction or under-extraction, frequent human-computer interaction, and inability to combine expert experience. Traditional analysis methods cannot meet the needs of accurate resource search and efficiency. In recent years, deep learning technology has made breakthrough progress in image processing such as segmentation, but for extremely small targets such as strawberry pyrite at the micro-nano level, it is often impossible to perform fine and accurate segmentation and parameter quantification analysis. Therefore, there is an urgent need to carry out intelligent identification and intelligent analysis of strawberry pyrite. Based on this, the inventors have made further research and development to make the present invention, providing a method and device for segmenting and quantifying strawberry pyrite based on scanning electron microscope images.

[0077] Example 1

[0078] The embodiment of the present invention provides a method for segmenting and quantifying strawberry pyrite based on scanning electron microscope images, referring to Figure 1 As shown, the method includes:

[0079] S101: Inputting the acquired scanning electron microscope image to be identified into the strawberry pyrite segmentation model based on Mask DINO to obtain a strawberry pyrite segmentation result.

[0080] S102: Inputting the scanning electron microscope image to be recognized into a scale recognition model to obtain a scale detection frame and a text detection frame.

[0081] S103: Measure based on the scale detection frame to obtain the scale length.

[0082] S104: Perform text recognition based on the text detection frame to obtain the actual distance represented by the scale length.

[0083] S105: Calculate and obtain a single-pixel scale based on the scale length and the actual distance represented by the scale length.

[0084] S106: Based on the strawberry pyrite segmentation result and the single pixel scale, a parameter quantification result of each strawberry pyrite in the scanning electron microscope image to be identified is obtained.

[0085] In the above step S101 , the acquired scanning electron microscope image to be identified is input into the strawberry pyrite segmentation model based on Mask DINO to obtain the strawberry pyrite segmentation result output by the strawberry pyrite segmentation model based on Mask DINO.

[0086] In the embodiment of the present application, the strawberry pyrite segmentation model based on Mask DINO is obtained by the following method:

[0087] First, we created a strawberry pyrite dataset. Based on multiple strawberry pyrite SEM images, we annotated each individual strawberry pyrite in each image, generating the corresponding annotated image for each strawberry pyrite SEM image. We then created a training set, validation set, and test set in an 8:1:1 ratio.

[0088] Then, the initial Mask DINO model is constructed. The framework diagram of the initial Mask DINO model is as follows Figure 2 As shown, it includes the backbone network, Transformer encoder (corresponding to Figure 2 Encoder Layer in ), unified and enhanced query selection strategy (corresponding to Figure 2 Unifield & Enhanced QS) and Transformer decoder (corresponding to Figure 2 in the DecoderLayer).

[0089] The step of training an initial Mask DINO model based on the strawberry pyrite dataset to obtain a Mask DINO-based strawberry pyrite segmentation model includes at least the following steps S1011-S1019:

[0090] S1011: Input each sample image in the strawberry pyrite dataset into the backbone network to obtain corresponding feature maps of multiple different scales.

[0091] The backbone network is a hierarchical structure that is used to reduce the resolution of the input sample image layer by layer to obtain feature maps of different scales. Figure 2 For example, the size of the input sample image is H×W. As the backbone network deepens, the resolution is gradually reduced to 1 / 4, 1 / 8, 1 / 16, 1 / 32, and 1 / 64 of the sample image, and five feature maps of different scales are obtained. Among them, the four feature maps of different scales with resolutions of 1 / 8, 1 / 16, 1 / 32, and 1 / 64 of the sample image are taken as the multiple feature maps of different scales corresponding to the sample image.

[0092] The backbone network may use a large computer vision network such as Swin-L (Swin Transformer Large). Those skilled in the art may set the size of the feature maps of different scales obtained, which is not specifically limited in the embodiments of this application.

[0093] S1012: Obtain position embedding features based on the sample image, the position embedding features correspond to Figure 2 PositionalEmbeddings in.

[0094] Those skilled in the art can implement the method for obtaining position embedded features based on the detailed description of the prior art, and the specific implementation method is not specifically limited in the embodiments of this application.

[0095] S1013: Input the position embedding features and multiple multi-scale feature maps into the Transformer encoder to obtain enhanced features.

[0096] The position embedding features and multiple multi-scale feature maps are flattened to obtain multiple tokens and then input into the Transformer encoder. The Transformer encoder captures the key information in each token to form enhanced features.

[0097] Among them, token in the image field of deep learning refers to the discrete representation unit generated after image processing, which is an abstract representation of the local area of ​​the image.

[0098] S1014: Using a unified and enhanced query selection strategy for the enhanced features to obtain anchor box queries and content queries.

[0099] Unified and Enhanced Query Selection will initialize anchor box query and content query based on enhanced features (corresponding to Figure 2 Init Contents and InitAnchors in , get the anchor box query and content query.

[0100] The classification score of each token in the enhanced feature is obtained, and the token with the highest classification score in the enhanced feature is selected as the content query.

[0101] S1015: Input the enhanced features, anchor box query and content query into the Transformer decoder to obtain the query embedding features.

[0102] The enhanced features (corresponding to Figure 2 Key & Value in the query, anchor frame query and content query are input into the Transformer decoder, combined with the hybrid matching algorithm to obtain the query embedding feature (corresponding to Figure 2 Query embedding in ), detection boxes (Boxes) and categories (Classes).

[0103] In addition, the input of the Transformer decoder also includes the labeled image corresponding to the noisy sample image (corresponding to Figure 2 GT+Noise in , used for denoising task training (corresponding to Figure 2 The initial MaskDINO model is trained to reconstruct the anchor box position and content query in the presence of noise, so that the strawberry pyrite segmentation model based on MaskDINO can learn to ignore the noise and improve the robustness.

[0104] S1016: Fusing the largest feature map among the multiple multi-scale feature maps with the upsampled enhanced features to obtain pixel embedding mapping features.

[0105] Unflatten the enhanced features to obtain multiple first hidden features of different scales, corresponding to Figure 2 1 / 8, 1 / 16, 1 / 32 and 1 / 64 not marked in the figure.

[0106] By upsampling (corresponding to Figure 2 The 2×upsample operation in

[15] expands the first hidden feature with the largest scale by two times along each dimension, restores the first hidden feature to 1 / 4 the size of the sample image, and obtains the second hidden feature.

[0107] The second hidden feature is fused with the largest feature map in multiple multi-scale feature maps to obtain the pixel embedding map feature (corresponding to Figure 2 Pixel embeadding map in .

[0108] S1017: Calculate the dot product of the query embedding feature and the pixel embedding mapping feature to obtain the predicted segmentation image corresponding to the sample image (corresponding to Figure 2 in the Masks).

[0109] S1018: Optimize the initial Mask DINO model using a gradient descent algorithm based on the predicted segmented image and the labeled image corresponding to the sample image.

[0110] S1019: Repeat the iterative training process from S1011 to S1018 until the preset conditions are met, stop training, and obtain a strawberry pyrite segmentation model based on Mask DINO.

[0111] The preset conditions may be set to reaching a fixed number of iterations, accuracy reaching a threshold, accuracy not changing within a preset number of iterations, etc. No specific limitation is imposed here.

[0112] In the embodiments of the present application, the flexible and scalable Mask DINO-based strawberry pyrite segmentation model can be used as the initial Mask DINO model, either an untrained Mask DINO model or a Mask DINO model pre-trained on a large dataset (such as the COCO dataset). The Mask DINO model pre-trained on a large dataset can better adapt to the edges, textures, and other features of strawberry pyrite, thereby improving segmentation performance and accuracy.

[0113] In the above step S102, the scanning electron microscope image to be identified is input into the scale recognition model to obtain the scale detection frame and the text detection frame. The scale detection frame and the text detection frame obtained by the scale recognition model from the scanning electron microscope image to be identified are as follows: Figure 3 As shown, the red rectangle enclosing "40μm" is the text detection frame, and the red rectangle enclosing the scale bar is the scale bar detection frame.

[0114] In the embodiment of the present application, the process of establishing the scale recognition model includes at least the following steps S1021-S1022:

[0115] S1021: Establish a scale extraction and recognition dataset and a pre-trained scale recognition model.

[0116] Acquire multiple strawberry-shaped pyrite scanning electron microscope images. Due to different acquisition equipment, multiple strawberry-shaped pyrite scanning electron microscope images include a variety of different scale styles, such as Figure 4 As shown, the red box marks the four different scale styles in the scanning electron microscope image of the strawberry pyrite.

[0117] A preset number of strawberry pyrite scanning electron microscope images containing each style and scale are selected to obtain an unlabeled scale extraction and recognition dataset.

[0118] The scale detection frame and text detection frame of each strawberry pyrite scanning electron microscope image in the unlabeled scale extraction and recognition dataset were annotated to obtain the scale extraction and recognition dataset. Based on the scale extraction and recognition dataset, the training set, validation set, and test set were set in an 8:1:1 ratio.

[0119] Those skilled in the art can select a suitable key point recognition model or target detection model as a pre-trained scale recognition model based on the detailed description of the prior art and actual conditions, such as using the HigherHRNet key point recognition model pre-trained using the COCO dataset as a pre-trained scale recognition model, or using the YOLOv5 target detection model pre-trained using the COCO128 dataset as a pre-trained scale recognition model. No specific limitation is made in the embodiments of the present application.

[0120] S1022: Training a pre-trained scale recognition model based on the scale extraction and recognition dataset to obtain a scale recognition model.

[0121] Set the training hyperparameters (such as learning rate, optimizer, number of training rounds, etc.) and loss functions (such as cross entropy loss, mean square error loss, etc.).

[0122] The pre-trained scale recognition model is trained based on the training set in the scale extraction and recognition dataset. In each iteration round, the pre-trained scale recognition model will receive the strawberry pyrite scanning electron microscope image in the scale extraction and recognition dataset to obtain the predicted scale detection box and text detection box.

[0123] The loss value is calculated based on the predicted and labeled scale detection boxes and text detection boxes, and the pre-trained scale recognition model is optimized using the gradient descent algorithm.

[0124] The training process of the pre-trained scale recognition model is repeated until the preset conditions are met, and then training is stopped to obtain the scale recognition model. The preset conditions may be set to, for example, reaching a fixed number of iterations, reaching a threshold accuracy, or maintaining no change in accuracy within a preset number of iterations. These conditions are not specifically limited here.

[0125] In the above step S103 , measurement is performed based on the scale bar detection frame to obtain the scale bar length.

[0126] In the embodiment of the present application, the scale detection frame obtained based on step S102 includes multiple key points, and scale detection frames of different styles can be obtained according to different scale recognition models.

[0127] If the YOLOv5 target detection model is used as the scale recognition model, the scale detection box contains two key points in the upper left and lower right. If the pixel coordinates of the two key points are (x1, y1) and (x2, y2) respectively, then the scale length = x2-x1.

[0128] If the HigherHRNet key point recognition model is used as the scale recognition model, the scale detection box contains four key points, namely the four corners of the rectangular detection box. The coordinates of the four key points are (x1, y1), (x2, y2), (x3, y3), and (x4, y4) clockwise from the upper left corner. The scale length = [(x2+x3)-(x1+x4)] / 2.

[0129] In the above step S104, text recognition is performed based on the text detection frame to obtain the actual distance represented by the scale length.

[0130] In the embodiment of the present application, the text detection frame obtained in step S102 is recognized by OCR (Optical Character Recognition) technology to obtain the actual distance represented by the scale length. The actual distance represented by the scale length includes the scale value and the unit, such as "50 μm".

[0131] Those skilled in the art can implement text recognition based on the detailed description of the prior art, such as using a pre-trained OCR model in an OCR open source toolbox, and no specific limitation is made in the embodiments of the present application.

[0132] In the above step S105 , the single-pixel scale is calculated based on the scale length and the actual distance represented by the scale length.

[0133] In the embodiment of the present application, the single-pixel scale is calculated based on the scale length obtained in step S103 and the scale value and unit of the actual distance represented by the scale length obtained in step S104, using the following formula 1:

[0134]

[0135] The ratio of the scale value to the scale length is the single-pixel scale, which represents the actual length of one pixel. For example, if the scale length represents a distance of 50 μm, the scale value is 50, and the scale length is 207, then the single-pixel scale is 50 / 207 = 0.241545894, meaning the actual length of one pixel is 0.241545894 μm.

[0136] To facilitate those skilled in the art to understand this solution, the specific implementation process of steps S102 to S105 provided in the embodiment of this application is described more clearly and completely below:

[0137] In the embodiment of the present application, the schematic diagram of the above steps S102 to S105 is as follows: Figure 5 As shown, it includes three parts: shale scanning electron microscope image, scale search, and scale image recognition. The shale scanning electron microscope image corresponds to the scanning electron microscope image to be identified above, the scale search corresponds to the scale recognition in step S102 above, and the scale image information recognition corresponds to steps S103-S105 above. The scale detection frame and text detection frame obtained by executing step S102 on the scanning electron microscope image to be identified are the red rectangles in the figure. The larger red rectangle is the scale detection frame, and the smaller red rectangle is the text detection frame. The scale length and scale representative distance obtained by executing steps S103-S105 based on the scale detection frame and text detection frame correspond to the actual distance represented by the scale length above, and the single-pixel scale is "207", "50μm", and "0.241545894", respectively, with the unit being "μm".

[0138] In the above step S106 , based on the strawberry pyrite segmentation result and the single pixel scale, a parameter quantification result of each strawberry pyrite in the scanning electron microscope image to be identified is obtained.

[0139] In the embodiment of the present application, the parameter quantification result of the strawberry pyrite includes at least one of the major and minor axes, area, contour circumference, equivalent circle diameter and roundness.

[0140] When it is necessary to obtain the major and minor axes, area, outline perimeter, equivalent circle diameter, and roundness of each strawberry-shaped pyrite in the scanning electron microscope image to be identified, step S106 at least includes the following steps S1061-S1066:

[0141] S1061: Based on the strawberry pyrite segmentation result, extract the outline boundary of the strawberry pyrite.

[0142] Contour detection is performed on the strawberry pyrite segmentation result obtained in step S101 to extract the contour boundary of each strawberry pyrite in the scanning electron microscope image to be identified.

[0143] The above-mentioned contour detection can be specifically implemented using the findContours() function in OpenCV (Open Source Computer Vision Library).

[0144] S1062: Determine the minimum circumscribed rectangle of the strawberry-shaped pyrite outline boundary.

[0145] The minimum bounding rectangle (MBR) of each strawberry pyrite outline was obtained using the minimum bounding rectangle fitting method.

[0146] The above minimum bounding rectangle fitting method can be implemented using the cv2.minAreaRect() function in OpenCV.

[0147] S1063: Obtain the major and minor axes of the strawberry pyrite according to the minimum circumscribed rectangle of the strawberry pyrite outline and the single-pixel scale.

[0148] The longer side of the minimum circumscribed rectangle of the strawberry-shaped pyrite outline is used as the major axis, and the shorter side is used as the minor axis. The major axis pixel length and minor axis pixel length are calculated based on the coordinates of the four corners of the minimum circumscribed rectangle. For example, the coordinates of the four points of the minimum circumscribed rectangle of the strawberry-shaped pyrite are A (a1, b1), B (a2, b2), C (a3, b3), and D (a4, b4) clockwise from the upper left corner, and the distance between point A and point B is greater than the distance between point A and point D. The major axis pixel length and minor axis pixel length can be calculated by Euclidean distance based on the following formulas 2 and 3:

[0149]

[0150]

[0151] The single-pixel scale is multiplied by the major axis pixel length and the minor axis pixel length respectively to obtain the actual length of the major axis and the actual length of the minor axis of the strawberry pyrite, that is, the major and minor axes of the strawberry pyrite.

[0152] S1064: Obtaining the area and / or perimeter of the strawberry pyrite based on the strawberry pyrite outline boundary and the single-pixel scale.

[0153] Based on the outline of the strawberry pyrite, we obtain a vertex set. Based on this vertex set, we calculate the outline area and obtain the pixel area. We multiply the square of the single-pixel scale by the pixel area to obtain the area of ​​the strawberry pyrite.

[0154] Based on the contour vertex set, the curve length is calculated through numerical integration to obtain the pixel contour perimeter. The single-pixel scale is multiplied by the pixel contour perimeter to obtain the contour perimeter of the strawberry pyrite.

[0155] The above-mentioned contour area calculation method and curve length calculation method can be specifically implemented using the contourArea() function and arcLength() function in OpenCV.

[0156] S1065: Calculate the equivalent circle diameter of the strawberry-shaped pyrite based on the area of ​​the strawberry-shaped pyrite.

[0157] According to the area of ​​the strawberry-shaped pyrite obtained in step S1064, the equivalent circle diameter of the strawberry-shaped pyrite is calculated based on the area calculation formula shown in the following formula 4.

[0158]

[0159] Where s is the area of ​​strawberry pyrite, and d is the equivalent circle diameter of strawberry pyrite.

[0160] S1066: Calculate the roundness of the strawberry-shaped pyrite based on the area and the perimeter of the outline of the strawberry-shaped pyrite.

[0161] According to the area and the perimeter of the strawberry-shaped pyrite obtained in step S1064 , the roundness of the strawberry-shaped pyrite is calculated based on the roundness calculation formula shown in the following formula 5.

[0162]

[0163] Where s is the area of ​​strawberry pyrite, l is the perimeter of the outline of strawberry pyrite, and p is the roundness of strawberry pyrite.

[0164] In a specific embodiment, the segmentation result of the strawberry-shaped pyrite obtained in step S101 is as follows: Figure 6 As shown, there are 4 strawberry-shaped pyrites. Steps S102 to S106 are performed on the strawberry-shaped pyrite segmentation result to obtain parameter quantification results of each strawberry-shaped pyrite.

[0165] The results of visualizing the parameter quantization results are as follows Figure 7 As shown, the parameter quantification results of each strawberry pyrite are presented. Figure 7 The parameter quantification visualization results of the strawberry-shaped pyrite at the top are as follows Figure 8As shown, the turquoise curve is the outline boundary of strawberry pyrite, the green rectangle is the minimum circumscribed rectangle of strawberry pyrite, the dark blue and yellow numbers "32.1" and "31.8" represent the major axis and minor axis respectively, and the unit is μm. The two vertical pink lines represent the line segments corresponding to the major axis and minor axis respectively. The bluish-purple "htk 0.969" in the middle of the strawberry pyrite, "htk" is the abbreviation of pyrite, and "0.969" is the area of ​​strawberry pyrite.

[0166] Table 1 shows examples of parameter quantification data obtained by applying this method to multiple SEM images to be identified.

[0167] Table 1

[0168]

[0169]

[0170] Table 1 includes two SEM images to be identified, numbered "h1-1" and "h1-2". "h1-1" contains 6 strawberry-shaped pyrites, and "h1-2" contains 11 strawberry-shaped pyrites. The unit of the major axis, minor axis, the average of the major and minor axes, and the equivalent circle diameter is "μm", and the unit of the area is "μm 2 ”, roundness is a unitless quantity.

[0171] Table 2 shows the statistical results of the equivalent circle diameters in the parameter quantification data of strawberry pyrite shown in Table 1.

[0172] Table 2

[0173]

[0174] Table 2 is a statistical analysis of the equivalent circle diameter based on the parameter quantification results of the 17 strawberry-shaped pyrites in Table 1. The maximum equivalent circle diameter, minimum equivalent circle diameter, average equivalent circle diameter, median equivalent circle diameter, 1 / 4 digit of equivalent circle diameter, and 3 / 4 digit of equivalent circle diameter of the 17 strawberry-shaped pyrites are statistically obtained.

[0175] Based on the statistical results of Table 1 and Table 2 above, a schematic diagram showing the statistical data of strawberry pyrite is obtained as follows: Figure 9 and Figure 10 As shown, Figure 9 This is the distribution diagram of the diameter area of ​​strawberry-shaped pyrite, which shows the proportion of strawberry-shaped pyrite with different equivalent circle diameters in the total area of ​​all strawberry-shaped pyrites. Figure 10 This is the distribution diagram of the diameter of strawberry pyrite, indicating the number of strawberry pyrites with different equivalent circle diameters.

[0176] In the embodiment of the present application, objective indicator evaluation of the strawberry pyrite segmentation model and scale recognition model based on Mask DINO is included.

[0177] The objective indicator evaluation of target detection performance and entity segmentation performance of the strawberry pyrite segmentation model based on Mask DINO in step S101 is performed respectively. The evaluation results are shown in Table 3.

[0178] Table 3

[0179] AP AP50 AP75 Object Detection 75.066 86.049 96.016 Entity segmentation 74.389 86.027 86.016

[0180] The objective indicator AP (Average Precision) in Table 3 is a commonly used performance evaluation indicator in target detection. AP50 and AP75 represent the average accuracy at 50% and 75% confidence thresholds, respectively.

[0181] The objective indicator evaluation of the target detection performance of the scale recognition model in step S102 is performed, and the evaluation results are shown in Table 4.

[0182] Table 4

[0183] mAP@.5 mAP@.5:.95 Scale detection frame 0.996 0.842 Text detection box 0.996 0.857 Scale length range 0.996 0.826

[0184] The objective metric mAP@.5 in Table 4 represents the average precision at an IoU (Intersection over Union) threshold of 0.5, that is, the average accuracy when the overlap between the object detection results and the ground truth reaches 50%. mAP@.5:.95 represents the average accuracy at all thresholds in the IoU range from 0.5 to 0.95, taking into account the accuracy in a wider range of overlap situations.

[0185] According to the evaluation results shown in Tables 3 and 4 above, it can be seen that the strawberry pyrite segmentation model and scale recognition model based on Mask DINO have excellent performance and high accuracy.

[0186] This invention combines computer vision technologies such as deep learning and image processing to develop an algorithm model suitable for shale image segmentation and its adaptive training strategy, thereby building a Mask DINO segmentation model. Intelligent means are used to quantify and analyze the parameters of strawberry pyrite, automatically extract parameters such as the major axis, minor axis, area, roundness, and equivalent circle diameter of strawberry pyrite, and visualize the distribution of strawberry pyrite diameter area ratios and diameter quantity distributions. Compared with traditional manual evaluation and analysis methods, this greatly improves the accuracy and objectivity of evaluation, with almost negligible time costs. It lays the foundation for subsequent statistics and a deeper understanding and quantification of the structure and characteristics of pyrite, and is an important intelligent solution and technological breakthrough in the field of reservoir analysis.

[0187] Example 2

[0188] Based on the same inventive concept, the embodiment of the present invention further provides a strawberry pyrite segmentation and quantification device based on scanning electron microscope images, referring to Figure 11 As shown, the device includes:

[0189] The model segmentation module 101 is used to input the acquired scanning electron microscope image to be identified into the strawberry pyrite segmentation model based on Mask DINO to obtain the strawberry pyrite segmentation result;

[0190] The detection frame recognition module 102 is used to input the scanning electron microscope image to be recognized into a scale recognition model to obtain a scale detection frame and a text detection frame;

[0191] A length measurement module 103 is configured to measure the scale bar based on the scale bar detection frame to obtain the scale bar length;

[0192] A text recognition module 104 is configured to perform text recognition based on the text detection frame to obtain the actual distance represented by the scale length;

[0193] A first calculation module 105 is configured to calculate a single-pixel scale based on the scale length and the actual distance represented by the scale length;

[0194] The parameter quantification module 106 is configured to obtain a parameter quantification result of each strawberry pyrite in the to-be-identified scanning electron microscope image based on the strawberry pyrite segmentation result and the single-pixel scale.

[0195] Example 3

[0196] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for segmenting and quantifying strawberry pyrite based on scanning electron microscope images as described in the first embodiment above is implemented.

[0197] Example 4

[0198] Based on the same inventive concept, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the strawberry pyrite segmentation and quantification method based on scanning electron microscope images as described in the first embodiment above is implemented.

[0199] Example 5

[0200] Based on the same inventive concept, an embodiment of the present invention further provides a computer program product comprising instructions. When the computer program product is run on a computer device, the computer device executes the strawberry pyrite segmentation and quantification method based on scanning electron microscope images as described in the first embodiment above.

[0201] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0202] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0203] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0205] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for segmenting and quantifying strawberry pyrite based on scanning electron microscope images, characterized in that: include: The acquired scanning electron microscope image to be identified is input into the strawberry pyrite segmentation model based on Mask DINO to obtain the strawberry pyrite segmentation result; Inputting the scanning electron microscope image to be identified into a scale recognition model to obtain a scale detection frame and a text detection frame; Perform measurement based on the scale detection frame to obtain the scale length; Performing text recognition based on the text detection frame to obtain the actual distance represented by the scale length; Calculating a single-pixel scale based on the scale length and the actual distance represented by the scale length; Based on the strawberry pyrite segmentation result and the single-pixel scale, a parameter quantification result of each strawberry pyrite in the scanning electron microscope image to be identified is obtained.

2. The method according to claim 1, wherein The strawberry pyrite segmentation model based on Mask DINO is obtained by the following method: Obtain the strawberry pyrite dataset and the initial Mask DINO model; The initial Mask DINO model is trained based on the strawberry pyrite dataset to obtain the Mask DINO-based strawberry pyrite segmentation model.

3. The method according to claim 2, wherein The initial Mask DINO model includes a backbone network, a Transformer encoder, a unified and enhanced query selection strategy, and a Transformer decoder; The initial Mask DINO model is trained based on the strawberry pyrite dataset to obtain the Mask DINO-based strawberry pyrite segmentation model, comprising: Input each sample image in the strawberry pyrite dataset into the backbone network to obtain corresponding feature maps of different scales; Obtaining a position embedding feature according to the sample image; Inputting the position embedding feature and the multiple multi-scale feature maps into the Transformer encoder to obtain enhanced features; Applying the unified and enhanced query selection strategy to the enhanced features to obtain anchor box queries and content queries; Inputting the enhanced features, the anchor box query, and the content query into the Transformer decoder to obtain a query embedding feature; Fusing the largest feature map among the multiple multi-scale feature maps with the upsampled enhanced features to obtain a pixel embedding mapping feature; Performing a dot product calculation on the query embedding feature and the pixel embedding mapping feature to obtain a predicted segmented image corresponding to the sample image; The initial Mask DINO model is optimized according to the predicted segmentation image to obtain the Mask DINO-based strawberry pyrite segmentation model.

4. The method according to claim 2, wherein The strawberry pyrite dataset was obtained in the following way: Acquire multiple scanning electron microscope images of strawberry pyrite; Each strawberry-shaped pyrite in each strawberry-shaped pyrite scanning electron microscope image is outlined and annotated to obtain a strawberry-shaped pyrite data set.

5. The method according to claim 1, wherein The scale recognition model is obtained by: Obtain scale extraction and recognition dataset and pre-trained scale recognition model; The pre-trained scale recognition model is trained based on the scale extraction and recognition data set to obtain the scale recognition model.

6. The method according to claim 5, wherein The scale extraction and recognition dataset is obtained in the following way: Acquire a plurality of strawberry-shaped pyrite scanning electron microscope images; the plurality of strawberry-shaped pyrite scanning electron microscope images include a plurality of different styles of scale bars; A preset number of strawberry pyrite scanning electron microscope images containing each scale pattern are taken to obtain an unlabeled scale extraction and recognition dataset; The scale detection frame and the text detection frame of each strawberry-shaped pyrite scanning electron microscope image in the unlabeled scale extraction and recognition data set are respectively marked to obtain a scale extraction and recognition data set.

7. The method according to claim 1, wherein The parameter quantification result of the strawberry pyrite includes at least one of the major and minor axes, area, outline perimeter, equivalent circle diameter and roundness; The step of obtaining a parameter quantification result of each strawberry-shaped pyrite in the scanning electron microscope image to be identified based on the strawberry-shaped pyrite segmentation result and the single-pixel scale includes: Extracting the outline boundary of the strawberry-shaped pyrite based on the strawberry-shaped pyrite segmentation result; Determining a minimum circumscribed rectangle of the strawberry-shaped pyrite outline boundary; Obtaining the major and minor axes of the strawberry-shaped pyrite according to the minimum circumscribed rectangle of the outline boundary of the strawberry-shaped pyrite and the single-pixel scale; Obtaining the area and / or perimeter of the strawberry-shaped pyrite based on the outline boundary of the strawberry-shaped pyrite and the single-pixel scale; Calculating the equivalent circle diameter of the strawberry-shaped pyrite according to the area of ​​the strawberry-shaped pyrite; The roundness of the strawberry-shaped pyrite is calculated according to the area and the perimeter of the outline of the strawberry-shaped pyrite.

8. A device for segmenting and quantifying strawberry pyrite based on scanning electron microscope images, characterized in that: include: A model segmentation module is used to input the acquired scanning electron microscope image to be identified into the strawberry pyrite segmentation model based on Mask DINO to obtain the strawberry pyrite segmentation result; A detection frame recognition module is used to input the scanning electron microscope image to be recognized into a scale recognition model to obtain a scale detection frame and a text detection frame; A length measurement module, configured to measure based on the scale detection frame to obtain the scale length; a text recognition module, configured to perform text recognition based on the text detection frame to obtain the actual distance represented by the scale length; A first calculation module is configured to calculate a single-pixel scale based on the scale length and the actual distance represented by the scale length; A parameter quantification module is used to obtain a parameter quantification result of each strawberry pyrite in the scanning electron microscope image to be identified based on the strawberry pyrite segmentation result and the single-pixel scale.

9. A computer-readable storage medium storing instructions, which, when executed on a terminal, causes the terminal to execute the method for segmenting and quantifying strawberry pyrite based on scanning electron microscope images according to any one of claims 1 to 7.

10. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for segmenting and quantifying strawberry pyrite based on scanning electron microscope images according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising instructions, which, when executed on a computer device, enables the computer device to execute the method for segmenting and quantifying strawberry pyrite based on scanning electron microscope images according to any one of claims 1 to 7.

Images