A SEM image particle size analysis method and system based on visual large model

Abstract

The present application discloses a method and system for SEM image particle size analysis based on a visual macro model, relating to the field of image recognition technology. The method comprises: acquiring an SEM image and automatically identifying the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel; collecting an SEM image set containing particle features, annotating the particles in the SEM image set, and constructing an SEM image training set and an SEM image test set based on the annotation results; constructing a visual macro model and fine-tuning the visual macro model using the SEM image training set; using the fine-tuned visual macro model to infer the SEM image test set to obtain particle segmentation results; and calculating the attributes of each segmented particle based on the actual length of each pixel for particle size analysis. This solution improves the efficiency, accuracy, and consistency of particle analysis, and meets the needs for efficient and accurate analysis of microscopic particles.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a SEM image particle size analysis method and system based on a visual macro model. Background Art

[0002] Traditional SEM (Scanning Electron Microscope) image analysis methods rely primarily on manual operation, a process that is often time-consuming and susceptible to subjective judgment. This not only limits analysis efficiency but also affects the accuracy of the results, potentially leading to significant differences in analysis results for the same sample. In the rapidly developing field of scientific research, especially in the field of materials science, the demand for efficient and accurate analysis of microscopic particle characteristics is becoming increasingly urgent.

[0003] With the advancement of science and technology, the application of computer vision and machine learning technologies has provided new solutions for SEM image analysis. These new technologies can identify and quantify particle features in images, improving the speed and consistency of analysis. Through automated processes, researchers can process image data more efficiently, thereby improving the overall efficiency of research. However, although automated analysis methods have been proposed in the existing technology, these analysis methods still face many challenges in practical application, such as insufficient particle segmentation accuracy, complex feature extraction, and difficulty in batch statistical analysis. These problems limit the application of automated technology in SEM image analysis.

[0004] Therefore, developing a new automated SEM image analysis method to address the limitations of existing SEM image analysis has become an important direction of current research. Summary of the Invention

[0005] The purpose of the present invention is to provide a SEM image particle size analysis method and system based on a visual macro model, which realizes efficient and accurate segmentation and quantitative analysis of particles in SEM images by combining a visual macro model with an automated image analysis method.

[0006] To achieve the above objectives, the present invention discloses the following technical solutions:

[0007] A first aspect of the present invention provides a method for SEM image particle size analysis based on a visual macromodel, the method comprising the following steps:

[0008] Acquire an SEM image, and automatically identify the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel;

[0009] Collecting a set of SEM images containing particle features, annotating the particles in the SEM image set, and constructing a SEM image training set for fine-tuning the large visual model and a SEM image test set for evaluating segmentation accuracy based on the annotation results;

[0010] Constructing a large visual model, and fine-tuning the large visual model using the SEM image training set;

[0011] Using the fine-tuned visual macro model to perform reasoning on the SEM image test set to obtain particle segmentation results;

[0012] The properties of each segmented particle are calculated based on the actual length of each pixel, including the particle's area, perimeter, equivalent diameter, and / or shape factor for particle size analysis.

[0013] Optionally, in the above-mentioned SEM image particle size analysis method, the automatic identification of the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel, includes:

[0014] Detecting and recognizing text in the SEM image using an OCR model;

[0015] Post-process and filter the text recognition results to obtain the actual length of the scale and units;

[0016] Expanding the character area of the scale to obtain a region of interest containing the scale, and performing binarization processing on the region of interest to obtain a binarized image of the region of interest;

[0017] Traverse the binary image row by row from the left and right sides respectively, and detect the continuous length greater than the set threshold The white area determines the coordinates of the left endpoint of the scale bar and the right endpoint coordinates , and calculate the pixel length of the scale ;

[0018] According to the actual length of the scale and pixel length , calculate the actual length of each pixel .

[0019] Optionally, in the above-mentioned SEM image particle size analysis method, the post-processing screening of the text recognition results includes deleting non-scale characters through key characters and retaining scale characters containing numbers and unit letters.

[0020] Optionally, in the above-mentioned SEM image particle size analysis method, the collecting of a SEM image set containing particle features and the labeling of particles in the SEM image set include:

[0021] Collecting SEM images containing particle features of different particle sizes, shapes and / or distributions to form an SEM image set;

[0022] The image particles in the SEM image set are annotated using polygons, and an attribute label is assigned to each particle, where the attribute label includes the boundary, shape and / or position of the particle.

[0023] Optionally, in the above-mentioned SEM image particle size analysis method, the step of constructing a large visual model and fine-tuning the large visual model using the SEM image training set comprises:

[0024] performing data preprocessing on the original images in the SEM image training set to generate standardized images;

[0025] A SAM model is constructed based on the standardized image, the SAM model comprising an image encoder, a hint encoder, and a mask decoder; the image encoder is configured to extract features of the standardized image and generate an image feature embedding; the hint encoder is configured to encode the coordinates of a rectangular bounding box of the mask into a spatial embedding and generate a hint spatial embedding; the mask decoder generates a mask image to indicate a target region based on the image feature embedding and the hint spatial embedding, and outputs a confidence score corresponding to the target region;

[0026] performing mask post-processing on the mask image to generate a binary mask image;

[0027] The binary mask image is compared with the annotation result, the total loss function is calculated, and the parameters of the SAM model are fine-tuned through an optimization algorithm according to the value of the total loss function until the model converges.

[0028] Optionally, in the above-mentioned SEM image particle size analysis method, the data preprocessing of the original images in the SEM image training set includes:

[0029] Scaling the long side length of the original image in the SEM image training set to the target size;

[0030] Normalize each target pixel value of the image;

[0031] The normalized image is filled with pixel values to generate a normalized image.

[0032] Optionally, in the above-mentioned SEM image particle size analysis method, the use of the fine-tuned visual macro model to perform reasoning on the SEM image test set to obtain particle segmentation results includes:

[0033] Inputting the standardized images in the SEM image test set into the fine-tuned SAM model for particle segmentation, and outputting a predicted segmentation mask image;

[0034] Performing segmentation mask post-processing on the segmentation mask image and restoring it to the original image size through bilinear interpolation;

[0035] The restored segmentation mask image is thresholded and normalized to generate the final particle segmentation result image.

[0036] Optionally, the SEM image particle size analysis method described above, wherein the properties of each segmented particle are calculated based on the actual length of each pixel for particle size analysis, comprises:

[0037] Calculating relevant properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle;

[0038] Input the attribute range of the particles, filter the particles that meet the conditions, and generate the corresponding particle segmentation map and edge map;

[0039] Statistical analysis is performed on the particles that meet the conditions, and properties of the particles that meet the conditions are calculated, including average diameter, average area, average perimeter and / or average circularity.

[0040] A second aspect of the present invention provides a SEM image particle size analysis system based on a visual macromodel, the system comprising:

[0041] A scale recognition module is used to obtain a SEM image and automatically recognize the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel;

[0042] A data annotation module is used to collect a set of SEM images containing particle features, annotate the particles in the SEM image set, and construct a SEM image training set for fine-tuning the visual macro model and a SEM image test set for evaluating segmentation accuracy based on the annotation results;

[0043] A model fine-tuning module, used to construct a large visual model and fine-tune the large visual model using the SEM image training set;

[0044] A particle segmentation module is used to use the fine-tuned visual macro model to perform inference on the SEM image test set to obtain particle segmentation results;

[0045] The particle size analysis module is used to calculate the properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle for particle size analysis.

[0046] According to a third aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for SEM image particle size analysis based on a visual large model as described in the first aspect is implemented.

[0047] The effects provided in the summary of the invention are only the effects of the embodiments, not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:

[0048] The SEM image particle size analysis method and system based on the visual large model provided in this application realizes efficient and accurate segmentation and quantitative analysis of particles in SEM images by combining the visual large model and automated image analysis technology. The method first automatically identifies the scale to accurately obtain the actual length of each pixel, providing a data basis for the quantitative analysis of particle size and characteristics; then, based on a diverse set of SEM images, the model's ability to recognize particle features is improved through labeling and training processes; next, the fine-tuned visual large model is used to segment particles, and the various properties of the particles are calculated based on the actual length of the pixels, thereby providing high-precision particle size analysis results. Compared with the existing technology, this solution significantly improves the efficiency, accuracy and consistency of particle analysis, and meets the needs of multiple fields for efficient and accurate analysis of microscopic particles. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] Figure 1 This is a flow chart of a SEM image particle size analysis method based on a visual macro model according to an embodiment of the present application;

[0051] Figure 2 for Figure 1 A flowchart of a method for implementing step S110;

[0052] Figure 3 This is an SEM image of a quaternary precursor of battery materials;

[0053] Figure 4 For Figure 3 The SEM images in the screening diagram are scaled;

[0054] Figure 5 For Figure 4The region of interest map after the character area of the medium scale is enlarged;

[0055] Figure 6 for Figure 5 Binarized image of the region of interest in ;

[0056] Figure 7 Based on Figure 6 The pixel length identification map of the scale obtained by the binarization map in ;

[0057] Figure 8 for Figure 1 A flowchart of a method for implementing step S130;

[0058] Figure 9 This is the segmentation result after deleting particles with incomplete edges;

[0059] Figure 10 This is the particle edge map after deleting particles with incomplete edges;

[0060] Figure 11 This is a schematic structural diagram of a SEM image particle size analysis system based on a visual macro model according to an embodiment of the present application;

[0061] Figure 12 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0063] It should be noted that references to "one embodiment," "an embodiment," "an example embodiment," etc., in this specification indicate that the described embodiment may include specific features, structures, or characteristics. However, not every embodiment must include these specific features, structures, or characteristics. In addition, such references do not necessarily refer to the same embodiment. Furthermore, when specific features, structures, or characteristics are described in conjunction with an embodiment, whether or not explicitly described, it is indicated that it is within the knowledge of those skilled in the art to incorporate such features, structures, or characteristics into other embodiments.

[0064] As described in the background technology, existing SEM image analysis methods have significant deficiencies in automation accuracy and consistency. In particular, in the analysis of microscopic particle features, manual operation is time-consuming and easily affected by subjective factors, making it difficult to meet the needs of efficient and accurate research. Although advances in computer vision and machine learning have brought certain improvements to automated analysis, there are still problems such as insufficient particle segmentation accuracy, complex feature extraction, and difficulty in batch statistical analysis. To this end, an embodiment of the present invention provides a SEM image particle size analysis method and system based on a large visual model, which aims to achieve automated, high-precision segmentation and quantitative analysis of SEM images, thereby improving analysis efficiency and accuracy.

[0065] refer to Figure 1 , Figure 1 A schematic flow chart of a SEM image particle size analysis method based on a visual macro model provided in an embodiment of the present application is shown, and the method comprises the following steps:

[0066] S110, acquiring a SEM image, and automatically identifying a scale of the SEM image to obtain an actual length and a pixel length of the scale, as well as an actual length of each pixel;

[0067] In this step, the actual length of each pixel obtained can be used for subsequent accurate quantitative analysis of particle size and characteristics.

[0068] S120, collecting a set of SEM images containing particle features, annotating the particles in the SEM image set, and constructing a SEM image training set for fine-tuning the large visual model and a SEM image test set for evaluating segmentation accuracy based on the annotation results;

[0069] In this step, SEM images containing particle features of different particle sizes, shapes and / or distributions are collected to form a diverse SEM image set, which provides a rich data basis for subsequent annotation and model training to ensure the recognition requirements of various particle features.

[0070] S130, constructing a large visual model and fine-tuning the large visual model using the SEM image training set;

[0071] This step fine-tunes the large visual model to enable it to more accurately identify particle features based on the SEM image training set, thereby improving the accuracy and effect of subsequent particle segmentation.

[0072] S140, using the fine-tuned visual large model to reason on the SEM image test set to obtain particle segmentation results;

[0073] S150 , calculating the properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle, for particle size analysis.

[0074] The method in this example achieves high-precision segmentation and quantitative analysis of particles in SEM images through automatic scale recognition, particle annotation and training, and fine-tuning of a large visual model, effectively improving the accuracy and consistency of particle analysis. Compared with existing manual analysis methods, this method significantly improves analysis efficiency, reduces human error, and enhances repeatability. It is particularly suitable for applications requiring efficient and accurate analysis of microscopic particles.

[0075] refer to Figure 2 In one embodiment, the automatic identification of the scale of the SEM image in step S110 to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel, includes:

[0076] S111. Use the OCR model to detect and recognize text in the SEM image;

[0077] Preferably, the open source OCR model PaddleOCR is used to perform text detection and recognition on SEM images. Figure 3 As shown, taking an SEM image of a quaternary precursor of a battery material as an example, the text in the image is recognized.

[0078] S112, post-processing and screening the text recognition results to obtain the actual length of the scale and units;

[0079] In some embodiments, the text recognition results are post-processed and screened, and confirmation can be performed by deleting non-scale characters and retaining scale characters. Among them, the non-scale characters are mainly deleted by some key characters, such as colon, dot, KYKY, Sample, etc., and the scale characters mainly include two parts: numbers and letters. Figure 4 As shown, Figure 4 For Figure 3 The SEM image in the scale scale is screened to obtain the actual length of the scale The unit is 5, um.

[0080] S113, expanding the character area of the scale to obtain a region of interest containing the scale, and performing binarization processing on the region of interest to obtain a binarized image of the region of interest;

[0081] Specifically, the coordinates of the upper left corner of the scale character can be Move to the left Move up , the lower right corner coordinates Move to the right Move down At the same time, the coordinates of the region of interest of the scale bar should not exceed the SEM image area. If the coordinates exceed the SEM image area, the SEM image boundary shall prevail. Figure 5 and Figure 6 , Figure 5 For Figure 4 The region of interest after the character area of the medium scale is enlarged, Figure 6 for Figure 5 Binarized image of the region of interest in .

[0082] S114, traverse the binary image row by row from the left and right sides respectively, and detect whether the continuous length is greater than the set threshold The white area determines the coordinates of the left endpoint of the scale bar and the right endpoint coordinates , and calculate the pixel length of the scale ;

[0083] refer to Figure 7 As shown, Figure 7 Based on Figure 6 The pixel length identification map of the scale obtained by the binarization map in .

[0084] S115, according to the actual length of the scale and pixel length , calculate the actual length of each pixel .

[0085] This embodiment can effectively identify the actual length and pixel length of the scale bar through steps such as text detection, screening, region extraction, and boundary measurement, thereby obtaining the actual physical size of each pixel in the image, providing a basis for subsequent accurate quantitative analysis of particles.

[0086] In one embodiment, in the above step S120, collecting a set of SEM images containing particle features and labeling the particles in the SEM image set includes:

[0087] Collecting SEM images containing particle features of different particle sizes, shapes, and / or distributions to form a diverse SEM image set;

[0088] The open source software Labelme was used to annotate the particles in the SEM images. The particle outlines were outlined in a polygonal manner, and attribute labels were assigned to each particle. The attribute labels included characteristics such as the particle boundary, shape and / or position.

[0089] The image set and annotated data constructed in this step provide rich data support for the training and verification of subsequent models, ensuring that the subsequent large visual model can adapt to the recognition needs of various particle features.

[0090] refer to Figure 8 In one embodiment, in step S130, constructing a large visual model and fine-tuning the large visual model using the SEM image training set includes:

[0091] S131, performing data preprocessing on the original images in the SEM image training set to generate standardized images;

[0092] Specifically, in this step, the long side length of the original image in the SEM image training set can be scaled to the target size (e.g., 1024 pixels) to ensure the consistency of the image size and facilitate subsequent applications. Next, each pixel value is standardized by the preset mean and standard deviation, where the preset mean is , the preset variance is , the standardized calculation formula is , and finally fill the edges of the normalized image with pixel values , generating standardized images that meet analysis needs.

[0093] S132. Construct a SAM model (Segment Anything Model) based on the standardized image. The SAM model includes an image encoder, a hint encoder, and a mask decoder.

[0094] The image encoder is used to extract the features of the standardized image and generate image feature embedding; the hint encoder is used to encode the rectangular bounding box coordinates of the mask into a spatial embedding and generate a hint spatial embedding; the mask decoder is based on the image feature embedding and the hint spatial embedding, and generates an enlarged mask image to indicate the target area, and outputs a confidence score corresponding to the target area.

[0095] In one embodiment, the image encoder uses the Vit_b visual transformer model to extract features from standardized images. The image encoder's specific structure consists of three parts: an image block embedding layer, a backbone network layer, and a feature compression layer. The image block embedding layer uses convolution operations with a kernel size of (16, 16) and a stride of 16 to divide a 1024x1024x3 image into a series of 16x16x3 image blocks. The backbone network layer contains 12 transformer layers, each consisting of a multi-head self-attention layer with normalization and a multi-layer perceptron layer. The feature compression layer compresses the image embedding dimension from 768 to 256 using two convolutional layers and a normalization layer. After processing by the image encoder, the image shape is transformed from 1024x1024x3 to a 64x64x256 image feature embedding. The cue encoder encodes the rectangular bounding box coordinates (top left and bottom right corner coordinates) of the mask into a 256-dimensional cue space embedding to accurately locate the boundaries of particles in the image. The mask decoder uses two transformer layers to fuse the image feature embedding and the cue space embedding, and uses two transposed convolutional layers to quadruple the resolution of the fused embedding to 256x256. It ultimately outputs the segmentation result and the corresponding confidence score. This design enables the SAM model to efficiently and accurately complete the particle segmentation task.

[0096] S133, performing mask post-processing on the mask image to generate a binary mask image;

[0097] Specifically, in this step, the low-resolution mask image generated by the SAM model is first enlarged to 1024x1024 by bilinear interpolation; then the pixel values filled in step S121 are deleted. The mask image is restored to its original size through bilinear interpolation. Finally, the final binary mask image is generated through thresholding and normalization to clearly define the boundaries of the target area.

[0098] S134, comparing the binary mask image with the annotation result, calculating the total loss function, and fine-tuning the parameters of the SAM model through the optimization algorithm according to the value of the total loss function until the model converges;

[0099] Specifically, in this step, the binary mask image is first compared pixel by pixel with the annotation result, and the cross entropy loss and Dice loss are calculated. The cross entropy loss is used to measure the difference between the predicted result and the true label, and the Dice loss is used to measure the similarity between the predicted mask and the true mask. The loss function is obtained by adding the cross entropy loss and the Dice loss, that is:

[0100] ;

[0101] in, represents the total loss function, Represents the cross entropy loss, which is calculated as follows:

[0102] ;

[0103] It represents the Dice loss, which is calculated as follows:

[0104] ;

[0105] Next, based on the total loss function, the Adam optimization algorithm is used to adjust the parameters of the mask decoder in the SAM model to gradually reduce the loss function value. This process continues until the SAM model's loss function converges, indicating that the SAM model has achieved optimal performance on the training data. In this step, because the image encoder has been pre-trained on a large dataset, its structure is complex and contains a large number of parameters, and the cue encoder is already able to effectively encode bounding box cues, fine-tuning the lightweight mask decoder makes the fine-tuning process simpler and faster.

[0106] In one embodiment, in step S140, the fine-tuned large visual model is used to perform inference on the SEM image test set to obtain particle segmentation results, including:

[0107] First, the original images in the SEM image test set are preprocessed to generate standardized images. The preprocessing process can be specifically referred to step S131. Then, the standardized images in the SEM image test set are input into the fine-tuned SAM model for particle segmentation, and the predicted segmentation mask image is output. Next, the segmentation mask image is post-processed and restored to the size of the original image through bilinear interpolation. Finally, the restored segmentation mask image is thresholded, binarized, and normalized to generate the final particle segmentation result image. Figure 9 and Figure 10 , Figure 9 This is the segmentation result after deleting the incomplete particles on the edge. Figure 10 This is the particle edge map after deleting particles with incomplete edges.

[0108] In one embodiment, in step S150, the properties of each segmented particle are calculated based on the actual length of each pixel for particle size analysis, including:

[0109] Calculating relevant properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle;

[0110] Input the attribute range of the particles, filter the particles that meet the conditions, and generate the corresponding particle segmentation map and edge map;

[0111] Statistical analysis is performed on the particles that meet the conditions, and properties of the particles that meet the conditions are calculated, including average diameter, average area, average perimeter and / or average circularity.

[0112] In this example, efficient and accurate SEM image particle segmentation and size analysis are achieved by combining a visual macromodel (SAM model) with an automated particle analysis method. Using a fine-tuned visual macromodel for particle segmentation and analysis of SEM images reduces manual intervention and analysis time, improving processing efficiency. Automated scale recognition and image preprocessing, as well as particle attribute calculation based on the actual length of each pixel, ensure the accuracy and consistency of particle segmentation results, reducing errors and deviations. Users can filter particles based on different attributes and generate corresponding segmentation maps and edge maps for further analysis. Calculation of various particle attributes provides detailed data support for subsequent particle size analysis, facilitating in-depth characterization and making it suitable for high-precision particle size analysis.

[0113] It should be noted that for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the order of the actions described, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0114] refer to Figure 11 , Figure 11 The present invention provides a system for SEM image particle size analysis based on a large visual model. This system can be used to perform SEM image particle size analysis. The system described below can be used in conjunction with the SEM image particle size analysis method based on a large visual model described above. Specifically, the system includes:

[0115] A scale recognition module 1110 is used to acquire a SEM image and automatically recognize the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel;

[0116] A data annotation module 1120 is used to collect a set of SEM images containing particle features, annotate the particles in the SEM image set, and construct a SEM image training set for fine-tuning the large visual model and a SEM image test set for evaluating segmentation accuracy based on the annotation results;

[0117] A model fine-tuning module 1130 is used to construct a large visual model and fine-tune the large visual model using a SEM image training set;

[0118] The particle segmentation module 1140 is used to use the fine-tuned visual macro model to perform inference on the SEM image test set to obtain particle segmentation results;

[0119] The particle size analysis module 1150 is used to calculate the properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle, for particle size analysis.

[0120] For the processes not described in detail in the SEM image particle size analysis system based on the visual large model in this embodiment, reference can be made to the relevant parts in the above-mentioned embodiment of the SEM image particle size analysis method based on the visual large model, which will not be repeated here.

[0121] In addition, the present invention also provides an electronic device, such as Figure 12 As shown, the electronic device may include an input unit 1210, a memory 1220, a processor 1230, and an output unit 1240. The memory 1220 stores program instructions that can be executed on the processor 1230. The processor 1230 invokes the program instructions to execute the methods and / or technical solutions based on the aforementioned embodiments. For processes not described in detail in the electronic device of this embodiment, please refer to the relevant sections of the embodiment of the SEM image particle size analysis method based on a large visual model, and will not be repeated here.

[0122] The above is a detailed introduction to the SEM image particle size analysis method and system based on a visual large model provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the equipment disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0123] Although the present application has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications or improvements may be made based on the present application. Therefore, such modifications or improvements, which do not depart from the spirit of the present application, are within the scope of protection claimed in the present application.

Claims

1. A SEM image particle size analysis method based on a visual macro model, characterized in that: The following steps are involved: Acquire an SEM image and automatically identify the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel, including: using an OCR model to detect and recognize text in the SEM image; post-processing and screening the text recognition results to obtain the actual length L of the scale real and units; expand the character area of the scale to obtain a region of interest containing the scale, and perform binarization on the region of interest to obtain a binarized image of the region of interest; traverse the binarized image in rows from the left and right directions, detect white areas with a continuous length greater than a set threshold T1, and determine the left endpoint coordinate (x begin ,y begin ) and the right endpoint coordinate (x end ,y begin ), and calculate the pixel length L of the scale rulerPixel =x end -x begin ; According to the actual length L of the scale real and pixel length L rulerPixel , calculate the actual length L of each pixel lengthPerPixel =L real / L rulerPixel ; Collecting a set of SEM images containing particle features, annotating the particles in the SEM image set, and constructing a SEM image training set for fine-tuning the large visual model and a SEM image test set for evaluating segmentation accuracy based on the annotation results; Constructing a large visual model and fine-tuning the large visual model using the SEM image training set, including: performing data preprocessing on the original images in the SEM image training set to generate standardized images; constructing a SAM model based on the standardized images, the SAM model including an image encoder, a prompt encoder and a mask decoder; the image encoder is used to extract features of the standardized image and generate an image feature embedding; the prompt encoder is used to encode the rectangular bounding box coordinates of the mask into a spatial embedding to generate a prompt spatial embedding; the mask decoder is based on the image feature embedding and the prompt spatial embedding , generating a mask image to indicate the target area, and outputting a confidence score corresponding to the target area; performing mask post-processing on the mask image to generate a binary mask image; comparing the binary mask image with the annotation result, calculating a total loss function, and fine-tuning the parameters of the mask decoder in the SAM model through an optimization algorithm according to the value of the total loss function until the model converges; wherein the total loss function is obtained by adding a cross entropy loss and a Dice loss, the cross entropy loss is used to measure the difference between the predicted result and the true label, and the Dice loss is used to measure the similarity between the predicted mask and the true mask; Using the fine-tuned visual macro model to perform reasoning on the SEM image test set to obtain particle segmentation results; The properties of each segmented particle are calculated based on the actual length of each pixel, including the particle's area, perimeter, equivalent diameter, and / or shape factor for particle size analysis.

2. The SEM image particle size analysis method according to claim 1, wherein The constructing of the SAM model based on the standardized image further comprises: The image encoder uses a visual transformer Vit_b model to extract features of the standardized image. The image encoder includes an image block embedding layer, a backbone network layer, and a feature compression layer. The image block embedding layer uses a convolution operation with a convolution kernel size of (16, 16) and a step size of 16 to divide an image of size 1024x1024x3 into a series of 16x16x3 image blocks. The backbone network layer contains 12 transformer layers, each of which consists of a multi-head self-attention layer with normalization and a multi-layer perceptron layer. The feature compression layer is constructed by two convolution layers and The normalization layer compresses the image embedding dimension from 768 to 256. After processing by the image encoder, the image shape is transformed from 1024x1024x3 to a 64x64x256 image feature embedding. The hint encoder is used to encode the rectangular bounding box coordinates of the mask into a 256-dimensional hint space embedding. The mask decoder uses two layers of transformer layers to fuse the image feature embedding and the hint space embedding, and uses two layers of transposed convolutional layers to expand the resolution of the fused embedding feature by 4 times to 256x256, and finally outputs the segmentation result and the corresponding confidence score.

3. The SEM image particle size analysis method according to claim 1, wherein The post-processing and screening of the text recognition results includes deleting non-scale characters through key characters and retaining scale characters including numbers and unit letters.

4. The SEM image particle size analysis method according to claim 3, characterized in that: The collecting of a SEM image set containing particle features and labeling the particles in the SEM image set includes: Collecting SEM images containing particle features of different particle sizes, shapes and / or distributions to form an SEM image set; The image particles in the SEM image set are annotated using polygons, and an attribute label is assigned to each particle, where the attribute label includes the boundary, shape and / or position of the particle.

5. The SEM image particle size analysis method according to claim 4, characterized in that: The data preprocessing of the original images in the SEM image training set includes: Scaling the long side length of the original image in the SEM image training set to the target size; Normalize each target pixel value of the image; The normalized image is filled with pixel values to generate a normalized image.

6. The SEM image particle size analysis method according to claim 5, characterized in that: The method of using the fine-tuned large visual model to infer the SEM image test set to obtain particle segmentation results includes: Inputting the standardized images in the SEM image test set into the fine-tuned SAM model for particle segmentation, and outputting a predicted segmentation mask image; Performing segmentation mask post-processing on the segmentation mask image and restoring it to the original image size through bilinear interpolation; The restored segmentation mask image is thresholded and normalized to generate the final particle segmentation result image.

7. The SEM image particle size analysis method according to claim 6, characterized in that: The properties of each segmented particle are calculated based on the actual length of each pixel for particle size analysis, including: Calculating relevant properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle; Input the attribute range of the particles, filter the particles that meet the conditions, and generate the corresponding particle segmentation map and edge map; Statistical analysis is performed on the particles that meet the conditions, and properties of the particles that meet the conditions are calculated, including average diameter, average area, average perimeter and / or average circularity.

8. A SEM image particle size analysis system based on a visual macro model, characterized in that: The system comprises: The scale recognition module is used to obtain the SEM image and automatically recognize the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel, including: using the OCR model to detect and recognize the text in the SEM image; post-processing and screening the text recognition results to obtain the actual length L of the scale real and units; expand the character area of the scale to obtain a region of interest containing the scale, and perform binarization on the region of interest to obtain a binarized image of the region of interest; traverse the binarized image in rows from the left and right directions, detect white areas with a continuous length greater than a set threshold T1, and determine the left endpoint coordinate (x begin ,y begin ) and the right endpoint coordinate (x end ,y begin ), and calculate the pixel length L of the scale rulerPixel =x end -x begin ; According to the actual length L of the scale real and pixel length L rulerPixel , calculate the actual length L of each pixel lengthPerPixel =L real / L rulerPixel ; A data annotation module is used to collect a set of SEM images containing particle features, annotate the particles in the SEM image set, and construct a SEM image training set for fine-tuning the visual macro model and a SEM image test set for evaluating segmentation accuracy based on the annotation results; The model fine-tuning module is used to construct a large visual model and fine-tune the large visual model using the SEM image training set, including: performing data preprocessing on the original images in the SEM image training set to generate standardized images; constructing a SAM model based on the standardized images, the SAM model including an image encoder, a prompt encoder and a mask decoder; the image encoder is used to extract features of the standardized image and generate image feature embedding; the prompt encoder is used to encode the rectangular bounding box coordinates of the mask into a spatial embedding to generate a prompt spatial embedding; the mask decoder is used to extract features of the standardized image and generate a prompt spatial embedding based on the image feature embedding and the prompt encoder; The method comprises the steps of: performing mask post-processing on the mask image to generate a binary mask image; comparing the binary mask image with the annotation result, calculating a total loss function, and fine-tuning the parameters of the mask decoder in the SAM model through an optimization algorithm according to the value of the total loss function until the model converges; wherein the total loss function is obtained by adding a cross entropy loss and a Dice loss, wherein the cross entropy loss is used to measure the difference between the predicted result and the true label, and the Dice loss is used to measure the similarity between the predicted mask and the true mask; A particle segmentation module is used to use the fine-tuned visual macro model to perform inference on the SEM image test set to obtain particle segmentation results; The particle size analysis module is used to calculate the properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle for particle size analysis.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for SEM image particle size analysis based on a visual macro model according to any one of claims 1 to 7 is implemented.

Images