SEM image granularity analysis method and system based on visual large model

Abstract

The invention discloses an SEM image granularity analysis method and system based on a visual large model, and relates to the technical field of image recognition. The method comprises the steps that an SEM image is acquired, automatic scale identification is performed on the SEM image, and the actual length and the pixel length of the scale and the actual length of each pixel are obtained; an SEM image set containing particle features is collected, particles in the SEM image set are labeled, and an SEM image training set and an SEM image test set are constructed based on a labeling result; constructing a visual large model, and performing fine tuning on the visual large model by using the SEM image training set; reasoning the SEM image test set by using the fine-tuned visual large model to obtain a particle segmentation result; attributes of each segmented particle are calculated based on the actual length of each pixel for particle size analysis. According to the scheme, the efficiency, precision and consistency of particle analysis are improved, and the efficient and accurate analysis requirements of microscopic particles are met.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a SEM image particle size analysis method and system based on a visual macro model. Background Art

[0002] Traditional SEM (Scanning Electron Microscope) image analysis methods mainly rely on manual operation, which is often time-consuming and easily affected by subjective judgment. This not only limits the efficiency of the analysis, but also affects the accuracy of the results, resulting in significant differences in the analysis results of the same sample. In the rapidly developing scientific research, especially in the field of materials science, the demand for efficient and accurate analysis of microscopic particle characteristics is becoming increasingly urgent.

[0003] With the development and progress of science and technology, the application of computer vision and machine learning technology has provided new solutions for the analysis of SEM images. These new technologies can identify and quantify particle features in images, improving the speed and consistency of analysis. Through automated processes, researchers can process image data more efficiently, thereby improving the overall efficiency of research. However, in the existing technology, although automated analysis methods have been proposed, these analysis methods still face many challenges in practical applications, such as insufficient accuracy of particle segmentation, complexity of feature extraction, and difficulty in batch statistical analysis, which limits the application of automated technology in SEM image analysis.

[0004] Therefore, developing a new automated SEM image analysis method to address the limitations of existing SEM image analysis has become an important direction of current research. Summary of the invention

[0005] The purpose of the present invention is to provide a SEM image particle size analysis method and system based on a visual macro model, which realizes efficient and accurate segmentation and quantitative analysis of particles in SEM images by combining a visual macro model and an automated image analysis method.

[0006] To achieve the above object, the present invention discloses the following technical solution: A first aspect of the present invention provides a method for SEM image particle size analysis based on a visual macromodel, the method comprising the following steps: Acquire a SEM image, and automatically identify the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel; Collecting a set of SEM images containing particle features, annotating the particles in the SEM image set, and constructing a SEM image training set for fine-tuning the visual macro model and a SEM image test set for evaluating segmentation accuracy based on the annotation results; Constructing a large visual model, and fine-tuning the large visual model using the SEM image training set; Using the fine-tuned visual macro model to infer the SEM image test set to obtain particle segmentation results; The properties of each segmented particle are calculated based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle for particle size analysis.

[0007] Optionally, in the above-mentioned SEM image particle size analysis method, the automatic identification of the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel, includes: Using an OCR model to detect and recognize text in the SEM image; Post-process and filter the text recognition results to obtain the actual length of the scale and units; Expanding the character area of ​​the scale to obtain a region of interest including the scale, and binarizing the region of interest to obtain a binarized image of the region of interest; Traverse the binary image row by row from the left and right sides respectively, and detect the continuous length greater than the set threshold The white area determines the coordinates of the left endpoint of the scale bar. and the right endpoint coordinates , and calculate the pixel length of the scale ; According to the actual length of the scale and pixel length , calculate the actual length of each pixel .

[0008] Optionally, in the above-mentioned SEM image particle size analysis method, the post-processing screening of the text recognition results includes deleting non-scale characters through key characters and retaining scale characters including numbers and unit letters.

[0009] Optionally, in the above-mentioned SEM image particle size analysis method, the collecting of a SEM image set containing particle features and labeling of particles in the SEM image set includes: Collecting SEM images containing particle features of different particle sizes, shapes and / or distributions to form a SEM image set; The image particles in the SEM image set are annotated by polygons, and an attribute label is assigned to each particle, wherein the attribute label includes the boundary, shape and / or position of the particle.

[0010] Optionally, in the above-mentioned SEM image particle size analysis method, the step of constructing a large visual model and fine-tuning the large visual model using the SEM image training set comprises: Performing data preprocessing on the original images in the SEM image training set to generate standardized images; A SAM model is constructed based on the standardized image, wherein the SAM model includes an image encoder, a hint encoder and a mask decoder; the image encoder is used to extract features of the standardized image and generate an image feature embedding; the hint encoder is used to encode the rectangular bounding box coordinates of the mask into a spatial embedding and generate a hint spatial embedding; the mask decoder generates a mask image to indicate a target area based on the image feature embedding and the hint spatial embedding, and outputs a confidence score corresponding to the target area; Performing mask post-processing on the mask image to generate a binary mask image; The binary mask image is compared with the annotation result, the total loss function is calculated, and the parameters of the SAM model are fine-tuned through an optimization algorithm according to the value of the total loss function until the model converges.

[0011] Optionally, in the above-mentioned SEM image particle size analysis method, the data preprocessing of the original images in the SEM image training set includes: Scaling the long side length of the original image in the SEM image training set to a target size; Normalize each target pixel value of the image; Fill the pixel values ​​of the standardized image to generate a standardized image.

[0012] Optionally, in the above-mentioned SEM image particle size analysis method, the use of the fine-tuned visual macro model to infer the SEM image test set to obtain the particle segmentation result includes: Inputting the standardized images in the SEM image test set into the fine-tuned SAM model for particle segmentation, and outputting a predicted segmentation mask image; Performing segmentation mask post-processing on the segmentation mask image and restoring it to the original image size by bilinear interpolation; The restored segmentation mask image is thresholded and normalized to generate the final particle segmentation result image.

[0013] Optionally, the SEM image particle size analysis method described above, wherein the property of each particle segmented is calculated based on the actual length of each pixel for particle size analysis, comprises: Calculating relevant properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle; Input the attribute range of particles, filter the particles that meet the conditions, and generate the corresponding particle segmentation map and edge map; The particles that meet the conditions are statistically analyzed to calculate the properties of the particles that meet the conditions, including average diameter, average area, average perimeter and / or average circularity.

[0014] A second aspect of the present invention provides a SEM image particle size analysis system based on a visual macromodel, the system comprising: A scale recognition module is used to obtain a SEM image and automatically recognize the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel; A data annotation module, used to collect a set of SEM images containing particle features, annotate the particles in the SEM image set, and construct a SEM image training set for fine-tuning the visual macro model and a SEM image test set for evaluating segmentation accuracy based on the annotation results; A model fine-tuning module, used for constructing a large visual model and fine-tuning the large visual model using the SEM image training set; A particle segmentation module is used to use the fine-tuned visual macro model to infer the SEM image test set to obtain a particle segmentation result; The particle size analysis module is used to calculate the properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle for particle size analysis.

[0015] According to a third aspect of the present invention, there is provided an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for SEM image particle size analysis based on a visual large model as described in the first aspect is implemented.

[0016] The effects provided in the content of the invention are only the effects of the embodiments, not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects: The SEM image particle size analysis method and system based on the visual large model provided in this application realizes efficient and accurate segmentation and quantitative analysis of particles in SEM images by combining the visual large model and automated image analysis technology. The method first automatically identifies the scale to accurately obtain the actual length of each pixel, providing a data basis for the quantitative analysis of particle size and characteristics; then based on a diverse set of SEM images, the model's ability to recognize particle features is improved through annotation and training processes; next, the fine-tuned visual large model is used to segment particles, and the various properties of the particles are calculated based on the actual length of the pixel, thereby providing high-precision particle size analysis results. Compared with the prior art, this solution significantly improves the efficiency, accuracy and consistency of particle analysis, and meets the needs of multiple fields for efficient and accurate analysis of microscopic particles. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] Figure 1 A schematic diagram of a flow chart of a SEM image particle size analysis method based on a visual macro model according to an embodiment of the present application; Figure 2 for Figure 1 A flowchart of a method for implementing step S110; Figure 3 This is a SEM image of a quaternary precursor of battery materials; Figure 4 For Figure 3 The SEM images in the screening diagram are scaled; Figure 5 For Figure 4 The region of interest map after the character area of ​​the medium scale is enlarged; Figure 6 for Figure 5 Binarized image of the region of interest in ; Figure 7 Based on Figure 6 The pixel length identification map of the scale obtained by the binarization map in; Figure 8 for Figure 1 A flowchart of a method for implementing step S130; Fig. 9 This is the segmentation result after deleting the particles with incomplete edges; Fig.10 This is the particle edge map after deleting particles with incomplete edges; Fig.11 A schematic diagram of the structure of a SEM image particle size analysis system based on a visual macro model according to an embodiment of the present application; Fig.12 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] It should be noted that references to "one embodiment", "embodiment", "example embodiment", etc. in this specification refer to the embodiment being described which may include specific features, structures or characteristics, but not every embodiment must include these specific features, structures or characteristics. In addition, such expressions do not refer to the same embodiment. Furthermore, when describing specific features, structures or characteristics in conjunction with an embodiment, whether or not there is an explicit description, it is indicated that incorporating such features, structures or characteristics into other embodiments is within the knowledge of those skilled in the art.

[0021] As described in the background technology, the existing SEM image analysis methods have significant deficiencies in automation accuracy and consistency. Especially in the analysis of microscopic particle features, manual operation is time-consuming and easily affected by subjective factors, making it difficult to meet efficient and accurate research needs. Although the progress of computer vision and machine learning has brought certain improvements to automated analysis, there are still problems such as insufficient particle segmentation accuracy, complex feature extraction, and difficulty in batch statistical analysis. To this end, an embodiment of the present invention provides a SEM image particle size analysis method and system based on a large visual model, which aims to achieve automated, high-precision segmentation and quantitative analysis of SEM images, and improve analysis efficiency and accuracy.

[0022] refer to Figure 1 , Figure 1 A schematic flow chart of a SEM image particle size analysis method based on a visual macro model provided in an embodiment of the present application is shown, and the method comprises the following steps: S110, acquiring a SEM image, and automatically identifying the scale of the SEM image to obtain the actual length and pixel length of the scale, and the actual length of each pixel; In this step, the actual length of each pixel obtained can be used for subsequent accurate quantitative analysis of particle size and characteristics.

[0023] S120, collecting a set of SEM images containing particle features, and annotating the particles in the SEM image set, and constructing a SEM image training set for fine-tuning the large visual model and a SEM image test set for evaluating segmentation accuracy based on the annotation results; In this step, SEM images containing particle features of different particle sizes, shapes and / or distributions are collected to form a diverse SEM image set, which provides a rich data basis for subsequent annotation and model training to ensure the recognition requirements of various particle features.

[0024] S130, constructing a large visual model, and fine-tuning the large visual model using the SEM image training set; In this step, by fine-tuning the large visual model, it can more accurately identify particle features based on the SEM image training set, thereby improving the accuracy and effect of subsequent particle segmentation.

[0025] S140, using the fine-tuned visual large model to infer the SEM image test set to obtain particle segmentation results; S150, calculating the properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle for particle size analysis.

[0026] The method of this embodiment achieves high-precision segmentation and quantitative analysis of particles in SEM images through automatic scale recognition, particle annotation and training, and fine-tuning of the visual large model, effectively improving the accuracy and consistency of particle analysis. Compared with existing manual analysis methods, it has the advantages of significantly improving analysis efficiency, reducing human errors, and enhancing repeatability, and is particularly suitable for fields that require efficient and accurate analysis of microscopic particles.

[0027] refer to Figure 2 In one embodiment, the automatic identification of the scale of the SEM image in the above step S110 to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel, includes: S111, using the OCR model to detect and recognize text in the SEM image; Preferably, the open source OCR model PaddleOCR is used to perform text detection and recognition on SEM images. Figure 3 As shown, taking a SEM image of a quaternary precursor of battery materials as an example, the text in the image is recognized.

[0028] S112, post-processing and screening the text recognition results to obtain the actual length of the scale and units; In some embodiments, the text recognition results are post-processed and screened, and the confirmation can be carried out by deleting non-scale characters and retaining scale characters. Among them, the non-scale character part is mainly deleted by some key characters, such as colon, dot, KYKY, Sample, etc., and the scale character part mainly includes two parts: numbers and letter units. Figure 4 As shown, Figure 4For Figure 3 The SEM image in the scale bar is screened to obtain the actual length of the scale bar. The unit is um.

[0029] S113, expanding the character area of ​​the scale to obtain a region of interest including the scale, and performing binarization processing on the region of interest to obtain a binarized image of the region of interest; Specifically, the coordinates of the upper left corner of the scale character can be Move to the left Move up , the lower right corner coordinates Move to the right Move down At the same time, the coordinates of the region of interest of the scale bar should not exceed the SEM image area. If it exceeds the SEM image area, the SEM image boundary shall prevail. Figure 5 and Figure 6 , Figure 5 For Figure 4 The region of interest after the character area of ​​the medium scale is enlarged. Figure 6 for Figure 5 Binarized image of the region of interest in .

[0030] S114, traverse the binary image row by row from the left and right sides respectively, and detect whether the continuous length is greater than the set threshold The white area determines the coordinates of the left endpoint of the scale bar. and the right endpoint coordinates , and calculate the pixel length of the scale ; refer to Figure 7 As shown, Figure 7 Based on Figure 6 The pixel length identification map of the scale obtained by binarizing the map in .

[0031] S115, according to the actual length of the scale and pixel length , calculate the actual length of each pixel .

[0032] This embodiment can effectively identify the actual length and pixel length of the scale through steps such as text detection, screening, region extraction and boundary measurement, thereby obtaining the actual physical size of each pixel in the image, providing a basis for subsequent accurate quantitative analysis of particles.

[0033] In one embodiment, in the above step S120, collecting a SEM image set containing particle features and annotating the particles in the SEM image set includes: Collecting SEM images containing particle features of different particle sizes, shapes and / or distributions to form a diverse SEM image set; The open source software Labelme was used to annotate the particles in the SEM images. The particle contours were outlined in a polygonal manner, and attribute labels were assigned to each particle. The attribute labels included features such as the particle boundary, shape and / or position.

[0034] The image set and annotated data constructed in this step provide rich data support for the training and verification of subsequent models, ensuring that the subsequent large visual model can adapt to the recognition needs of various particle features.

[0035] refer to Figure 8 In one embodiment, in the above step S130, constructing a large visual model and fine-tuning the large visual model using the SEM image training set includes: S131, performing data preprocessing on the original images in the SEM image training set to generate standardized images; Specifically, in this step, the long side length of the original image in the SEM image training set can be first scaled to the target size (e.g., 1024 pixels) to ensure the consistency of the image size and facilitate subsequent applications. Next, each pixel value is standardized by the preset mean and standard deviation, where the preset mean is , the preset variance is , the standardized calculation formula is , and finally fill the edges of the standardized image with pixel values , generate standardized images that meet analysis needs.

[0036] S132. Construct a SAM model (Segment Anything Model) based on the standardized image. The SAM model includes an image encoder, a hint encoder, and a mask decoder. The image encoder is used to extract the features of the standardized image and generate image feature embedding; the hint encoder is used to encode the rectangular bounding box coordinates of the mask into a spatial embedding and generate a hint spatial embedding; the mask decoder generates an enlarged mask image to indicate the target area based on the image feature embedding and the hint spatial embedding, and outputs a confidence score corresponding to the target area.

[0037] In one embodiment, the image encoder uses a visual transformer Vit_b model to extract features of the standardized image. The specific structure of the image encoder can be divided into three parts: an image block embedding layer, a backbone network layer, and a feature compression layer. Among them, the image block embedding layer uses a convolution operation with a convolution kernel size of (16, 16) and a step size of 16 to divide an image of 1024x1024x3 into a series of 16x16x3 image blocks. The backbone network layer contains 12 layers of transformer layers, each of which consists of a multi-head self-attention layer with normalization and a multi-layer perceptron layer. The feature compression layer compresses the image embedding dimension from 768 to 256 through two layers of convolution layers and a normalization layer. After being processed by the image encoder, the image shape is transformed from 1024x1024x3 to an image feature embedding of 64x64x256. The prompt encoder is used to encode the rectangular bounding box coordinates (upper left corner and lower right corner coordinates) of the mask into a 256-dimensional prompt space embedding so as to accurately locate the boundaries of the particles in the image. The mask decoder uses two layers of transformer layers to fuse the image feature embedding and the hint space embedding, and uses two layers of transposed convolutional layers to expand the resolution of the fused embedding features by 4 times to 256x256, and finally outputs the segmentation result and the corresponding confidence score. This design enables the SAM model to efficiently and accurately complete the particle segmentation task.

[0038] S133, performing mask post-processing on the mask image to generate a binary mask image; Specifically, in this step, the low-resolution mask image generated by the SAM model is first enlarged to 1024x1024 by bilinear interpolation; then the pixel values ​​filled in step S121 are deleted. ; and restore the mask image size to the original image size again through bilinear interpolation. Finally, through thresholding and normalization operations, the final binary mask image is generated to clarify the boundary of the target area.

[0039] S134, comparing the binary mask image with the annotation result, calculating the total loss function, and fine-tuning the parameters of the SAM model through an optimization algorithm according to the value of the total loss function until the model converges; Specifically, in this step, the binary mask image is first compared pixel by pixel with the annotation result, and the cross entropy loss and Dice loss are calculated. The cross entropy loss is used to measure the difference between the predicted result and the true label, and the Dice loss is used to measure the similarity between the predicted mask and the true mask. The loss function is obtained by adding the cross entropy loss and the Dice loss, that is: ; in, represents the total loss function, Represents the cross entropy loss, which is calculated as follows: ; It represents Dice loss, which is calculated as follows: ; Next, according to the calculation result of the total loss function, the parameters of the mask decoder in the SAM model are adjusted through the Adam optimization algorithm to gradually reduce the loss function value. This process will continue until the loss function of the SAM model converges, indicating that the SAM model has achieved optimal performance on the training data. In this step, since the image encoder has been pre-trained on a large-scale dataset, the structure is complex and contains a large number of parameters, and the prompt encoder can effectively encode the bounding box prompt, therefore, choosing to fine-tune the lightweight mask decoder can make the fine-tuning process easier and faster.

[0040] In one embodiment, in the above step S140, the fine-tuned visual macro model is used to infer the SEM image test set to obtain the particle segmentation result, including: First, the original images in the SEM image test set are preprocessed to generate standardized images. The preprocessing process can be specifically referred to step S131; then the standardized images in the SEM image test set are input into the fine-tuned SAM model for particle segmentation, and the predicted segmentation mask image is output; next, the segmentation mask image is post-processed and restored to the size of the original image through bilinear interpolation; finally, the restored segmentation mask image is thresholded and normalized to generate the final particle segmentation result image. Fig. 9 and Fig.10 , Fig. 9 This is the segmentation result after deleting the incomplete particles on the edge. Fig.10 This is the particle edge map after deleting particles with incomplete edges.

[0041] In one embodiment, in the above step S150, the property of each segmented particle is calculated based on the actual length of each pixel for particle size analysis, including: Calculating relevant properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle; Input the attribute range of particles, filter the particles that meet the conditions, and generate the corresponding particle segmentation map and edge map; The particles that meet the conditions are statistically analyzed to calculate the properties of the particles that meet the conditions, including average diameter, average area, average perimeter and / or average circularity.

[0042] In this embodiment, by combining the visual macro model (SAM model) and the automated particle analysis method, efficient and accurate SEM image particle segmentation and particle size analysis are achieved. The fine-tuned visual macro model is used to segment and analyze SEM images, which reduces manual intervention and analysis time and improves processing efficiency. Through automated scale recognition and image preprocessing, as well as particle attribute calculation based on the actual length of each pixel, the accuracy and consistency of the particle segmentation results are ensured, and errors and deviations are reduced. When in use, users can screen according to different particle attributes, generate corresponding segmentation maps and edge maps, which are convenient for further analysis, and provide detailed data support for subsequent particle size analysis by calculating various properties of the particles, which is convenient for in-depth characteristic research and suitable for high-precision particle size analysis scenarios.

[0043] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0044] refer to Fig.11 , Fig.11 A SEM image particle size analysis system based on a visual large model provided in an embodiment of the present application is shown. The SEM image particle size analysis can be realized by using the system. The system described below can be referred to in correspondence with the SEM image particle size analysis method based on a visual large model described above. Specifically, the system includes: A scale recognition module 1110 is used to obtain a SEM image and automatically recognize the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel; A data annotation module 1120 is used to collect a set of SEM images containing particle features, annotate the particles in the SEM image set, and construct a SEM image training set for fine-tuning the visual macro model and a SEM image test set for evaluating segmentation accuracy based on the annotation results; A model fine-tuning module 1130 is used to construct a large visual model and fine-tune the large visual model using a SEM image training set; The particle segmentation module 1140 is used to use the fine-tuned visual macro model to infer the SEM image test set and obtain the particle segmentation result; The particle size analysis module 1150 is used to calculate the properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle, for particle size analysis.

[0045] For the processes not described in detail in the SEM image particle size analysis system based on the visual large model in this embodiment, reference can be made to the relevant parts in the above-mentioned SEM image particle size analysis method embodiment based on the visual large model, which will not be repeated here.

[0046] In addition, the present application also provides an electronic device, such as Fig.12 As shown, the electronic device may include an input unit 1210, a memory 1220, a processor 1230, and an output unit 1240. The memory 1220 stores program instructions that can be run on the processor 1230, and the processor 1230 calls the program instructions to execute the methods and / or technical solutions based on the aforementioned multiple embodiments. For the processes not described in detail in the electronic device in this embodiment, reference may be made to the relevant parts in the above-mentioned embodiment of the SEM image particle size analysis method based on the visual large model, which will not be repeated here.

[0047] The above is a detailed introduction to a SEM image particle size analysis method and system based on a visual large model provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referenced to each other. For the equipment disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0048] Although the present application has been described in detail above with general descriptions and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements may be made to the present application. Therefore, these modifications or improvements made without departing from the spirit of the present application all fall within the scope of protection claimed in the present application.

Claims

1. A SEM image particle size analysis method based on a visual macromodel, characterized in that: The following steps are involved: Acquire a SEM image, and automatically identify the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel; Collecting a set of SEM images containing particle features, annotating the particles in the SEM image set, and constructing a SEM image training set for fine-tuning the visual macro model and a SEM image test set for evaluating segmentation accuracy based on the annotation results; Constructing a large visual model, and fine-tuning the large visual model using the SEM image training set; Using the fine-tuned visual macro model to infer the SEM image test set to obtain particle segmentation results; The properties of each segmented particle are calculated based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle for particle size analysis.

2. The SEM image particle size analysis method according to claim 1, characterized in that: The automatic identification of the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel, includes: Using an OCR model to detect and recognize text in the SEM image; Post-process and filter the text recognition results to obtain the actual length of the scale and units; Expanding the character area of ​​the scale to obtain a region of interest including the scale, and binarizing the region of interest to obtain a binarized image of the region of interest; Traverse the binary image row by row from the left and right sides respectively, and detect the continuous length greater than the set threshold The white area determines the coordinates of the left endpoint of the scale bar. and the right endpoint coordinates , and calculate the pixel length of the scale ; According to the actual length of the scale and pixel length , calculate the actual length of each pixel .

3. The SEM image particle size analysis method according to claim 2, characterized in that: The post-processing and screening of the text recognition results includes deleting non-scale characters through key characters and retaining scale characters including numbers and unit letters.

4. The SEM image particle size analysis method according to any one of claims 1 to 3, characterized in that: The collecting of a SEM image set containing particle features and labeling the particles in the SEM image set includes: Collecting SEM images containing particle features of different particle sizes, shapes and / or distributions to form a SEM image set; The image particles in the SEM image set are annotated by polygons, and an attribute label is assigned to each particle, wherein the attribute label includes the boundary, shape and / or position of the particle.

5. The SEM image particle size analysis method according to claim 4, characterized in that: The step of constructing a large visual model and fine-tuning the large visual model using the SEM image training set includes: Performing data preprocessing on the original images in the SEM image training set to generate standardized images; A SAM model is constructed based on the standardized image, wherein the SAM model includes an image encoder, a hint encoder and a mask decoder; the image encoder is used to extract features of the standardized image and generate an image feature embedding; the hint encoder is used to encode the rectangular bounding box coordinates of the mask into a spatial embedding and generate a hint spatial embedding; the mask decoder generates a mask image to indicate a target area based on the image feature embedding and the hint spatial embedding, and outputs a confidence score corresponding to the target area; Performing mask post-processing on the mask image to generate a binary mask image; The binary mask image is compared with the annotation result, the total loss function is calculated, and the parameters of the SAM model are fine-tuned through an optimization algorithm according to the value of the total loss function until the model converges.

6. The SEM image particle size analysis method according to claim 5, characterized in that: The data preprocessing of the original images in the SEM image training set includes: Scaling the long side length of the original image in the SEM image training set to a target size; Normalize each target pixel value of the image; Fill the pixel values ​​of the standardized image to generate a standardized image.

7. The SEM image particle size analysis method according to claim 6, characterized in that: The method of using the fine-tuned visual macro model to infer the SEM image test set to obtain particle segmentation results includes: Inputting the standardized images in the SEM image test set into the fine-tuned SAM model for particle segmentation, and outputting a predicted segmentation mask image; Performing segmentation mask post-processing on the segmentation mask image and restoring it to the original image size by bilinear interpolation; The restored segmentation mask image is thresholded and normalized to generate the final particle segmentation result image.

8. The SEM image particle size analysis method according to claim 7, characterized in that: The method of calculating the properties of each segmented particle based on the actual length of each pixel for particle size analysis includes: Calculating relevant properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle; Input the attribute range of particles, filter the particles that meet the conditions, and generate the corresponding particle segmentation map and edge map; The particles that meet the conditions are statistically analyzed to calculate the properties of the particles that meet the conditions, including average diameter, average area, average perimeter and / or average circularity.

9. A SEM image particle size analysis system based on a visual macromodel, characterized in that: The system comprises: A scale recognition module is used to obtain a SEM image and automatically recognize the scale of the SEM image to obtain the actual length and pixel length of the scale, as well as the actual length of each pixel; A data annotation module, used to collect a set of SEM images containing particle features, annotate the particles in the SEM image set, and construct a SEM image training set for fine-tuning the visual macro model and a SEM image test set for evaluating segmentation accuracy based on the annotation results; A model fine-tuning module, used for constructing a large visual model and fine-tuning the large visual model using the SEM image training set; A particle segmentation module is used to use the fine-tuned visual macro model to infer the SEM image test set to obtain a particle segmentation result; The particle size analysis module is used to calculate the properties of each segmented particle based on the actual length of each pixel, including the area, perimeter, equivalent diameter and / or shape factor of the particle for particle size analysis.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the SEM image particle size analysis method based on a visual macro model as described in any one of claims 1 to 8 is implemented.

Images