Method for scanning electron microscope image stitching, electronic device, and storage medium

Abstract

The application discloses a method for scanning electron microscope image splicing, an electronic device and a storage medium. The method comprises the following steps: collecting multiple sub-images with an overlapping area under a scanning electron microscope; performing particle segmentation on each sub-image respectively by using an image segmentation model to obtain a mask contour corresponding to the particles in each sub-image; traversing the multiple sub-images to determine a reference sub-image and a to-be-spliced sub-image; calculating the range of the overlapping area between the reference sub-image and the to-be-spliced sub-image; determining incomplete particles of the mask contour in the reference sub-image within the image boundary of the reference sub-image; determining complete particles of the mask contour in the to-be-spliced sub-image within the range of the overlapping area; and splicing the reference sub-image with the incomplete particles deleted and the to-be-spliced sub-image with the complete particles deleted. By using the scheme, the particle segmentation splicing with a large field of view and high precision can be realized, and the image quality is improved.

Description

Technical Field

[0001] This application generally relates to the field of scanning electron microscope (SEM) imaging technology. More specifically, this application relates to a method, electronic device, and computer-readable storage medium for stitching SEM images. Background Technology

[0002] Scanning electron microscopy (SEM) can acquire clear morphological features of materials at the micrometer or even nanometer scale, providing an effective means for studying the physical and chemical properties of particulate materials. In the study of particulate materials, it is necessary to clearly separate each particle to ensure analytical accuracy, while also taking into account the field of view of the scanning electron microscope to ensure the accuracy of the particle statistical results.

[0003] Computer vision-based segmentation methods have become mainstream. For example, the powerful and versatile image segmentation model (SAM model) is used for many visual tasks. It can be adapted to the segmentation of particle-based scanning electron microscopy images without fine-tuning and can accurately distinguish particles from the background. For particle segmentation, the scanning electron microscope images need to ensure that the particle edges are as clear as possible. The smaller the field of view, the sharper the edge of a single particle, which is more conducive to the versatility of the segmentation. However, while a smaller field of view and sharper particle edges result in higher segmentation accuracy, it also leads to a smaller number of particles captured, resulting in a large deviation between the statistical results and the true distribution of the sample. Image stitching can balance image quality and field of view, but the resulting large image size requires scaling when directly segmenting with a deep learning model, leading to a decrease in image quality and extremely high requirements for computer graphics card configuration.

[0004] In view of this, there is an urgent need to provide a solution for image stitching in scanning electron microscopy in order to achieve large field of view, high precision particle segmentation and stitching, and improve image quality. Summary of the Invention

[0005] In order to at least solve one or more of the technical problems mentioned above, this application proposes a scheme for stitching scanning electron microscope images in several aspects.

[0006] In a first aspect, this application provides a method for stitching images from a scanning electron microscope, comprising: acquiring multiple sub-images with overlapping regions under a scanning electron microscope; performing particle segmentation on each sub-image using an image segmentation model to obtain mask contours corresponding to particles in each sub-image; traversing the multiple sub-images to determine a reference sub-image and a sub-image to be stitched; calculating the range of the overlapping region between the reference sub-image and the sub-image to be stitched; determining incomplete particles in the reference sub-image whose mask contours are within the image boundary of the reference sub-image; determining complete particles in the sub-image to be stitched whose mask contours are within the range of the overlapping region; and stitching the reference sub-image (with the incomplete particles removed) and the sub-image to be stitched (with the complete particles removed) together.

[0007] In some embodiments, the method further includes: performing particle segmentation on each sub-image using the image segmentation model, performing a first scaling operation on each sub-image to adapt to the input of the image segmentation model; and after obtaining the mask contours corresponding to the particles in each sub-image, performing a second scaling operation on the corresponding mask contours to adapt to the original size of the corresponding sub-image.

[0008] In some embodiments, the reference sub-image includes an initially acquired image that overlaps with the sub-image to be stitched, or an image that has undergone particle removal and overlaps with the next sub-image to be stitched.

[0009] In some embodiments, the overlap region range between a reference sub-image and a sub-image to be stitched is calculated by the following operations: calculating the percentage of the overlap region between the reference sub-image and the sub-image to be stitched; and calculating the target coordinate values ​​of the overlap region based on the percentage of the overlap region and the original size of the corresponding sub-image to obtain the overlap region range.

[0010] In some embodiments, incomplete particles of the mask contour in the reference sub-image within the image boundary of the reference sub-image are determined by the following operations: setting a preset value, determining a target rectangle within the reference sub-image at a distance of the preset value from the image boundary; and determining the particles in the reference sub-image where the mask contour intersects with the target rectangle as the incomplete particles.

[0011] In some embodiments, complete particles in the sub-image to be stitched are determined by the following operation: particles in the sub-image to be stitched whose mask contours are completely within the overlapping region are identified as complete particles.

[0012] In some embodiments, the reference sub-images include one or more, and in response to the inclusion of multiple reference sub-images, the overlapping region range between each reference sub-image and the sub-image to be stitched is calculated; incomplete particles with mask contours in each reference sub-image within the image boundary of the corresponding reference sub-image are determined; complete particles with mask contours in the sub-image to be stitched within the corresponding overlapping region range are determined; and multiple reference sub-images with incomplete particles removed are stitched together with the sub-image to be stitched with complete particles removed.

[0013] In some embodiments, the method further includes: traversing all valid particle masks in the stitched image; and sequentially indexing each valid particle mask, overwriting the initial index number.

[0014] In a second aspect, this application provides an electronic device, including: a processor; and a memory storing program instructions for stitching scanning electron microscope images, wherein when the program instructions are executed by the processor, the electronic device causes the electronic device to implement one or more embodiments of the first aspect described above.

[0015] In a third aspect, this application provides a computer-readable storage medium having stored thereon computer program instructions for stitching scanning electron microscope images, which, when executed by one or more processors, cause one or more embodiments of the first aspect to be implemented.

[0016] Using the scanning electron microscope image stitching scheme provided above, this embodiment of the application performs particle segmentation on each individual sub-image with overlapping regions through an image segmentation model. By selecting particles near the boundary in the baseline sub-image and particles located in the overlapping area in the sub-image to be stitched, it ensures that each particle remains intact and non-repeating during the stitching process. Sub-image stitching is achieved through particle-level mask fusion. Based on this, it not only achieves large field-of-view coverage but also retains the segmentation accuracy of individual particles. The stitching result can accurately restore the morphology and overall distribution of each particle, providing reliable particle-level data support for performance studies such as size statistics and morphology analysis of particulate materials, while taking into account the limitations of segmentation accuracy and computational resources. Attached Figure Description

[0017] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein:

[0018] Figure 1 This is an exemplary schematic diagram illustrating an image acquired by a scanning electron microscope according to an embodiment of this application; Figure 2 This is an exemplary flowchart illustrating a method 200 for stitching scanning electron microscope images according to an embodiment of this application; Figure 3 This is an exemplary schematic diagram illustrating image stitching according to an embodiment of this application; Figure 4 This is yet another exemplary schematic diagram illustrating image stitching according to an embodiment of this application; Figure 5 This is an exemplary schematic diagram illustrating an overall method for stitching scanning electron microscope images according to an embodiment of this application; Figure 6 This is an exemplary structural block diagram illustrating an electronic device 600 according to an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] It should be understood that the terms "comprising" and "including" used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0022] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0023] Scanning electron microscopy (SEM) uses a focused electron beam to scan the sample surface, converting the physical signals generated by the interaction between electrons and the sample into digital images reflecting the sample's morphology. First, the SEM's electron gun generates an electron beam through thermal emission or field emission. This beam is then focused step-by-step by multiple electromagnetic lenses to form a fine, focused electron beam with a diameter of only a few nanometers. Under the control of scanning coils, the focused electron beam performs a raster scan of the sample surface row by row and column by column. Next, the secondary electrons generated by the sample are captured by a detector and converted into weak electrical signals. These signals are amplified to a processable level by a signal amplifier and then converted into digital signals by an analog-to-digital converter. The intensity of the digital signal corresponds to the number of secondary electrons, which is then converted into grayscale values, ultimately forming a single-channel grayscale image reflecting the sample's surface morphology.

[0024] The field of view (FOP) in SEM refers to the size of the sample area covered by a single electron beam scan, which is determined by the electron beam scanning range. A smaller scanning range (i.e., a smaller FOP) results in higher electron beam scanning density in the local area, higher resolution, sharper particle edge details, and higher segmentation accuracy. Conversely, a larger scanning range (i.e., a larger FOP) leads to lower scanning density, lower resolution, and more blurred particle edges. Furthermore, the electron beam deflection range in SEM is physically limited, making it impossible to directly scan and cover an extremely large sample area with an ultra-large FOP. Therefore, precise movement of the sample stage is required to acquire multiple sub-images by scanning different regions, for example... Figure 1 As shown.

[0025] Figure 1 This is an exemplary schematic diagram illustrating an image acquired using a scanning electron microscope according to an embodiment of this application. For example... Figure 1 As shown, a scanning electron microscope scans row by row and column by column to obtain multiple sub-images, such as sub-image 101, sub-image 102, ..., sub-image 107, sub-image 108, ..., sub-image N. In this embodiment, there are overlapping regions between the sub-images. For example, taking sub-image 101 and sub-image 102 as an example, there is an overlapping region A. In addition, in the implementation scenario, while acquiring the sub-images, their respective position indices (i,j) can be recorded, representing the i-th row and j-th column. For example, sub-image 101 corresponds to (0,0); sub-image 102 corresponds to (0,1); and sub-image 107 corresponds to (1,0).

[0026] As described in the background section above, when capturing scanning electron microscope (SEM) images, factors such as field of view size, grain size, and image quality need to be considered. Image stitching can balance image quality and field of view size. However, the stitched image usually has a large image size; for example, stitching sub-images 101, 102, ..., 107, 108, ..., N into a single image. When using a deep learning model to segment the entire image subsequently, image scaling is often required, which significantly reduces image quality.

[0027] Based on this, this application proposes a new method for stitching scanning electron microscope images. By segmenting each sub-image into particles, and performing particle deletion processing based on the image boundaries and overlapping areas of each sub-image, the images are then stitched and fused together to achieve large field of view and high-precision particle segmentation stitching, thereby improving image quality.

[0028] The specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0029] Figure 2 This is an exemplary flowchart illustrating a method 200 for stitching scanning electron microscope images according to an embodiment of this application. Figure 2As shown, the method 200 may include: Step S201: acquiring multiple sub-images with overlapping regions under a scanning electron microscope; Step S202: using an image segmentation model to perform particle segmentation on each sub-image to obtain the mask contours corresponding to the particles in each sub-image; Step S203: traversing multiple sub-images to determine a reference sub-image and a sub-image to be stitched; Step S204: calculating the overlapping region range between the reference sub-image and the sub-image to be stitched; Step S205: determining incomplete particles in the reference sub-image whose mask contours are within the image boundary of the reference sub-image; Step S206: determining complete particles in the sub-image to be stitched whose mask contours are within the overlapping region range; Step S207: stitching the reference sub-image with incomplete particles removed and the sub-image to be stitched with complete particles removed.

[0030] First, in step S201, multiple sub-images with overlapping regions are acquired using a scanning electron microscope. As mentioned earlier, the scanning electron microscope captures images of the particulate material sample region by region in a row-plus-column order, obtaining multiple sub-images. Each sub-image records its position index i and j, where i represents the row number and j represents the column number. Based on this, each sub-image can be quickly located. The size of each sub-image can be uniformly set to H pixels in height and W pixels in width. The overlapping region is the part covered by adjacent sub-images when the scanning electron microscope captures images in different regions.

[0031] Based on the acquired multiple sub-images, in step S202, an image segmentation model is used to perform particle segmentation on each sub-image to obtain the mask contours corresponding to the particles in each sub-image. In some implementation scenarios, the image segmentation model can be, for example, the SAM model. This model can be pre-trained using a large amount of image data, and by inputting sub-images, the mask contours corresponding to the particles in each sub-image can be obtained. The mask contours are the edge lines of the segmented binary image; white areas with a pixel value of 1 correspond to particles, and black areas with a pixel value of 0 correspond to the background.

[0032] In some embodiments, the method further includes: performing particle segmentation on each sub-image using an image segmentation model, performing a first scaling operation on each sub-image to adapt to the input of the image segmentation model; and after obtaining the mask contours corresponding to the particles in each sub-image, performing a second scaling operation on the corresponding mask contours to adapt to the original size of the corresponding sub-image.

[0033] It's understandable that the SAM model has memory limitations; directly processing large sub-images can cause model lag or even crashes. The core purpose of the first scaling operation is to adapt to the model's input requirements. In some implementation scenarios, a single-channel grayscale image can first be converted into a three-channel color image. This conversion involves copying the grayscale values ​​of the grayscale image to the red, green, and blue channels to create a pseudo-color image, satisfying the SAM model's requirement that it only accepts three-channel image input. Next, the scaling ratio (scale) is calculated. For example, when the maximum value in the sub-image's height and width is less than 1024 pixels, scale is set to 1, and no scaling is needed; when the maximum value is greater than or equal to 1024 pixels, scale is equal to the ratio of that maximum value to 1024. Specifically, this can be expressed by the following formula:

[0034] The sub-image has dimensions H (height) and W (width). This represents the maximum value.

[0035] For the mask contours corresponding to particles in each sub-image obtained after segmentation, the scale can be applied to the original image scale, as mentioned above. For example, the `findContours` function can be used to extract contour lines. This function can traverse the white areas in the binary image, extract continuous edge coordinate points, and form closed contour lines. For example, the contour of a circular particle is a series of coordinate points distributed in an approximately circular pattern, while the contour of an irregularly shaped particle is a set of coordinate points corresponding to its shape. After contour extraction, the x-coordinate and y-coordinate of each coordinate point are divided by `scale` to restore the contour size. This ensures that the position and size of the restored contour are perfectly aligned with the particles in the original sub-image, laying the foundation for accurate matching during subsequent stitching.

[0036] In some implementation scenarios, an initial index number ID can be assigned to the mask contour corresponding to each particle to distinguish different particles in each sub-image and avoid confusion during subsequent processing.

[0037] Next, in step S203, multiple sub-images are traversed to determine the reference sub-image and the sub-image to be stitched. That is, the image containing the affected mask and the image containing the new mask are determined. The traversal process is performed sequentially based on the position index of the sub-images. For example, starting with the sub-image at position index (0,0), the horizontal sub-images at position indices (0,1) and (0,2) are processed sequentially, followed by the vertical sub-images at indices (1,0) and (1,1). In some embodiments, the reference sub-image includes either an initially acquired image that overlaps with the sub-image to be stitched, or an image that has undergone particle removal and overlaps with the next sub-image to be stitched. The sub-image to be stitched is the image currently to be fused with the reference sub-image; their roles dynamically switch as the processing progresses.

[0038] For example, during the initial stitching, the sub-image at position index (0,0) is used as the reference sub-image, and the sub-image at position index (0,1) is used as the sub-image to be stitched. Alternatively, after the sub-images at position index (0,0) and position index (0,1) have undergone particle removal, when merging them with the next sub-image to be stitched (e.g., position index (1,0)), the sub-images at position index (0,0) and position index (0,1) are used as the reference sub-images, and the sub-image at position index (1,0) is used as the sub-image to be stitched. This process is repeated to complete the stitching of all sub-images.

[0039] Further, in step S204, the overlap region range between the reference sub-image and the sub-image to be stitched is calculated. In some embodiments, the overlap region range between the reference sub-image and the sub-image to be stitched can be calculated by the following operations: calculating the percentage of the overlap region between the reference sub-image and the sub-image to be stitched; calculating the target coordinate values ​​of the overlap region based on the percentage of the overlap region and the original size of the corresponding sub-image to obtain the overlap region range.

[0040] In some implementation scenarios, the overlapping area can be defined by an overlapping area rectangle, the extent of which is determined by the coordinates of its top-left corner. , and the coordinates of the bottom right corner , Confirmed. That is, the target coordinates are... As an example, suppose the overlap ratio k between the reference sub-image and the sub-image to be stitched is [ratio k], and the original size of the corresponding sub-image is [original size k]. Then it can be based on , , , Calculate the range of the overlapping region Based on this, the location and extent of the overlapping area can be accurately pinpointed, providing an accurate basis for subsequent determination of whether particles are located within the overlapping area.

[0041] After determining the overlapping region, in step S205, incomplete particles with mask contours within the image boundaries of the reference sub-image are identified. In step S206, complete particles with mask contours within the overlapping region in the sub-image to be stitched are identified.

[0042] It is understandable that during sub-image stitching, some particles near the boundary of the overlapping area in the reference sub-image are truncated due to the limited shooting range. The extensions of these truncated, incomplete particles often appear in the sub-image to be stitched. If these incomplete particles are retained, they will overlap with the corresponding extensions in the sub-image to be stitched, affecting the completeness of the particle representation. Therefore, the reference sub-image needs to remove these incomplete particles, retaining only the complete, untruncated particles that are entirely within the image, providing a reliable particle reference for the overlapping area.

[0043] Furthermore, for complete particles within the overlapping region, since the corresponding physical particle already exists in the reference sub-image, and due to differences in brightness, contrast, and other factors during scanning electron microscopy, the segmentation mask for the same particle in the two images will exhibit slight pixel differences. If such complete particles are retained, the stitched image will have two different masks for the same particle, compromising particle uniqueness. Therefore, these complete particles entirely within the overlapping region are removed from the sub-image to be stitched, retaining only complete particles in the non-overlapping region and incomplete particles intersecting the overlapping region boundary. Based on this, the reference sub-image provides the complete particles within the overlapping region, and the sub-image to be stitched supplements the particles in the non-overlapping region and the extended portion of the overlapping region boundary, thus ensuring the uniqueness and continuity of the stitched particles.

[0044] In some embodiments, incomplete particles of the mask contour within the image boundary of the reference sub-image can be determined by the following operations: setting a preset value, determining a target rectangle within the reference sub-image at a distance of the preset value from the image boundary; and identifying particles where the mask contour intersects with the target rectangle as incomplete particles. For example, setting a preset value δ, the target rectangle intersects with the image boundary of the reference sub-image by shrinking it inward by δ pixels. The particles where the mask contour intersects with the target rectangle in the reference sub-image are then identified as incomplete particles.

[0045] In some embodiments, complete particles whose mask contours are located within the overlapping region in the sub-image to be stitched can be determined by the following operation: particles whose mask contours are completely located within the overlapping region in the sub-image to be stitched are determined as complete particles. In the implementation scenario, when the x-coordinates of all contour points of the mask contour in the sub-image to be stitched are within the overlapping region... Within the specified range, the particle is determined to be a complete particle located within the overlapping region.

[0046] Finally, in step S207, the reference sub-image with incomplete particles removed is stitched together with the sub-image to be stitched with complete particles removed. That is, incomplete particles are removed from the reference sub-image, and complete particles are removed from the sub-image to be stitched before the two sub-images are stitched together. Specifically, the remaining valid mask contours in the sub-image to be stitched are fused with the valid mask contours of the reference sub-image to form a unified mask set covering the range of both sub-images. The valid mask contours include the complete particles in the non-overlapping areas of the sub-image to be stitched, the intersecting particles at the boundaries of the overlapping areas, and the complete particles in the reference sub-image that were not removed.

[0047] In some embodiments, the reference sub-images may include one or more, and in response to the presence of multiple reference sub-images, the overlapping region range between each reference sub-image and the sub-image to be stitched is calculated; incomplete particles with mask contours in each reference sub-image within the image boundary of the corresponding reference sub-image are determined; complete particles with mask contours in the sub-image to be stitched within the corresponding overlapping region range are determined; and the multiple reference sub-images with incomplete particles removed are stitched together with the sub-image to be stitched with complete particles removed.

[0048] For example, when stitching two reference sub-images, one at position index (0,0) and the other at position index (0,1), with the next sub-image to be stitched (position index (1,0)), the overlapping areas of the sub-images at positions index (0,0) and (1,0), and the sub-images at positions index (0,1) and (1,0) are first determined. Next, incomplete particles are identified and deleted from the sub-images at positions index (0,0) and (0,1), and complete particles are identified and deleted from the sub-image at position index (1,0). The sub-images at positions index (0,0), (0,1), and (1,0) after particle deletion are then stitched together to obtain the stitched image.

[0049] Similarly, when stitching the three reference sub-images at position index (0,0), position index (0,1), and position index (1,0) with the next sub-image to be stitched (position index (1,1)), the incomplete particles of the sub-images at position index (0,0), position index (0,1), and position index (1,0) are deleted respectively, and the complete particles are deleted from the sub-image at position index (1,1) before stitching to obtain the stitched image.

[0050] In some embodiments, the method further includes: traversing all valid particle masks in the stitched image; and sequentially indexing each valid particle mask, overwriting the initial index number. That is, after stitching is completed, all valid particle masks need to be reassigned globally unique index numbers to avoid omissions or duplicate particles, which would prevent accurate acquisition of information such as particle quantity and size distribution, and ensure that a complete, high-quality image is obtained after stitching.

[0051] As described above, this embodiment of the application, based on sub-images with preset overlapping areas, utilizes a small field-of-view shooting method to sharpen the edges of each particle, laying a data foundation for precise particle-level segmentation. The image segmentation model segments each sub-image into particles one by one, accurately extracting the independent mask contour of each particle. Then, through the orderly division of the reference sub-image and the sub-image to be stitched, particles near the boundary in the reference sub-image and particles located in the overlapping area in the sub-image to be stitched are selectively chosen. This effectively solves the problem of mask duplication caused by differences in brightness and contrast in the segmentation results, ensuring that each particle remains intact and non-repeating during the stitching process.

[0052] Finally, sub-image stitching is achieved through granular mask fusion, gradually expanding the field of view in the form of data stream. Without the need for high-end hardware support, it achieves both large field of view coverage and retains the segmentation accuracy of individual particles. The stitching result can accurately restore the morphology and overall distribution of each particle, providing reliable granular data support for the performance research of particulate materials, such as size statistics and morphology analysis.

[0053] Figure 3 This is an exemplary schematic diagram illustrating image stitching according to an embodiment of this application. Figure 3 The image shown is a stitched image of two images. Taking the reference sub-image as the sub-image with position index (0, 0) and the sub-image to be stitched as the sub-image with position index (0, 1) as an example, ... Figure 3 In Figure (a), the left side shows the baseline sub-image (0, 0), and the right side shows the sub-image to be stitched (0, 1). The two images have overlapping region 'a'. As mentioned earlier, the particle mask contours are first segmented based on the image segmentation model. Actual segmentation involves a large number of particle masks; only a simplified illustration is shown in the figure. Figure 3 Figure (b) shows the mask contours O1, O2, and O3 in the reference sub-image (0, 0) and the mask contours S1, S2, and S3 in the sub-image to be stitched (0, 1). Next, the overlap region between the reference sub-image and the sub-image to be stitched is calculated. For details, please refer to the above. Figure 1 The overlapping area is described as shown by the dashed rectangle H1 in the figure.

[0054] In this scenario, incomplete particles in the reference sub-image, such as O2 and O3, can first be identified based on a preset value δ and removed from the reference sub-image. Next, complete particles in the sub-image to be stitched, such as S1, are identified and removed from the sub-image. Finally, the reference sub-images with the corresponding particles removed are stitched together to obtain the stitched image. Figure 3 As shown in Figure (c), the complete particle O1 in the overlapping area comes from the reference sub-image, while the complete particles S2 and S3 at the splicing point come from the sub-image to be spliced. Figure 3 Figure (d) in the diagram is an exemplary schematic diagram of the spliced ​​parts in a real-world scenario.

[0055] Figure 4 This is yet another exemplary schematic diagram illustrating image stitching according to an embodiment of this application. For example... Figure 4 The image shown is a composite of four images, similar to the one described above. Figure 3 Similarly, the sub-image at position index (0, 0) and the sub-image at position index (0, 1) are first stitched together after particle deletion (e.g.) Figure 4 As shown in Figure (a), when stitching with the next sub-image to be stitched (such as the sub-image at position index (1,0)), the previously processed sub-images at position index (0,0) and position index (0,1) are used as the reference sub-images. In this scenario, the overlapping area range H2 is first calculated. Referring to the aforementioned stitching process, incomplete particles (e.g., o2 and s2) are removed from the sub-images at position index (0,0) and position index (0,1), and complete particles (e.g., e1 and e2) are removed from the sub-image at position index (1,0) to be stitched, thus completing the stitching. Figure 4 As shown in Figure (b).

[0056] Furthermore, for the next sub-image to be stitched, such as the sub-image at position index (1,1), using the previously processed sub-images at position index (0,0), position index (0,1), and position index (1,0) as reference sub-images, incomplete particles are removed from each image, and complete particles are removed from the sub-image at position index (1,1) to be stitched, thereby completing the stitching and obtaining the final complete image. For example... Figure 4 As shown in Figure (c).

[0057] Figure 5 This is an exemplary schematic diagram illustrating an overall method for stitching scanning electron microscope images according to an embodiment of this application. Figure 5As shown, in step S501, multiple sub-images with overlapping regions are acquired. In step S502, particle segmentation is performed on each sub-image to obtain the mask contours corresponding to the particles in each sub-image. Specifically, segmentation can be completed using a trained image segmentation model, and the obtained mask contours can be initially indexed.

[0058] In step S503, multiple sub-images are traversed. In step S504, it is determined whether there is an overlapping region between the current sub-image to be stitched and the initial sub-image or a previously processed sub-image, and a reference sub-image is determined. If there is, in step S505, the overlapping region range between the reference sub-image and the sub-image to be stitched is calculated; if not, the process returns to step S503 to continue to the next sub-image. Based on the aforementioned overlapping region range, in step S506, incomplete particles within the image boundary of the reference sub-image are identified and deleted. In step S507, complete particles whose mask contours are within the overlapping region range in the sub-image to be stitched are identified and deleted.

[0059] Further, in step S508, the reference sub-image with incomplete particles removed is stitched together with the sub-image to be stitched with complete particles removed to obtain the stitched image. In step S509, after processing the removed particles, the mask index is numbered and updated to avoid omissions or inclusion of duplicate particles. For more details on each step, please refer to the above. Figure 2 The content described herein will not be repeated here.

[0060] Figure 6 This is an exemplary structural block diagram illustrating an electronic device 600 according to an embodiment of this application. For example... Figure 6 As shown, the electronic device 600 of this application may include a processor 601 and a memory 602, wherein the processor 601 and the memory 602 communicate via a bus. The memory 602 stores program instructions for stitching scanning electron microscope images. When these program instructions are executed by the processor 601, they enable the stitching of images according to the foregoing description. Figure 2 A method for stitching images from a scanning electron microscope is described.

[0061] Based on the foregoing description in conjunction with the accompanying drawings, those skilled in the art will understand that the embodiments of this application can also be implemented by software programs. Therefore, this application also provides a computer-readable storage medium. This computer-readable storage medium stores computer-readable instructions thereon for stitching scanning electron microscope images. When these computer-readable instructions are executed by one or more processors, they implement the embodiments of this application in conjunction with the accompanying drawings. Figure 2 The method described is for stitching images from a scanning electron microscope.

[0062] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0063] It should be noted that although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart can be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0064] It should be understood that when the terms "first," "second," "third," and "fourth," etc., are used in the claims, specification, and drawings of this application, they are used only to distinguish different objects and not to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0065] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0066] Although the embodiments of this application are described above, the content is merely an example adopted for the purpose of facilitating understanding of this application and is not intended to limit the scope and application scenarios of this application. Any person skilled in the art described in this application may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in this application, but the scope of patent protection of this application shall still be determined by the scope defined in the appended claims.

Claims

1. A method for stitching images from a scanning electron microscope, comprising: Acquire multiple sub-images with overlapping areas under a scanning electron microscope; The image segmentation model is used to perform particle segmentation on each sub-image to obtain the mask contours corresponding to the particles in each sub-image. Traverse the multiple sub-images to determine the reference sub-image and the sub-image to be stitched together; Calculate the extent of the overlapping region between the reference sub-image and the sub-image to be stitched; Determine the incomplete grains within the image boundary of the reference sub-image to define the mask contour in the reference sub-image; Identify complete particles in the sub-image to be stitched whose mask contours are located within the overlapping region; The reference sub-image with the incomplete particles removed is stitched together with the sub-image to be stitched after removing the complete particles.

2. The method according to claim 1, further comprising: The image segmentation model is used to perform granular segmentation on each sub-image, and a first scaling operation is performed on each sub-image to adapt to the input of the image segmentation model. After obtaining the mask contours corresponding to the particles in each sub-image, the corresponding mask contours are scaled a second time to fit the original size of the corresponding sub-image.

3. The method according to claim 2, wherein the reference sub-image includes an initially acquired image that overlaps with the sub-image to be stitched, or an image that has undergone particle removal and overlaps with the next sub-image to be stitched.

4. The method of claim 3, wherein the overlap region between the reference sub-image and the sub-image to be stitched is calculated by the following operations: Calculate the percentage of the overlapping area between the reference sub-image and the sub-image to be stitched; The target coordinates of the overlapping region are calculated based on the proportion of the overlapping region and the original size of the corresponding sub-image to obtain the range of the overlapping region.

5. The method of claim 1, wherein the incomplete grains of the mask contour in the reference sub-image within the image boundary of the reference sub-image are determined by the following operation: Set a preset value to determine a target rectangular box within the reference sub-image whose distance from the image boundary is the preset value; The particles whose mask contours intersect with the target rectangle in the reference sub-image are identified as the incomplete particles.

6. The method of claim 1, wherein complete particles in the sub-image to be stitched, whose mask contours lie within the overlapping region, are determined by the following operations: Particles whose mask contours are completely within the overlapping area in the sub-image to be stitched are identified as complete particles.

7. The method of claim 3, wherein the reference sub-image comprises one or more, and, Since the reference sub-image contains multiple sub-images, the overlapping region range between each reference sub-image and the sub-image to be stitched is calculated. Determine the incomplete particles of the mask contour within the image boundary of the corresponding reference sub-image in each of the reference sub-images; Determine complete particles in the sub-image to be stitched whose mask contours are located within the corresponding overlapping area; Multiple reference sub-images with incomplete particles removed are stitched together with the sub-image to be stitched after removing complete particles.

8. The method according to claim 1, further comprising: Iterate through all valid particle masks in the stitched image; Each valid particle mask is indexed sequentially, overwriting the initial index number.

9. An electronic device, comprising: processor; as well as A memory storing program instructions for stitching scanning electron microscope images, which, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1-8.

10. A computer-readable storage medium storing computer program instructions for stitching scanning electron microscope images thereon, wherein the computer program instructions, when executed by one or more processors, cause the method according to any one of claims 1-8 to be implemented.

Images