Method, device and storage medium for collision monitoring in a scanning electron microscope

Abstract

The present application relates to the technical field of scanning electron microscopes, and discloses a method, device, and storage medium for collision monitoring in a scanning electron microscope. The method comprises: collecting a reference frame image corresponding to a sample stage not placed in a sample compartment of a scanning electron microscope and an image data stream corresponding to a sample stage placed therein; performing inter-frame difference between the reference frame image and the initial frame image in the image data stream to extract an initial mask for placing the sample stage; based on the initial mask and historical frame image information in the image data stream, using a collision monitoring model to extract a target mask for placing the sample stage in the current frame; and performing collision monitoring in a scanning electron microscope based on the distance between the target mask and the target pole shoe. Using the scheme of the present application, accurate moving targets can be determined and the accuracy of collision detection can be improved.

Description

Technical Field

[0001] The present application generally relates to the field of scanning electron microscopy technology. More specifically, the present application relates to a method, an apparatus and a computer-readable storage medium for collision monitoring in a scanning electron microscope. Background Art

[0002] The structure of the scanning electron microscope is very complex, and its parts are numerous and expensive. If the sample stage and the pole shoe collide due to misoperation, resulting in damage to the parts, it will cause serious economic losses. Therefore, monitoring the real-time height from the sample stage to the pole shoe in the sample chamber of the scanning electron microscope and giving appropriate warnings when the distance between the two is close has great practical value in the scanning electron microscope.

[0003] At present, the collision monitoring method in scanning electron microscopes is usually achieved by using image processing for target recognition and tracking. For example, the frame difference method in image processing is used for monitoring. The aforementioned frame difference method uses the changes in pixel values ​​in adjacent frames of continuous images to capture moving objects in the image. However, the frame difference method is only applicable to scenes with no other interference except moving objects. In the sample chamber of the scanning electron microscope, due to the high processing accuracy of mechanical parts and the low surface roughness of parts, the moving sample stage will be reflected on the surface of other parts. This will interfere with the results of the frame difference method, causing misjudgment of moving objects, thereby further affecting the accuracy of collision monitoring.

[0004] In view of this, there is an urgent need to provide a solution for collision monitoring in a scanning electron microscope in order to determine the accurate moving target and improve the accuracy of collision detection. Summary of the invention

[0005] In order to at least solve one or more of the above-mentioned technical problems, the present application proposes a solution for collision monitoring in a scanning electron microscope in multiple aspects.

[0006] In a first aspect, the present application provides a method for collision monitoring in a scanning electron microscope, comprising: acquiring a reference frame image corresponding to a sample stage not placed in a sample chamber of the scanning electron microscope and an image data stream corresponding to a sample stage placed therein; performing inter-frame difference between the reference frame image and an initial frame image in the image data stream to extract an initial mask for placing the sample stage; based on the initial mask and historical frame image information in the image data stream, using a collision monitoring model to extract a target mask for placing the sample stage in a current frame; and performing collision monitoring in the scanning electron microscope according to the distance between the target mask and the target pole shoe.

[0007] In some embodiments, before performing frame difference on the reference frame image and the initial frame image in the image data stream, it also includes: performing noise reduction and / or standardization operations on the reference frame image and the initial frame image in the image data stream.

[0008] In other embodiments, performing frame difference between the reference frame image and the initial frame image in the image data stream to extract an initial mask for placing the sample stage includes: performing frame difference between the reference frame image and the initial frame image in the image data stream to determine a motion pixel mask; determining a target connected domain based on the motion pixel mask; and calculating a convex hull of outer contour points based on the contour of the target connected domain, and filling the convex hull to extract the initial mask for placing the sample stage.

[0009] In some further embodiments, determining the target connected domain based on the motion pixel mask includes: obtaining an initial connected domain in the motion pixel mask; calculating a contour area of ​​each of the initial connected domains; and determining an initial connected domain having a contour area greater than a first preset threshold as the target connected domain.

[0010] In some other embodiments, the collision monitoring model includes an encoder, a fusion module and a decoder, and based on the initial mask and the historical frame image information in the image data stream, using the collision monitoring model to extract the target mask where the sample stage is placed in the current frame includes: using the initial mask as prompt information, using the encoder to extract the mask features of the current frame image; using the fusion module to fuse the mask features and the historical frame image information to obtain a fusion result; and based on the fusion result, using the decoder to perform a decoding operation to extract the target mask where the sample stage is placed in the current frame.

[0011] In some further embodiments, it further includes: performing image scaling and / or normalization operations on the initial mask and the current frame image.

[0012] In some further embodiments, it further includes: determining whether the amount of the historical frame image information exceeds a preset amount; and in response to the amount of the historical frame image information exceeding the preset amount, deleting target historical frame image information from the historical frame image information.

[0013] In some further embodiments, it further includes: in response to the target mask containing different connected domains, deleting the connected domains whose contour area is smaller than a second preset threshold; and performing convex hull filling of outer contour points on the remaining connected domains to obtain a final target mask.

[0014] In some other embodiments, collision monitoring in a scanning electron microscope based on the distance between the target mask and the target pole shoe includes: obtaining the minimum vertical coordinate of the target motion pixel in the target mask and the target vertical coordinate of the target pole shoe; and in response to the distance between the minimum vertical coordinate and the target vertical coordinate being greater than a distance threshold, determining that there is a collision risk in the scanning electron microscope.

[0015] In a second aspect, the present application provides a device for collision monitoring in a scanning electron microscope, comprising: a processor; and a memory, in which program instructions for collision monitoring in a scanning electron microscope are stored, and when the program instructions are executed by the processor, the device implements one or more embodiments of the aforementioned first aspect.

[0016] In a third aspect, the present application provides a computer-readable storage medium having stored thereon computer-readable instructions for collision monitoring in a scanning electron microscope, wherein when the computer-readable instructions are executed by one or more processors, one or more embodiments of the aforementioned first aspect are implemented.

[0017] Through the scheme for collision monitoring in a scanning electron microscope provided above, the embodiment of the present application first performs inter-frame difference between the reference frame image corresponding to the unplaced sample stage and the initial frame image in the image data stream corresponding to the placed sample stage to extract the initial mask. Then, based on the initial mask and the historical frame image information, the collision monitoring model is used to extract the target mask. That is, the embodiment of the present application, on the initial mask obtained based on the inter-frame difference, combines the historical frame image information, and uses deep learning (or machine learning) to track the moving object identified in the image data stream, which can effectively reduce the discreteness of the tracking result of the frame difference method, effectively track the changes in the shape of the tracked object caused by the change of the camera perspective, so as to better adapt to the recognition of the sample stage in the scanning electron microscope, and at the same time suppress the "mirror effect" that cannot be avoided by the inter-frame difference. In this way, an accurate target mask can be obtained, and then accurate monitoring of the collision can be achieved through the distance between the target mask and the target pole shoe. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0019] Figure 1 is an exemplary flowchart showing a method for collision monitoring in a scanning electron microscope according to an embodiment of the present application;

[0020] Figure 2is an exemplary flowchart showing the extraction of an initial mask for placing a sample stage according to an embodiment of the present application;

[0021] Figure 3 is an exemplary flowchart showing the extraction of a target mask for placing a sample stage according to an embodiment of the present application;

[0022] Figure 4 is an exemplary flow chart showing an overall process for collision monitoring in a scanning electron microscope according to an embodiment of the present application;

[0023] Figure 5 is an exemplary schematic diagram showing collision monitoring in a scanning electron microscope according to an embodiment of the present application;

[0024] Figure 6 is an exemplary structural block diagram showing a device for collision monitoring in a scanning electron microscope according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0026] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present application indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0027] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0028] As used in this specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0029] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0030] Figure 1 FIG. 1 is an exemplary flowchart of a method 100 for collision monitoring in a scanning electron microscope according to an embodiment of the present application. Figure 1 As shown in , at step S101, a reference frame image corresponding to a sample stage not being placed in a sample chamber of a scanning electron microscope and an image data stream corresponding to a sample stage being placed are collected. In some implementation scenarios, a reference frame image corresponding to a sample stage not being placed in a sample chamber of a scanning electron microscope (i.e., a frame image of an initial state) and a corresponding image data stream (i.e., continuous frame image data) can be collected at preset time intervals (e.g., 0.02s) after the sample stage is placed, for example, by a grayscale camera.

[0031] Based on the reference frame image and image data stream collected as described above, at step S102, the reference frame image and the initial frame image in the image data stream are subjected to inter-frame difference to extract the initial mask for placing the sample stage. It can be understood that when using a grayscale camera to collect images, the grayscale camera will automatically adjust the brightness due to the change in light in the sample chamber before and after the sample stage is placed, resulting in a large error in the inter-frame difference. Therefore, before inter-frame difference is performed on the reference frame image and the initial frame image in the image data stream, pre-processing operations such as noise reduction and / or standardization operations can be performed on the reference frame image and the initial frame image in the image data stream. This can reduce the noise in the image, reduce or eliminate the change in pixel value in the non-sample stage area caused by brightness changes, and reduce the error in inter-frame difference to facilitate subsequent accurate calculations.

[0032] In one implementation scenario, the reference frame image and the initial frame image in the image data stream may be subjected to noise reduction operations, for example, by using Gaussian filtering. In another implementation scenario, the reference frame image and the initial frame image in the image data stream may be subjected to standardization operations, respectively, by using the following formulas:

[0033] (1)

[0034] in, represents the standardized result, Represents the reference frame image or the initial frame image, and Denote the pixel mean and pixel variance corresponding to the reference frame image or the initial frame image respectively. After performing the noise reduction operation and / or the standardization operation, inter-frame difference can be performed to extract the initial mask of the sample stage.

[0035] In some embodiments, the reference frame image and the initial frame image in the image data stream may be inter-frame differentiated to determine a motion pixel mask, and the target connected domain may be determined based on the motion pixel mask, and then the convex hull of the outer contour points may be calculated according to the contour of the target connected domain, and the convex hull may be filled to extract the initial mask for placing the sample stage. In some embodiments, by obtaining the initial connected domain in the motion pixel mask and calculating the contour area of ​​each initial connected domain, the initial connected domain whose contour area is greater than a first preset threshold value may be determined as the target connected domain.

[0036] In some implementation scenarios, when performing inter-frame difference, the motion pixel mask in the image can be determined by, for example, binary threshold processing. As an example, the motion pixel mask can be determined by the following formula:

[0037] (2)

[0038] Where D represents the motion pixel mask, represents the normalized result of the reference frame image without the sample stage. ) represents the normalized result of the initial frame image in the image data stream corresponding to the sample stage, represents the frame difference, Indicates setting a threshold.

[0039] Since the images captured by the grayscale camera usually contain noise, the above-mentioned noise reduction operation alone may not be able to completely eliminate the influence, so the above-mentioned motion pixel mask can be further processed to improve the image accuracy. In the embodiment of the present application, all independent connected domains in the above-mentioned motion pixel mask can be obtained, and then the contour area of ​​each connected domain is calculated, and the contours of the connected domains whose contour area is greater than the first preset threshold are retained. Finally, all the contour points obtained by screening are subjected to convex hull processing to obtain the convex hull area of ​​the motion area mask as the initial mask.

[0040] Specifically, in an implementation scenario, the aforementioned operation of extracting the initial mask can be expressed by the following formula:

[0041] (3)

[0042] in, represents the initial mask extracted, Represents motion pixel mask A single connected domain in represents the outer contour point of the i-th connected domain, represents the contour area of ​​the i-th connected domain, Indicates the first preset threshold.

[0043] Next, at step S103, based on the initial mask and the historical frame image information in the image data stream, the collision monitoring model is used to extract the target mask where the sample stage is placed in the current frame. In some embodiments, the collision monitoring model can be a network model in deep learning or machine learning. Preferably, the collision monitoring model can be, for example, SegmentAnything2 ("SAM2"). In some implementation scenarios, the aforementioned collision monitoring model can include at least an encoder, a fusion module, and a decoder.

[0044] In some embodiments, the initial mask is used as the prompt information, and the encoder is used to extract the mask features of the current frame image, and then the mask features and the historical frame image information are fused using the fusion module to obtain the fusion result, and then based on the fusion result, the decoder is used to perform a decoding operation to extract the target mask of the sample stage in the current frame. In some implementation scenarios, the aforementioned mask features and the historical frame image information can be fused by, for example, attention mechanism calculation.

[0045] Specifically, the initial mask and the remaining image frames (i.e., image frames other than the initial frame) in the image data stream corresponding to the sample stage are input into the collision monitoring model. The encoder first extracts the mask features of the current frame image, and then the fusion module performs, for example, an attention operation based on the mask features and the historical frame image information to fuse the mask features and the historical frame image information to obtain a fusion result. Finally, the decoder decodes the fusion result to obtain the target mask of the sample stage in the current frame.

[0046] In some embodiments, before extracting the target mask, the initial mask and the current frame image may be subjected to image scaling and / or normalization operations to ensure the reasoning speed of the collision monitoring model. In other embodiments, considering the camera resolution and reasoning speed, target tracking may also be performed using, for example, lightweight model weights. As an example, the image may be scaled from the original 2000x1544 to a long side of 1024, and to ensure the corresponding model weight, the grayscale image may be converted to an RGB image, and the pixel value may be divided by 255 for normalization. Further, the initial mask and the current frame image after the scaling and / or normalization operations are input into the collision monitoring model to extract the target mask. It should be understood that the features obtained by the current frame are saved in the video memory, and no prompt mask will be input for tracking of non-starting frames, and only the processed (scaled and normalized) current frame image will be input.

[0047] In some embodiments, it can also be determined whether the number of historical frame image information exceeds a preset number (for example, 30). In response to the number of historical frame image information exceeding the preset number, the target historical frame image information is deleted from the historical frame image information. The aforementioned target historical frame image information is the earliest historical image information. In this way, the problem of video memory overflow caused by too many historical features can be prevented. In other embodiments, in response to the target mask containing different connected domains, the connected domains with a contour area less than a second preset threshold (for example, 20 pixels) are deleted, and the convex hull filling of the outer contour points of the remaining connected domains is performed to obtain the final target mask. Based on this, discrete points or holes in the mask can be eliminated to obtain a more accurate target mask.

[0048] Further, at step S104, collision monitoring in the scanning electron microscope is performed according to the distance between the target mask and the target pole shoe. In some embodiments, by obtaining the minimum ordinate of the target motion pixel in the target mask and the target ordinate of the target pole shoe, in response to the distance between the minimum ordinate and the target ordinate being greater than a distance threshold, it is determined that there is a collision risk in the scanning electron microscope.

[0049] It can be understood that when the grayscale camera in the sample chamber of the scanning electron microscope is fixed, the pixel position of the pole shoe in the grayscale image is fixed. When performing collision monitoring, if the distance between the minimum ordinate of the target moving pixel and the position of the pole shoe (target ordinate) is greater than the distance threshold, it is determined that there is a collision risk to start the collision warning. Otherwise, there is no collision risk.

[0050] In an exemplary scenario, assume that the target vertical coordinate is marked as , the minimum vertical coordinate of the target motion pixel is marked as , and the distance between the minimum ordinate and the target ordinate mark is recorded as ,but In addition, the distance threshold is recorded as G, and the collision monitoring in the scanning electron microscope can be performed by the following formula:

[0051] (4)

[0052] Combined with the above description, it can be seen that the embodiment of the present application first uses a camera to collect a reference frame image of a scanning electron microscope without a sample stage and an image data stream after the sample stage is placed, and uses the frame difference method to obtain the initial mask of the placed sample stage. Using the initial mask as prompt information, the target mask of the sample stage tracked in the current frame is obtained in combination with the deep learning model (collision monitoring model) and the historical frame image information. Based on this, the discreteness of the tracking results of the frame difference method can be effectively reduced, and the changes in the shape of the tracked object caused by the change of the camera's viewing angle can be effectively tracked, so as to better adapt to the recognition of the sample stage in the scanning electron microscope, while suppressing the "mirror effect" that cannot be avoided by the inter-frame difference. In this way, an accurate target mask can be obtained, and then accurate monitoring of the collision can be achieved through the distance between the target mask and the target pole shoe.

[0053] Figure 2 FIG. 1 is an exemplary flowchart showing the extraction of the initial mask of the sample stage according to an embodiment of the present application. It should be understood that Figure 2 is the above Figure 1 is a specific embodiment of step S102 in method 100, so the above Figure 1 The description also applies to Figure 2 .

[0054] like Figure 2 As shown in , at step S201, a reference frame image corresponding to a sample stage not being placed and an initial frame image in an image data stream corresponding to a sample stage being placed are obtained. Next, at step S202, a noise reduction operation and / or a standardization operation are performed on the reference frame image and the initial frame image in the image data stream to reduce noise in the image, and reduce or eliminate pixel value changes in the non-sample stage area caused by brightness changes, thereby reducing errors in inter-frame differences. In some embodiments, the aforementioned noise reduction operation can be performed by, for example, Gaussian filtering, and the aforementioned standardization operation can be implemented based on the above formula (1).

[0055] Furthermore, at step S203, the reference frame image and the initial frame image in the image data stream are subjected to inter-frame difference, and at step S204, the inter-frame difference result is subjected to binary threshold processing (see the above formula (2)) to obtain a motion pixel mask. At step S205, the initial connected domain in the motion pixel mask is obtained, and the contour area of ​​each initial connected domain is calculated, and at step S206, the initial connected domain whose contour area is greater than the first preset threshold is determined as the target connected domain. Based on the target connected domain, at step S207, the convex hull of the outer contour points is calculated according to the contour of the target connected domain, and the convex hull is filled to extract the initial mask for placing the sample stage, and the details can be referred to the above formula (3). Based on this, noise can be effectively eliminated and image accuracy can be improved.

[0056] Figure 3FIG. 1 is an exemplary flowchart showing the process of extracting a target mask for placing a sample stage according to an embodiment of the present application. It should be understood that Figure 3 is the above Figure 1 is a specific embodiment of step S103 in method 100, so the above Figure 1 The description also applies to Figure 3 .

[0057] like Figure 3 As shown in, at step S301, an image scaling operation and / or normalization operation are performed on the initial mask and the current frame image. Based on the initial mask and the current frame image after the scaling and / or normalization operation, at step S302, the initial mask and the current frame image are input into the collision monitoring model. As can be seen from the foregoing, the collision monitoring model can be, for example, a SAM2 model, and the collision monitoring model can include at least an encoder, a fusion module, and a decoder. At step S303, the initial mask is used as a prompt information, and the encoder is used to extract the mask features of the current frame image, and then at step S304, the fusion module is used to fuse the mask features and the historical frame image information to obtain a fusion result. In some embodiments, the mask features and the historical frame image information can be fused by, for example, an attention mechanism. In other embodiments, in order to place the problem of video memory overflow caused by too many historical features, the earliest historical image information can be deleted when the number of historical frame image information exceeds a preset number.

[0058] Furthermore, at step S305, the fusion result is decoded using a decoder to obtain a target mask where the sample stage is placed in the current frame. In addition, in order to eliminate the discrete and hole problems in the mask result, at step S306, the connected domains in the target mask whose contour area is less than the second preset threshold are deleted, and at step S307, the convex hull filling of the outer contour points of the remaining connected domains is performed to obtain the final target mask, so as to obtain a more accurate target mask.

[0059] Figure 4 FIG. 1 is an exemplary flow chart showing the overall process of collision monitoring in a scanning electron microscope according to an embodiment of the present application. Figure 4 As shown in , at step S401, a reference frame image corresponding to a sample stage not placed in a sample chamber of a scanning electron microscope and an image data stream corresponding to a sample stage placed are obtained. At step S402, a noise reduction operation and / or a normalization operation are performed on the reference frame image and the initial frame image in the image data stream, and then at step S403, an inter-frame difference is performed on the reference frame image and the initial frame image in the image data stream to extract an initial mask for placing the sample stage. For more details on extracting the initial mask for placing the sample stage, please refer to the above Figure 1 , Figure 2 The relevant description of this application will not be repeated here.

[0060] Further, at step S404, the remaining image frames in the image data stream corresponding to the sample stage are obtained, and at step S405, the initial mask and the remaining image frames in the image data stream corresponding to the sample stage are input into the collision monitoring model to extract the target mask of the sample stage in the current frame. For more details on extracting the target mask of the sample stage in the current frame, please refer to the above Figure 1 , Figure 3 The relevant description of this application will not be repeated here.

[0061] Based on the extracted target mask, at step S406, the minimum ordinate of the target motion pixel in the target mask can be obtained: and the target ordinate At step S407, it is determined that the distance between the minimum ordinate of the target moving pixel and the target ordinate of the pole shoe is greater than the distance threshold, that is, it is determined that Is it established? If yes, then at step S408, it is determined that there is no collision risk. If not, in step S409, it is determined that there is a collision risk and a warning prompt is issued.

[0062] Figure 5 is an exemplary schematic diagram showing collision monitoring in a scanning electron microscope according to an embodiment of the present application. Figure 5 As shown in , a reference frame image 501 corresponding to a sample stage not being placed and an initial frame image 502 in an image data stream corresponding to a sample stage being placed are obtained. In some implementation scenarios, a noise reduction operation and / or a normalization operation may be first performed on the reference frame image and the initial frame image in the image data stream, and then an inter-frame difference may be performed on the reference frame image and the initial frame image in the image data stream, and a motion pixel mask 503 may be extracted by performing, for example, a binary threshold processing. Further, all independent connected domains in the aforementioned motion pixel mask are obtained, and the contour area of ​​each connected domain is calculated, and the contours of the connected domains whose contour area is greater than a first preset threshold are retained 504. Further, convex hull processing is performed on all the contour points obtained by screening to obtain the convex hull area of ​​the motion area mask as the initial mask 505.

[0063] In addition, the remaining image frames 506 in the image data stream are obtained and input into the SAM2 model 507 together with the initial mask 505. The SAM2 model 507 may include an encoder 507-1, a fusion module 507-2, and a decoder 507-3. In some implementation scenarios, the encoder 507-1 is used to extract the mask features of the current frame image, and the fusion module 507-2 uses, for example, an attention mechanism to fuse it with the historical image information 508 to obtain a fusion result. The decoder 507-3 decodes the above fusion result to extract the target mask 509 where the sample stage is placed in the current frame.

[0064] In some embodiments, in order to avoid discrete points or holes in the target mask, connected domains with contour areas less than a second preset threshold value can be deleted from the target mask, and the convex hull filling of the outer contour points of the remaining connected domains is performed to obtain the final target mask. Finally, collision monitoring is achieved by determining whether the distance between the minimum ordinate of the target motion pixel and the target ordinate of the pole shoe (for example, the distance between the two dotted lines shown in the rightmost figure) is greater than the distance threshold. For example, when the distance between the minimum ordinate of the target motion pixel and the position of the pole shoe is greater than the distance threshold, it is determined that there is a collision risk to initiate a collision warning. Otherwise, there is no collision risk.

[0065] Figure 6 FIG. 6 is an exemplary structural block diagram showing a device 600 for collision monitoring in a scanning electron microscope according to an embodiment of the present application. Figure 6 As shown in , the device 600 of the present application may include a processor 601 and a memory 602, wherein the processor 601 and the memory 602 communicate with each other via a bus. The memory 602 stores program instructions for collision monitoring in a scanning electron microscope. When the program instructions are executed by the processor 601, the method steps described in the above text in combination with the accompanying drawings are implemented: collecting a reference frame image corresponding to a sample stage not placed in a sample compartment of a scanning electron microscope and an image data stream corresponding to a sample stage placed; performing inter-frame difference between the reference frame image and the initial frame image in the image data stream to extract an initial mask for placing the sample stage; based on the initial mask and the historical frame image information in the image data stream, using a collision monitoring model to extract a target mask for placing the sample stage in the current frame; and performing collision monitoring in a scanning electron microscope according to the distance between the target mask and the target pole shoe.

[0066] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by a software program. Therefore, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores computer-readable instructions for collision monitoring in a scanning electron microscope. When the computer-readable instructions are executed by one or more processors, the present application in combination with the accompanying drawings can be implemented. Figure 1 A method for collision monitoring in a scanning electron microscope is described.

[0067] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0068] It should be noted that although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0069] It should be understood that when the terms "first", "second", "third" and "fourth" are used in the claims, the specification and the drawings of the present application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0070] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0071] Although the implementation methods of the present application are as above, the contents described are only examples adopted to facilitate the understanding of the present application, and are not intended to limit the scope and application scenarios of the present application. Any technician in the technical field described in the present application can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present application, but the scope of patent protection of the present application shall still be subject to the scope defined in the attached claims.

Claims

1. A method for collision monitoring in a scanning electron microscope, characterized in that include: Collecting a reference frame image corresponding to a sample stage not being placed in a sample chamber of a scanning electron microscope and an image data stream corresponding to a sample stage being placed; Performing inter-frame difference between the reference frame image and the initial frame image in the image data stream to extract an initial mask for placing the sample stage; Based on the initial mask and the historical frame image information in the image data stream, a collision monitoring model is used to extract a target mask for placing the sample stage in the current frame; as well as, performing collision monitoring in a scanning electron microscope based on the distance between the target mask and the target pole piece, The step of performing inter-frame difference between the reference frame image and the initial frame image in the image data stream to extract an initial mask for placing the sample stage comprises: Performing inter-frame difference between the reference frame image and the initial frame image in the image data stream to determine a motion pixel mask; determining a target connected domain based on the motion pixel mask; and, Calculating the convex hull of the outer contour points according to the contour of the target connected domain, and filling the convex hull to extract the initial mask for placing the sample stage; The collision monitoring model includes an encoder, a fusion module and a decoder, and based on the initial mask and the historical frame image information in the image data stream, extracting the target mask for placing the sample stage in the current frame using the collision monitoring model includes: Using the initial mask as prompt information, using the encoder to extract mask features of the current frame image; Using the fusion module to fuse the mask feature and the historical frame image information to obtain a fusion result; and, Based on the fusion result, the decoder is used to perform a decoding operation to extract the target mask where the sample stage is placed in the current frame.

2. The method according to claim 1, characterized in that Before performing inter-frame difference between the reference frame image and the initial frame image in the image data stream, the method further includes: A noise reduction operation and / or a standardization operation is performed on the reference frame image and the initial frame image in the image data stream.

3. The method according to claim 1, characterized in that Wherein determining the target connected domain based on the motion pixel mask comprises: Obtaining an initial connected domain in the motion pixel mask; Calculating the contour area of ​​each of the initial connected domains; and, An initial connected domain whose contour area is greater than a first preset threshold is determined as the target connected domain.

4. The method according to claim 1, characterized in that Also includes: An image scaling operation and / or a normalization operation is performed on the initial mask and the current frame image.

5. The method according to claim 1, characterized in that Also includes: Determining whether the amount of the historical frame image information exceeds a preset amount; as well as, In response to the amount of the historical frame image information exceeding a preset amount, target historical frame image information is deleted from the historical frame image information.

6. The method according to claim 1, characterized in that Also includes: In response to the target mask containing different connected domains, deleting the connected domains whose contour area is smaller than a second preset threshold; as well as, The convex hull of the outer contour points of the remaining connected domain is filled to obtain the final target mask.

7. The method according to claim 1, characterized in that The method of performing collision monitoring in a scanning electron microscope according to the distance between the target mask and the target pole shoe comprises: Obtaining the minimum ordinate of the target motion pixel in the target mask and the target ordinate of the target pole shoe; and, In response to a distance between the minimum ordinate and the target ordinate being greater than a distance threshold, it is determined that there is a collision risk in the scanning electron microscope.

8. A device for collision monitoring in a scanning electron microscope, characterized in that include: processor; as well as, A memory, wherein program instructions for collision monitoring in a scanning electron microscope are stored, and when the program instructions are executed by the processor, the device implements the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: Computer-readable instructions for collision monitoring in a scanning electron microscope are stored thereon, and when the computer-readable instructions are executed by one or more processors, the method according to any one of claims 1 to 7 is implemented.

Images