Electron microscope sample stage three-dimensional navigation method based on topography reconstruction, storage medium and system

Abstract

The present application relates to a method, medium and system for three-dimensional navigation of an electron microscope sample stage based on topography reconstruction, the method comprising the steps of: driving an electron microscope sample stage loaded with a sample to the center of the sample chamber camera shot; collecting multiple sets of electron microscope sample stage images at different angles; performing image preprocessing on the collected multi-angle image data; extracting sample topography data from the preprocessed images; combining the sample topography data to stitch and reconstruct the three-dimensional topography of the sample, construct and present an AR scene of the sample chamber; obtaining a sample observation point selected by a user in the AR scene, calculating a path in real time, driving the electron microscope sample stage to move to the sample observation point for electron microscope image observation, and controlling the AR scene to update in real time. The present application combines system control, augmented reality, image processing, three-dimensional topography reconstruction and other methods to enhance the sample chamber scene, develop a three-dimensional navigation method and system for the sample stage, and realize real-time five-axis monitoring control and automatic rapid navigation of the scanning electron microscope sample stage under the condition of ensuring safety.

Description

Technical Field

[0001] This invention relates to the field of scanning electron microscopy, and in particular to a three-dimensional navigation method, storage medium, and system for an electron microscope sample stage based on topography reconstruction. Background Technology

[0002] The sample stage, an important component of a scanning electron microscope, is a device for sample support and precise five-axis positioning located inside the vacuum sample chamber.

[0003] Current electron microscopes cannot achieve real-time five-axis monitoring and navigation of the sample stage. Conventional sample stage monitoring uses a camera through the sample chamber window, allowing for real-time monitoring from a single viewpoint, while the sample stage is manually controlled. Pre-imported images of the sample stage plan are used to simply mark the observation points. Figure 1 As shown in the image. This method is not very intuitive, is time-consuming and laborious to operate, has a large margin of error, and is prone to safety issues such as collisions. Furthermore, it cannot be automated. Summary of the Invention

[0004] To achieve the above-mentioned objectives and other advantages of the present invention, a first objective of the present invention is to provide a three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction, comprising the following steps:

[0005] Drive the electron microscope sample stage loaded with the sample to the center of the camera's field of view in the sample chamber;

[0006] Multiple sets of electron microscope sample stage images were acquired from different angles;

[0007] Image preprocessing is performed on the acquired multi-angle image data;

[0008] Extract sample morphology data from the preprocessed image;

[0009] The current position parameters of the electron microscope sample stage and system parameters are obtained, and combined with the sample morphology data, the three-dimensional morphology of the sample is stitched together and reconstructed. Coordinate alignment calculation is performed to construct and present an AR scene of the sample chamber.

[0010] The system acquires the sample observation point in the AR scene selected by the user, calculates the path in real time based on the position of the sample observation point and the position of the electron microscope sample stage, drives the electron microscope sample stage to move to the sample observation point for electron microscope image observation, and controls the AR scene to update in real time according to the sample stage position parameters, synchronizing with the actual sample stage movement.

[0011] Furthermore, acquiring multiple sets of electron microscope sample stage images at different angles includes the following steps:

[0012] The sampling angle interval is determined by the sample morphology reproduction accuracy.

[0013] The electron microscope sample stage is driven to rotate, and during the rotation, multiple sets of electron microscope sample stage images at different angles are acquired at the specified acquisition angle intervals.

[0014] Furthermore, the image preprocessing of the acquired multi-angle image data includes using a bilateral filtering algorithm to remove redundant noise points in the image while preserving the image edge information.

[0015] Furthermore, the extraction of sample morphology data from the preprocessed image includes the following steps:

[0016] Background subtraction and inter-frame differencing are used to process the denoised image;

[0017] The image is processed using grayscale similarity and structural similarity discriminant functions and morphological processing algorithms to obtain the sample stage contour image;

[0018] Extract the sample and sample stage topography contour lines from the sample stage contour image;

[0019] The sample morphology data is calculated by combining the outline with the actual size data of the electron microscope sample stage.

[0020] Furthermore, the process of processing the denoised image using background subtraction and inter-frame differencing includes the following steps:

[0021] Perform a difference operation between the current frame image and the two frames before and after it to obtain three difference images, and calculate the mean image of the three difference images;

[0022] The mean image is thresholded to obtain a first binarized image;

[0023] The process of processing the image using grayscale similarity and structural similarity discriminant functions and morphological processing algorithms to obtain the sample stage contour image includes the following steps:

[0024] The current frame image and the background frame image are compared using grayscale similarity. The formula for calculating grayscale similarity is:

[0025]

[0026] Where X(i,j) and Y(i,j) are the grayscale values ​​of the corresponding pixels in the current frame and the background frame, respectively;

[0027] Thresholding is applied to the calculated gray-level similarity. If the gray-level similarity of the current pixel is greater than 1, the pixel value is retained in the second binarized image. If the gray-level similarity of the current pixel is not greater than 1, the pixel value is discarded.

[0028] The structural similarity between the current frame image and the background frame image is calculated using a template image of a preset size, centered on the measured pixel. The formula for calculating structural similarity is:

[0029]

[0030] Where, μ x and μ y σ represents the grayscale mean of the two template images, used as a brightness estimate; x and σ y σ is the standard deviation, used as an estimate of contrast. xy The covariance value of the two template images serves as a measure of structural similarity.

[0031] If the structural similarity is less than 0.5, the pixel value is retained in the third binarized image; if the structural similarity is not less than 0.5, the pixel value is discarded.

[0032] After performing an OR operation between the first binarized image and the second binarized image, and then performing an AND operation between the first binarized image and the third binarized image, the output image is processed through morphological processing to enhance connectivity, thus obtaining the sample stage contour image.

[0033] Further, the step of thresholding the mean image to obtain the first binarized image includes the following steps:

[0034] The image is divided into foreground and background based on its grayscale characteristics using the maximum inter-class variance method. The grayscale value that maximizes the inter-class variance between the foreground and background is then used as the threshold output.

[0035] The extraction of sample morphology data from the preprocessed image further includes a step of fitting and smoothing the sample morphology, which includes:

[0036] When the spikes on the contour line reach a preset value, a smooth shape envelope is fitted using multi-segment template lines:

[0037] The outline is broken down into multiple segments, and each segment is structurally matched with the line segment data in the template library. If the match is successful, the segment data is replaced with the template line segment data to obtain a fitted smooth envelope.

[0038] Furthermore, the construction and presentation of the sample room AR scene includes the following steps:

[0039] The 3D model is rendered based on the dimensions of the actual sample chamber and electron microscope stage, and its relative position in the 3D model is changed according to the current position parameters of the electron microscope stage.

[0040] A cube 3D model is pre-rendered above the sample stage, and the cube 3D model is cut 360 degrees according to the sample morphology data to obtain the current sample 3D model.

[0041] When the five-axis parameters of the sample stage change, the current sample 3D model calculates and changes its position in the AR scene in real time based on the five-axis parameters to achieve coordinate alignment.

[0042] Furthermore, the step of acquiring the sample observation point in the AR scene selected by the user, calculating the path in real time based on the position of the sample observation point and the position of the electron microscope sample stage, and driving the electron microscope sample stage to move to the sample observation point for electron microscope image observation includes the following steps:

[0043] Calculate the five-axis motion distance based on the current sample stage coordinates and the target position;

[0044] Based on the sample morphology data and the current five-axis motion speed, the sample coverage space range is calculated and determined when the sample stage moves simultaneously along all five axes.

[0045] If the spatial data of other devices in the sample chamber do not overlap with this range, then drive the five axes to move simultaneously. If the spatial data of other devices in the sample chamber overlap with this range, then translate this range until it no longer overlaps, and record the current translation direction and distance.

[0046] The sample stage is first driven to perform a translation operation, then the five axes are driven to move simultaneously, and finally the reverse translation is performed to return to the target point.

[0047] A second objective of this invention is to provide a computer-readable storage medium having program instructions stored thereon, which, when executed, implement a three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction.

[0048] A third objective of this invention is to provide a 3D navigation system for an electron microscope sample stage based on topography reconstruction, comprising a sample chamber camera, control system software, and an electron microscope sample stage; wherein, the sample chamber camera acquires data of the sample chamber scene, and the control system software processes and executes the 3D navigation method for the electron microscope sample stage based on topography reconstruction, performs coordinate alignment and virtual-real scene fusion on the acquired image data, presents a synthesized real-time 3D AR scene, and provides human-computer interaction control of the electron microscope sample stage.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] This invention combines multiple methods such as system control, augmented reality, image processing, and 3D topography reconstruction to enhance the sample chamber scene, and develops a 3D navigation method and system for the sample stage, enabling real-time five-axis monitoring and control and automatic rapid navigation of the scanning electron microscope sample stage while ensuring safety.

[0051] Compared to existing electron microscope stage control methods, this system's stage navigation method is simpler, more intuitive, and safer, enabling automatic stage movement control. The system itself provides three-view images of the scene, such as... Figure 5 As shown. Figure 6 As shown, users can also zoom in and out and change the viewing angle of the 3D AR scene using the mouse wheel or dragging, no longer limited by the fixed position of the camera in the sample chamber, thus avoiding a single viewing angle; for example Figure 7 As shown, the system can also calculate and display the real five-axis motion status of the sample stage in real time, allowing users to view the motion of the sample stage and the current observation point position more intuitively and in detail.

[0052] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0053] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0054] Figure 1 A schematic diagram showing the locations of existing observation points;

[0055] Figure 2 This is a schematic diagram of the three-dimensional navigation system for an electron microscope sample stage based on topography reconstruction, as described in Example 1.

[0056] Figure 3 Here is a flowchart of the three-dimensional navigation method for electron microscope sample stage based on topography reconstruction in Example 2;

[0057] Figure 4 This is a flowchart of the steps for extracting sample morphology data from the preprocessed image in Example 2;

[0058] Figure 5 A three-view diagram of an AR scene;

[0059] Figure 6 A schematic diagram illustrating zooming in / out and arbitrary viewpoint transformation of a 3D AR scene;

[0060] Figure 7 This is a schematic diagram of the five-axis motion state of a real sample stage;

[0061] Figure 8 This is a schematic diagram of the electronic device in Example 3;

[0062] Figure 9This is a schematic diagram of a computer-readable storage medium according to Example 4. Detailed Implementation

[0063] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0064] In practical applications, it is often necessary to set or modify the internal parameters of infrared sensors based on local geographical environment, customer needs and other information. This invention provides a three-dimensional navigation method for electron microscope sample stages based on topography reconstruction, which does not increase the cost of infrared sensors and does not cause waste of resources.

[0065] Example 1

[0066] A three-dimensional navigation system for electron microscope sample stages based on topography reconstruction, such as Figure 2 As shown, the system includes a sample chamber camera, control system software, and an electron microscope sample stage. The sample chamber camera acquires data about the sample chamber scene, and the control system software processes and executes a 3D navigation method for the electron microscope sample stage based on topography reconstruction. The acquired image data is then aligned to coordinates and fused with the virtual-real scene to present a synthesized real-time 3D AR scene and provide human-computer interaction control of the electron microscope sample stage. For a detailed description of the method, please refer to the corresponding description in the following method embodiments.

[0067] Example 2

[0068] A three-dimensional navigation method for electron microscope sample stages based on topography reconstruction, such as Figure 3 As shown, it includes the following steps:

[0069] Load the sample onto the electron microscope stage;

[0070] The electron microscope sample stage, loaded with the sample, is driven by the control system software to the center of the camera's image in the sample chamber.

[0071] The acquisition angle interval is determined by the sample morphology reproduction degree, such as determining to acquire one image every 5 degrees or every 10 degrees.

[0072] The electron microscope stage is driven to rotate 360 ​​degrees, and multiple sets of electron microscope stage images at different angles are acquired at intervals during the rotation.

[0073] Image preprocessing is performed on the acquired multi-angle image data. Specifically, a bilateral filtering algorithm is used to remove redundant noise points while preserving image edge information. Bilateral filtering can achieve the effects of edge preservation and noise reduction smoothing. Bilateral filtering uses a weighted averaging method, using the weighted average of the brightness values ​​of surrounding pixels to represent the intensity of a pixel. The weighted average used is based on a Gaussian distribution. Most importantly, the weights of bilateral filtering not only consider the Euclidean distance between pixels (such as ordinary Gaussian low-pass filtering, which only considers the influence of position on the center pixel), but also the radiative differences in the pixel range domain (such as the similarity between pixels in the convolution kernel and the center pixel, color intensity, depth distance, etc.).

[0074] Extract sample morphology data from the denoised image.

[0075] In a stable surveillance scene, with no moving targets and unchanged lighting, the grayscale values ​​of each pixel in the video image conform to a random probability distribution. However, due to the inevitable introduction of noise during image acquisition, these grayscale values ​​oscillate randomly within a certain range around a certain mean, forming what is known as the "background." Background subtraction is similar in concept to inter-frame differencing, both using difference operations between different images to extract the target region. However, unlike inter-frame differencing, background subtraction does not subtract the current frame from adjacent frames; instead, it subtracts the current frame from a continuously updated background model, extracting moving targets from the resulting difference image.

[0076] Video sequences captured by cameras are continuous. If there are no moving targets in the scene, the changes between consecutive frames are very subtle; however, if there are moving targets, there will be significant changes between consecutive frames. Temporal difference (TPD) borrows this idea. Because targets in the scene are moving, their images are positioned differently in different image frames. TPD performs a difference operation on two or three temporally consecutive images, subtracting the corresponding pixels from each frame and determining the absolute value of the grayscale difference. When the absolute value exceeds a certain threshold, the target can be identified as a moving target, thus achieving target detection.

[0077] In this embodiment, background subtraction and inter-frame difference methods are used to process the denoised image.

[0078] By enhancing connectivity through grayscale similarity and structural similarity discriminant functions and morphological processing algorithms (erosion, dilation, etc.), a binarized image after threshold segmentation is generated, which shows the overall outline of the sample morphology above the electron microscope sample stage during its movement.

[0079] Extract the sample and sample stage topography contour lines from the sample stage contour image;

[0080] The sample morphology data is calculated by combining the contour lines with the actual size data of the electron microscope sample stage.

[0081] Specifically, such as Figure 4 As shown, it includes the following steps:

[0082] Perform a difference operation between the current frame image and the two frames before and after it to obtain three difference images. Calculate the mean image of the three difference images by adding the three difference images together and dividing by 3.

[0083] The mean image is segmented by thresholding to obtain the first binarized image. The segmentation threshold is calculated using the maximum inter-class variance method (i.e., Otsu thresholding). This method divides the image into foreground and background parts according to the image grayscale characteristics, and calculates the grayscale value that maximizes the inter-class variance between the foreground and background as the threshold output.

[0084] The current frame image and the background frame image are compared using grayscale similarity. The background frame image is the background image of the empty sample stage. The grayscale similarity calculation formula is as follows:

[0085]

[0086] Where X(i,j) and Y(i,j) are the grayscale values ​​of the corresponding pixels in the current frame and the background frame, respectively;

[0087] Threshold judgment is performed on the calculated gray-level similarity. When the gray-level similarity of the current pixel is greater than 1, it indicates that the gray-level difference between the current frame and the background frame at this pixel is large, that is, the pixel belongs to the sample stage area of ​​the current frame. The pixel value is retained in the second binarized image. When the gray-level similarity of the current pixel is not greater than 1, the pixel value is discarded.

[0088] The structural similarity between the current frame image and the background frame image is calculated using a template image of a preset size (e.g., 5*5) centered on the measured pixel. This reduces the likelihood of misclassifying the target as background due to similar pixel values. The structural similarity calculation formula is as follows:

[0089]

[0090] Where, μ x and μ y σ represents the grayscale mean of the two template images, used as a brightness estimate; x and σ y σ is the standard deviation, used as an estimate of contrast. xy The covariance value of the two template images serves as a measure of structural similarity.

[0091] If the structural similarity is less than 0.5, it indicates that the two template images have a large structural difference, and the pixel value is retained in the third binarized image. If the structural similarity is not less than 0.5, the pixel value is discarded.

[0092] After performing an OR operation between the first and second binarized images, and then performing an AND operation between the first and second binarized images and the third binarized image, the output image is subjected to morphological processing (erosion, dilation) to enhance connectivity, resulting in a more complete sample stage contour image.

[0093] After obtaining the contour image, the contour lines of the sample and the sample stage can be extracted. The sample shape data (i.e., the spatial coordinate data of the sample shape relative to the sample stage plane at the current angle) can be calculated by combining the image contour line data with the actual size data of the sample stage.

[0094] Extracting sample morphology data from the preprocessed image also includes a step of fitting and smoothing the sample morphology. This step includes:

[0095] When the sharp points of the outline reach a preset value and affect the appearance, a smooth shape envelope is fitted by matching multiple template lines. Here, a multi-threaded parallel processing method is used to improve the matching efficiency.

[0096] The outline is broken down into multiple segments, and each segment is structurally matched with the line segment (straight line, polyline, wavy line, triangle, etc.) data in the template library. If the match is successful, the segment data is replaced with the template line segment data to obtain the fitted smooth envelope.

[0097] The current position parameters of the electron microscope sample stage and system parameters are obtained, and combined with the sample morphology data, the three-dimensional morphology of the sample is reconstructed by stitching together, coordinate alignment calculations are performed, and an AR scene of the sample chamber is constructed and presented. In this embodiment, the system parameters include five-axis boundary values, namely the current X, Y, Z, R, and T axis coordinates of the sample stage, the maximum coordinate values ​​of the five-axis motion, and the position and size parameters of other components in the sample chamber. Specifically, the following steps are included:

[0098] The 3D model is rendered based on the dimensions of the actual sample chamber and electron microscope stage, and its relative position in the 3D model is changed according to the current position parameters of the electron microscope stage.

[0099] A cube 3D model is pre-rendered above the sample stage, and the cube 3D model is cut 360 degrees according to the sample topography data, that is, the height part that exceeds the relative topography data is discarded and re-rendered. After completion, the current sample 3D model is obtained.

[0100] When the five-axis parameters of the sample stage change, the current sample 3D model calculates and changes its position in the AR scene in real time based on the five-axis parameters to achieve coordinate alignment.

[0101] The system acquires the sample observation point selected by the user in the AR scene. Based on the location of the sample observation point and the position of the electron microscope sample stage, it calculates the optimal and safest navigation path in real time, ensuring that the sample does not collide with other components in the sample chamber during movement, while the sample stage moves to the target point in the shortest possible time. It then drives the electron microscope sample stage to the sample observation point for electron microscopy image observation, controlling the AR scene to update in real time according to the sample stage position parameters, synchronizing with the actual movement of the sample stage. Specifically, it includes the following steps:

[0102] Calculate the five-axis motion distance based on the current sample stage coordinate position Point1 and the target position Point2;

[0103] By combining the sample morphology data and the current five-axis motion speed, the sample coverage space range when the sample stage moves simultaneously along all five axes is calculated and determined.

[0104] If the spatial data of other devices in the sample chamber do not overlap with this range, then drive the five axes to move simultaneously. If the spatial data of other devices in the sample chamber overlap with this range, then translate this range until it no longer overlaps, and record the current translation direction and distance.

[0105] The sample stage is first driven to perform a translation operation, then the five axes are driven to move simultaneously, and finally the reverse translation is performed to return to the target point.

[0106] Example 3

[0107] An electronic device 200, such as Figure 8 As shown, the method includes, but is not limited to: a memory 201 storing program code; and a processor 202 connected to the memory, which, when the program code is executed by the processor, implements a three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction. For a detailed description of the method, please refer to the corresponding description in the above method embodiments, which will not be repeated here.

[0108] Example 4

[0109] A computer-readable storage medium, such as Figure 9 As shown, it stores program instructions, which, when executed, implement a three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction. For a detailed description of the method, please refer to the corresponding description in the above method embodiments; it will not be repeated here.

[0110] Example 5

[0111] A computer program product includes a computer program / instructions that, when executed by a processor, implement a three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction. For a detailed description of the method, please refer to the corresponding description in the above method embodiments, which will not be repeated here.

[0112] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0113] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0114] The above are merely embodiments of this specification and are not intended to limit the scope of the one or more embodiments herein. For those skilled in the art, various modifications and variations can be made to the one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the one or more embodiments of this specification should be included within the scope of the claims of the one or more embodiments of this specification.

Claims

1. A three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction, characterized in that, Includes the following steps: Drive the electron microscope sample stage loaded with the sample to the center of the camera's field of view in the sample chamber; Multiple sets of electron microscope sample stage images were acquired from different angles; Image preprocessing is performed on the acquired multi-angle image data; Extract sample morphology data from the preprocessed image; The current position parameters of the electron microscope sample stage and system parameters are obtained, and combined with the sample morphology data, the three-dimensional morphology of the sample is stitched together and reconstructed. Coordinate alignment calculation is performed to construct and present an AR scene of the sample chamber. The system acquires the sample observation point in the AR scene selected by the user, calculates the path in real time based on the position of the sample observation point and the position of the electron microscope sample stage, drives the electron microscope sample stage to move to the sample observation point for electron microscope image observation, and controls the AR scene to update in real time according to the sample stage position parameters, synchronizing with the actual sample stage movement. The extraction of sample morphology data from the preprocessed image includes the following steps: Background subtraction and inter-frame differencing are used to process the denoised image; The image is processed using grayscale similarity and structural similarity discriminant functions and morphological processing algorithms to obtain the sample stage contour image; Extract the sample and sample stage topography contour lines from the sample stage contour image; The sample morphology data is calculated by combining the outline with the actual size data of the electron microscope sample stage.

2. The three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction according to claim 1, characterized in that, The acquisition of multiple sets of electron microscope sample stage images at different angles includes the following steps: The sampling angle interval is determined by the sample morphology reproduction accuracy. The electron microscope sample stage is driven to rotate, and during the rotation, multiple sets of electron microscope sample stage images at different angles are acquired at the specified acquisition angle intervals.

3. The three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction according to claim 1, characterized in that, The image preprocessing of the acquired multi-angle image data includes using a bilateral filtering algorithm to remove redundant noise points in the image while preserving the image edge information.

4. The three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction according to claim 1, characterized in that, The process of processing the denoised image using background subtraction and inter-frame differencing includes the following steps: Perform a difference operation between the current frame image and the two frames before and after it to obtain three difference images, and calculate the mean image of the three difference images; The mean image is thresholded to obtain a first binarized image; The process of processing the image using grayscale similarity and structural similarity discriminant functions and morphological processing algorithms to obtain the sample stage contour image includes the following steps: The current frame image and the background frame image are compared using grayscale similarity. The formula for calculating grayscale similarity is: in, and These are the grayscale values ​​of the corresponding pixels in the current frame and the background frame, respectively. Thresholding is applied to the calculated grayscale similarity. If the grayscale similarity of the current pixel is greater than 1, the grayscale value of the pixel is retained in the second binarized image. If the grayscale similarity of the current pixel is not greater than 1, the grayscale value of the pixel is discarded. The structural similarity between the current frame image and the background frame image is calculated using a template image of a preset size, centered on the measured pixel. The formula for calculating structural similarity is: in, and The grayscale mean values ​​of the two template images are used as brightness estimates. and The standard deviation is used as a contrast estimate. The covariance value of the two template images serves as a measure of structural similarity. If the structural similarity is less than 0.5, the gray value of the pixel is retained in the third binarized image; if the structural similarity is not less than 0.5, the gray value of the pixel is discarded. After performing an OR operation between the first binarized image and the second binarized image, and then performing an AND operation between the first binarized image and the third binarized image, the output image is processed through morphological processing to enhance connectivity, thus obtaining the sample stage contour image.

5. The three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction according to claim 4, characterized in that, The step of thresholding the mean image to obtain the first binarized image includes the following steps: The image is divided into foreground and background based on its grayscale characteristics using the maximum inter-class variance method. The grayscale value that maximizes the inter-class variance between the foreground and background is then used as the threshold output. The extraction of sample morphology data from the preprocessed image further includes a step of fitting and smoothing the sample morphology, which includes: When the spikes on the contour line reach a preset value, a smooth shape envelope is fitted using multi-segment template lines: The outline is broken down into multiple segments, and each segment is structurally matched with the line segment data in the template library. If the match is successful, the segment data is replaced with the template line segment data to obtain a fitted smooth envelope.

6. The three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction according to claim 1, characterized in that, The construction and presentation of the sample chamber AR scene includes the following steps: The 3D model is rendered based on the dimensions of the actual sample chamber and electron microscope stage, and its relative position in the 3D model is changed according to the current position parameters of the electron microscope stage. A cube 3D model is pre-rendered above the sample stage, and the cube 3D model is cut 360 degrees according to the sample morphology data to obtain the current sample 3D model. When the five-axis parameters of the sample stage change, the current sample 3D model calculates and changes its position in the AR scene in real time based on the five-axis parameters to achieve coordinate alignment.

7. The three-dimensional navigation method for an electron microscope sample stage based on topography reconstruction according to claim 1, characterized in that, The process of acquiring the sample observation point in the AR scene selected by the user, calculating the path in real time based on the position of the sample observation point and the position of the electron microscope sample stage, and driving the electron microscope sample stage to move to the sample observation point for electron microscope image observation includes the following steps: Calculate the five-axis motion distance based on the current sample stage coordinates and the target position; Based on the sample morphology data and the current five-axis motion speed, the sample coverage space range is calculated and determined when the sample stage moves simultaneously along all five axes. If the spatial data of other devices in the sample chamber do not overlap with this range, then drive the five axes to move simultaneously. If the spatial data of other devices in the sample chamber overlap with this range, then translate this range until it no longer overlaps, and record the current translation direction and distance. The sample stage is first driven to perform a translation operation, then the five axes are driven to move simultaneously, and finally the reverse translation is performed to return to the target point.

8. A computer-readable storage medium, characterized in that, It stores program instructions that, when executed, implement the method as described in any one of claims 1 to 7.

9. A three-dimensional navigation system for an electron microscope sample stage based on topography reconstruction, characterized in that: The system includes a sample chamber camera, control system software, and an electron microscope sample stage. The sample chamber camera acquires data of the sample chamber scene, and the control system software processes and executes the 3D navigation method for the electron microscope sample stage based on topography reconstruction as described in any one of claims 1 to 7. The acquired image data is then aligned with coordinates and fused with virtual and real scenes to present a synthesized real-time 3D AR scene and provide human-computer interaction control of the electron microscope sample stage.

Images