Electron microscope imaging system and imaging method

Abstract

Electron microscope imaging system and imaging method. The present invention provides systems and methods for electron microscope imaging, including methods of imaging a sample with a charged particle microscope, the method comprising: acquiring a first initial image of a surface of the sample with the charged particle microscope; during imaging with the charged particle microscope, receiving automatic function instructions for one or more initial regions of interest; during imaging with the charged particle microscope, acquiring a series of subsequent images of subsequent surfaces of the sample. Wherein, for each subsequent image, acquiring the subsequent image comprises exposing a respective subsequent surface of the sample; identifying one or more regions of interest corresponding to the one or more initial regions of interest; imaging the respective subsequent surface, wherein imaging the respective subsequent surface comprises applying one or more automatic functions based on at least one identified region of interest corresponding to the one or more initial regions of interest.

Description

Technical Field

[0001] This invention relates to a method for imaging regions of interest in a sample using a charged particle microscope. Specifically, it mainly describes a method and system for automatically setting or adjusting relevant parameters during the identification of regions of interest in a sample slice. Technical Background

[0002] As is well known, charged particle microscopy is a microscopic imaging technique. Charged particle microscopy uses a beam of charged particles to illuminate a sample for imaging. Generally, charged particle microscopy primarily utilizes charged particles for electron imaging, offering higher image resolution compared to visible light microscopy. Charged particle microscopy equipment includes scanning electron microscopy (SEM), focused ion beam scanning electron microscopy (FIB-SEM), transmission electron microscopy (TEM), and scanning transmission electron microscopy (STEM), among others.

[0003] To acquire high-resolution images of a given sample, various parameters of the charged particle microscope need to be set or adjusted. These parameters include objective calibration, conicity height, focusing, astigmatism, centering and focusing, beam alignment, contrast, and brightness. A new function has been developed that can automatically adjust these parameters; this function will be referred to as the "automatic function" below. This function typically analyzes only a local area of ​​the sample surface (or a local area of ​​the sample image) to determine one or more applicable parameter values. The automatic function (AF) includes, but is not limited to, objective calibration, astigmatism correction, focusing, contrast / brightness, centering and focusing, and / or astigmatism correction and centering. Among these functions, autofocus is particularly important for charged particle microscopy and unattended imaging. One method of autofocus is to use a laser to measure the distance to the sample surface. The focusing parameters are set using the offset between the sample surface and the actual focal point. Another method of autofocus is image-based autofocus. The core of image-based autofocus is to detect the image sharpness at different focus positions within a desired range and maximize it. Since the grayscale difference between adjacent pixels naturally increases as the image focus is adjusted, the optical system can be adjusted accordingly until a sharpness peak is detected.

[0004] It should be noted that various types of samples, such as biological samples (e.g., frozen cells on a grid, autoclaved samples, resin-embedded samples, etc.), semiconductor samples, material samples, or any other samples used for charged particle microscopy, may be sensitive to radiation damage. This damage can reduce the sharpness of the sample image. In some cases, the sample may develop a charging effect during electron microscopy imaging, affecting image quality during long-term data acquisition. Not all parts of the sample are susceptible to radiation damage and charging effects. However, even localized sensitivity can reduce image quality during long-term data acquisition. To avoid this, users typically manually adjust the charged particle microscope and execute automatic functions in sample areas outside the desired imaging region to prevent radiation damage. However, manually adjusting the charged particle microscope and executing automatic functions in sample areas outside the desired imaging region can be time-consuming when acquiring data from highly sensitive samples.

[0005] Furthermore, when imaging one or more regions of interest (e.g., specific cells, tissue regions, organelles, cell walls, etc.), sample studies can be performed via sections. This enables imaging of regions of interest (e.g., 3D structures). For samples with smooth and well-defined surfaces, the location of the automated function should be determined based on the geometry between the sample stage, the beam, and the sample surface. However, because target features may have irregular surfaces in three-dimensional space, these features may not appear in the same planar region across all consecutive sections of the sample. In such cases, conventional imaging using the automated function may be ineffective. Similarly, users can manually adjust the charged particle microscope used for section analysis, but this can be time-consuming. Summary of the Invention

[0006] This invention aims to solve the problems encountered in using automated imaging. Specifically, this invention utilizes machine learning and / or image processing techniques to select appropriate regions of interest (ROIs) for automatic function application. Therefore, while maintaining image quality, automatic functions can be applied to ROIs without human intervention. This invention offers significant advantages in multi-slice (or surface) imaging of samples with unknown internal geometry. In this invention, automatic functions are implemented throughout the entire data acquisition process without human intervention. It is important to emphasize that the terms "slice" and "surface" used in this invention have the same meaning and can be used interchangeably throughout the imaging process.

[0007] While the above discussion pertains to biological samples, it is equally applicable to any type of sample studied under a charged particle microscope. Therefore, the scope of this invention is not limited to data acquisition from biological samples, but also includes data acquisition from semiconductor samples and materials science samples.

[0008] For similar regions of interest imaged using automatic functions in electron microscopy, the methods and systems described in this invention can also be used for rapid tracking and localization.

[0009] In a first aspect, the present invention provides a method for imaging a sample using a charged particle microscope. The method includes: acquiring an initial image of a first surface of the sample using a charged particle microscope; receiving instructions to perform one or more automatic functions on one or more initial regions of interest during charged particle microscope imaging; and acquiring a series of images of subsequent surfaces of the sample during charged particle microscope imaging. For each subsequent image, the subsequent image acquisition includes exposing the corresponding subsequent surface of the sample; identifying one or more regions of interest corresponding to the one or more initial regions of interest; and imaging the corresponding subsequent surface, wherein imaging the corresponding subsequent surface includes: applying the one or more automatic functions based on at least one identified region of interest corresponding to the one or more initial regions of interest. Receiving instructions for the initial regions includes (or equivalent to) a user-defined interpretation for labeling the initial regions of interest. The sample to be imaged can be a 2D or 3D sample. For example, a 2D or 3D biological sample prepared by cryogenic freezing, a 2D or 3D biological sample embedded in resin, a 2D or 3D materials science sample, or a 2D or 3D semiconductor sample. As used herein, 2D samples include samples with 2D properties, that is, samples that are not explicitly 2D but have certain 2D characteristics, such as wafers with resin-embedded sample sheets.

[0010] Subsequent surface exposure of the sample can be achieved using focused ion beam or electron beam milling, laser or diamond cutter methods.

[0011] Examples of applicable automatic functions may include one or more of the following: objective lens calibration function, astigmatism correction function, focusing function, contrast / brightness function, centering and focusing function, and astigmatism correction centering function.

[0012] In some embodiments, identifying one or more surface regions of interest corresponding to one or more initial regions of interest includes: identifying one or more surface regions of interest corresponding to one or more initial regions of interest in a previous image (or a previous surface image); and imaging each subsequent surface, wherein imaging each subsequent surface includes applying one or more automatic functions based on at least one identified region of interest identified in the previous image.

[0013] In some embodiments, identifying one or more surface regions of interest corresponding to one or more initial regions of interest may include: surface imaging; identifying one or more regions of interest corresponding to one or more initial regions of interest; and imaging (or re-imaging) the surface. In this invention, imaging (or re-imaging) the corresponding subsequent surface may include applying one or more automated functions to at least one identified region of interest.

[0014] In some embodiments, the method further includes: calculating a similarity score between one or more initial regions of interest (ROIs) of the initial image and multiple sub-regions of the corresponding subsequent surface image of the sample; thereby, based on the corresponding similarity scores (such as cross-correlation or cosine similarity), selecting one or more sub-regions as candidate ROIs in the corresponding subsequent surface of the sample. The method may also include: calculating reference codes for one or more initial ROIs of the initial image and codes for multiple sub-regions of the corresponding subsequent surface image of the sample, respectively. In this invention, the similarity score between each pair of initial ROIs and sub-regions is the similarity score between their codes.

[0015] The reference encoding can be a deep embedding of a trained machine learning algorithm (or model) (e.g., VisionTransformer neural network computation), and the encoding of the multiple sub-regions is also a deep embedding of multiple sub-regions computed by a trained machine learning model.

[0016] In some embodiments, the reference code and sub-region code can be corresponding image histograms, in which case the similarity score is the Kullback-Leibler divergence between the corresponding histograms. In some embodiments, the reference code and sub-region code are generated by a Gabor filter.

[0017] In some embodiments, regions of interest are selected from candidate regions of interest based on one or more screening criteria. Screening criteria may include one or more of the following: distance measures to the initial region of interest; exclusion regions (or defined regions) set on the sample surface; and distance measures to the center of the field of view.

[0018] In some embodiments, for identifying one or more regions of interest (ROIs), a trained machine learning model (e.g., a video object segmentation model) may be utilized, wherein the one or more initial ROIs are used to define one or more features to be determined. The identification described in this invention involves using a trained machine learning model to track one or more target features through a series of previous images. The trained machine learning model may be configured to segment each subsequent image using the initial ROIs as a reference segmentation result for the target features.

[0019] The present invention also provides an apparatus corresponding to the above-described method and comprising elements, modules, or components configured to perform the above-described method, or a device including said elements, modules, or components, such as one or more appropriately configured computing devices as described below.

[0020] Therefore, the present invention specifically provides a system for imaging a sample using a charged particle microscope. The system includes one or more processors and a memory for storing instructions. During operation, the system performs the following operations to execute instructions: acquiring (or receiving from) an initial image of the sample surface using the charged particle microscope; receiving instructions from the charged particle microscope to perform one or more automatic functions on one or more initial regions of interest; and acquiring (or receiving) a series of subsequent images of the sample's subsequent surface using the charged particle microscope. For each subsequent image, acquiring the subsequent image includes exposing the corresponding subsequent surface of the sample; identifying one or more regions of interest corresponding to the one or more initial regions of interest; and imaging the corresponding subsequent surface, wherein imaging the corresponding subsequent surface includes applying one or more automatic functions based on at least one identified region of interest corresponding to the one or more initial regions of interest. Based on this, the system can be equipped with the charged particle microscope, or optionally configured to communicate with and be connected to the charged particle microscope.

[0021] The present invention also provides one or more computer programs suitable for execution by one or more processors, such computer programs being configured to perform the methods described above and by the present invention. The present invention also provides one or more computer-readable media and / or network data signals, the one or more computer-readable media comprising (or storage media) such one or more computer programs. Brief description of the attached diagram Embodiments of the present invention will be described by way of example only with reference to the accompanying drawings, wherein: Figure 1 An example scenario of analyzing a sample using a charged particle microscope is shown; Figure 2 An example of a computer system used in this invention is shown; Figure 3A This illustrates a scenario where the automatic function fails when identifying similar regions to the initial region of interest on different surfaces; Figure 3B This illustrates a scenario where the automatic function fails when identifying / tracking regions of interest on different surfaces; Figure 4A An example of a system for imaging a sample surface is shown; Figure 4B A system embodiment of the region of interest module 4300 is shown; Figure 4C A system embodiment of the region of interest module 4300 is shown; Figure 5 A flowchart of the subsequent sample surface imaging method 500 is shown; Figure 6A A flowchart of the subsequent sample surface imaging method 610 is shown; Figure 6B A flowchart of the subsequent sample surface imaging method 620 is shown; Figure 7A This illustrates the process of identifying similar regions of interest in subsequent samples based on an initial region of interest; Figure 7B The process of identifying (or tracking) similar regions of interest in subsequent samples based on an initial region of interest is illustrated.

[0022] Detailed Description of Embodiments of the Invention The following is a description and accompanying drawings of certain embodiments of the present invention. However, it should be understood that the present invention is not limited to the described embodiments, and some embodiments may not include all the features described below. It should also be understood that various modifications and alterations can be made herein without departing from the broader spirit and scope of the invention as defined by the appended claims.

[0023] Figure 1 An example system 100 is shown that analyzes sample 120 using a charged particle microscope device 110 (or charged particle microscope). Types of charged particle microscope devices 110 include SEM, FIB-SEM, STEM, TEM, etc. Taking a biological sample as an example, the target features 130 contained in sample 120 are the objects to be imaged.

[0024] For example, sample 120 may be a biological sample. Sample 120 has a target feature 130. The sample may have multiple target features 130. The target feature 130 in sample 120 typically refers to a feature that the user expects to be clearly imaged in all subsequent slices. For this purpose, the user needs to select a region of interest that covers the target feature. An autofunction region (i.e., the region to which autofunctions are applied) may be defined based on the region of interest. The area of ​​the autofunction region is typically several times larger than the region of interest and lies outside the region of interest to avoid radiation damage to the region of interest when performing autofunctions. Specifically, the autofunction region may be defined based on a constant offset (e.g., pixel offset) relative to the region of interest. Typically, machine learning techniques are used to track or identify the region of interest based on subsequent sample slice analysis. The region of interest is based on the target feature, i.e., the region of interest is selected as the entire region or at least a portion of the region including the target feature 130. The charged particle microscope configuration 110 applies one or more autofunctions to the autofunction region defined based on the region of interest, and then configures the microscope 110 according to the output of these functions (i.e., the imaging parameters generated by the autofunctions). Subsequently, the charged particle microscope configuration uses parameters generated by the operation of the automated function to image the region of interest on the sample slice. Typically, the entire sample slice is imaged and the region of interest is identified by the charged particle microscope. However, it should be understood that only a portion of the surface, including the region of interest, is available for imaging.

[0025] The geometry of feature of interest 130 may be unknown. Sample 120 may be prepared by slicing using the external system or the internal system of the charged particle microscope apparatus 110. The sample may be sliced ​​into several slices 151, 152, 153 to 156, and each slice (or surface) may be imaged separately (or individually) by the charged particle microscope apparatus 110.

[0026] The charged particle microscope apparatus 110 is used to receive the sample 120. The charged particle microscope apparatus 110 can be configured to prepare sections of the sample using a diamond scalpel, an ion beam, or a laser.

[0027] The sample 120 can be prepared by slicing using various techniques in the field of charged particle microscopy that are well known to those skilled in the art.

[0028] The sample 120 used for imaging includes any one or more of the following: 2D or 3D biological samples prepared by cryogenic freezing, resin-embedded 2D or 3D biological samples, 2D or 3D materials science samples, 2D or 3D semiconductor samples, or samples prepared using one or more different techniques known in the field of charged particle microscopy. Such preparation techniques will not be described in detail in this invention.

[0029] During the imaging process, the charged particle microscope device 110 is used to image the first surface (or initial surface) of the sample 120. During the surface imaging of the sample 120, the charged particle microscope device 110 sets (or executes) its various imaging parameters by performing one or more functions. Based on this, the resulting image of the sample 120 surface is obtained using various imaging parameter values ​​set by one or more functions (or automatic functions). It should be understood that, in addition to using imaging parameters set by one or more automatic functions, image acquisition may also be based on other imaging parameters set by the user (or imaging parameters predefined based on standard values). Such automatic functions include, but are not limited to, any one or more of the following: objective lens calibration function, astigmatism correction function, focusing function, contrast / brightness function, centering and focusing function, astigmatism correction and centering function, etc.

[0030] The automatic adjustment function is used to set parameters such as automatic adjustment objective lens calibration, confluence height, focusing, astigmatism, centering, contrast, and brightness for charged particle microscopes.

[0031] To ensure that the target feature 130 on the surface of sample 110 can be correctly (or reliably or accurately) imaged, one or more automatic functions need to be applied to an automatic function region defined according to the region of interest (ROI) of the surface. The ROI is a selected region that includes at least a portion of the target feature 130. The relationship between the automatic function region and the ROI is defined to ensure that the automatic function region represents the ROI. In this way, high-quality resolution of the target feature in a given image can be ensured.

[0032] As described above, subsequent surfaces of sample 120 (or subsequent slices of sample 110) are exposed by slicing sample 120. After each subsequent surface of the sample is exposed, the charged particle microscope apparatus 110 images the subsequent surface. This imaging method is consistent with the initial surface imaging principle described above. Specifically, the charged particle microscope configuration 110 performs one or more functions in an automated functional area defined based on the region of interest (ROI) of the subsequent surface to set various imaging parameters suitable for itself. Similarly, the ROI is identified based on the target features, i.e., the region that includes all or at least a portion of the target features 130 on the surface should be selected.

[0033] Therefore, a specific target feature in a sample that may appear in multiple slices can be correctly imaged in all the slices in which it appears.

[0034] While the above discussion is based on treating each surface as a subsequent surface of a sliced ​​3D sample, this also applies to a series of surfaces from different samples that may contain common target features in 2D or 3D (e.g., specific organelles imaged in different cell samples).

[0035] The results of the region of interest analysis for subsequent surfaces not based on target feature 130 will be discussed below. Figure 3A and 3B Let's have a discussion.

[0036] Figure 2 An example of a computer system 1000 that can be used in embodiments of the present invention is shown. The computer system 1000 includes a computer 1020. The computer 1020 includes a storage medium 1040, a memory 1060, a processor 1080, an interface 1100, a user output interface 1120, a user input interface 1140, and a network interface 1160, all connected via one or more communication buses 1180. The charged particle microscope device 110 may have the computer system 1000 internally located or externally located. If the computer system is external to the charged particle microscope device 110, the two can be connected via a network interface or other data connection means.

[0037] Storage medium 1040 may be any form of non-volatile data storage device, such as one or more of hard disk drives, magnetic disks, optical disks, ROMs, etc. Storage medium 1040 may store an operating system for execution by processor 1080, thereby enabling computer 1020 to function properly. Storage medium 1040 may also store one or more computer programs (or software or instructions or code).

[0038] The memory 1060 is any random access memory (memory cell or volatile storage medium) used for storing data and / or computer programs (or software or instructions or code).

[0039] The processor 1080 may be any data processing unit suitable for executing one or more computer programs. These program portions may be stored in storage medium 1040 and / or memory 1060, and some of these programs may be programs according to embodiments of the present invention, or programs that, when executed by the processor 1080, enable the processor 1080 to perform methods according to embodiments of the present invention and configure system 1000 as a system according to embodiments of the present invention. The processor 1080 may include a single data processing unit or multiple data processing units operating in parallel or cooperatively with each other. When performing data processing operations according to embodiments of the present invention, the processor 1080 may store data in and / or retrieve data from storage medium 1040 and / or memory 1060. The processor 1080 may include one or more graphics processing units (GPUs) operating cooperatively with other data processing units of the processor 1080.

[0040] Interface 1100 can be any unit that provides an interface to a device 1220 external to or detachable from the computer 1020. Device 1220 can be a data storage device, such as one or more of optical discs, magnetic disks, solid-state storage devices, etc. Device 1220 has processing capabilities; for example, the device includes a smart card. Therefore, interface 1100 can access data from, provide data to, or connect to the device according to one or more instructions received by device 1220 from processor 1080.

[0041] User input interface 1140 is configured to receive input commands from a user or operator of system 1000. The user may provide this input via one or more input devices of system 1000, such as a mouse (or other instruction device) 1260 and / or a keyboard 1240, which are connected to or communicate with user input interface 1140. However, it should be understood that the user may input to computer 1020 via one or more other or alternative input devices, such as a touchscreen. Computer 1020 may store the input received via user input interface 1140 in memory 1060 for subsequent access and processing by processor 1080; or it may pass it directly to processor 1080, enabling the processor to respond to the user input.

[0042] User output interface 1120 is configured to provide image / video and / or audio output to the user or operator of system 1000. Thus, processor 1080 can be configured to instruct user output interface 1120 to generate an image / video signal conforming to the expected output requirements and provide that signal to the monitor (or display screen or display unit) 1200 of system 1000, with system 1000 connected to user output interface 1120. Additionally or alternatively, processor 1080 can be configured to instruct user output interface 1120 to generate an audio signal conforming to the expected output requirements and provide that signal to one or more speakers 1210 of system 1000, with system 1000 connected to user output interface 1120.

[0043] Finally, network interface 1160 is used by computer 1020 to download data from one or more data communication networks and / or upload data to one or more data communication networks.

[0044] It should be understood that, Figure 2 The architecture of the system 1000 shown is merely exemplary. In embodiments of the present invention, the computer system 1000 (e.g., compared to...) is utilized. Figure 2 As shown, it has fewer components, or compared to Figure 2As shown, (with additional and / or optional components), there may be different architectures. For example, computer system 1000 may include one or more of the following: personal computer; server computer; mobile phone; tablet computer; portable laptop computer; other mobile device or consumer electronic device; distributed (cloud) computing system, etc.

[0045] Figure 3A This illustrates a scenario where automated functions fail when identifying regions similar to the initial region of interest on different surfaces. For example, a user annotates an initial region of interest (ROI) on an initial surface 310 to apply one or more automated functions. The user has selected to study the cell wall and annotated ROI 302 on the initial surface 310. Therefore, the cell wall is the target feature 130 as described in this embodiment. On a subsequent surface 320, the user's initial annotation of ROI 302 remains close to the target feature, but is offset due to the inhomogeneity and curved 3D structure of the cells. On a subsequent surface 330, the user's initial annotation is no longer close to the target feature because the surface of each slice varies due to the inhomogeneity and curved 3D structure of the cells.

[0046] Surface 320, which is the target feature 130, remains close to the initial region of interest 302. This initial region of interest can be used to define the auto-function region so that automatic functions can be applied when imaging the sample surface. For the target feature, the image generated from surface 320 is correctly resolved, therefore, the generated image correctly (or accurately) reflects the target feature. However, the initial region of interest of surface 330 is not close to the target feature 304. Therefore, if this initial region of interest is used to define the auto-function region to apply automatic functions when imaging the sample surface, the generated image will not correctly reflect the target feature. For example, the target feature may be out of focus, or there may be no effective contrast to distinguish the target feature. This is because the automatic functions (or multiple automatic functions) that determine these attribute parameters do not consider the target feature 130, as it is not within the application range of the automatic functions (or multiple automatic functions).

[0047] Similarly, Figure 3B This illustrates a scenario where automated functions fail when identifying / tracking regions of interest (ROIs) on different surfaces using different slices (ROIs on different surfaces). For example, imaging of object 350 is required within sample 340. The user annotates the RIO 365 on an image of the first sample surface 360, which will be tracked across subsequent surfaces, so that one or more automated functions can be applied to the RIO-related regions being tracked. However, because the 3D structure of the target feature 350 is unknown and its distribution within cells is uneven, the application of automated functions based on the initial user annotations fails on subsequent surfaces 370, 380, and 390.

[0048] Figure 4A An example system for imaging the surface of sample 120 is shown. System 4000 includes multiple modules configured to send and receive inputs and outputs. System 4000 includes a receiving module 4100, a command module 4200, and a region of interest module 4300. The system optionally includes an automatic functional area module 4400. System 4000 may be part of the charged particle microscope 110 or a separate external system connected to the charged particle microscope 110.

[0049] The receiving module 4100 is configured to receive a sample surface image 4025. The sample surface image 4025 received by the receiving module 4100 can be an image of the sample surface to be re-imaged, or it can be an image of the sample surface before the sample surface to be re-imaged.

[0050] The receiving module 4100 can be configured to directly receive a sample surface image 4025 from the charged particle microscope 110, for example, where the system 4000 is part of the charged particle microscope 110, and the receiving module 4100 is associated with the acquisition of the sample surface image. The receiving module 4100 can also be arranged to indirectly receive input of the sample surface image 4025 from the charged particle microscope, for example, via a communication network between the system 4000 and the charged particle microscope 110, retrieved from the memory of the microscope 110. It should be understood that the receiving module 4100 can also be configured to receive input of the sample surface image 4025 from a suitable external system.

[0051] The instruction module 4200 is configured to receive one or more initial regions of interest 4050 for the surface of sample 120. The one or more initial regions of interest may be located on the initial surface (or slice) of the sample. However, it should be understood that the one or more initial regions of interest are not always on the initial surface, as they may not yet be visible and may only become visible on subsequent surfaces. The instruction module 4200 is configured to receive one or more initial regions of interest 4050 from a user. The user can annotate (or instruct) on an image of one or more regions of interest on the initial surface of sample 120. As previously mentioned, the user typically selects regions of interest to include all or part of the features of target feature 130. The target feature is a feature that the user expects to be tracked (or identified) on subsequent sample surfaces. Alternatively, the instruction module 4200 may be configured to receive one or more initial regions of interest 4050 from an external / other system configured to generate one or more initial regions of interest.

[0052] The region of interest module 4300 is configured to identify one or more regions of interest corresponding to one or more initial regions of interest. The identified one or more regions of interest are relative to a given surface of the sample 130. The initial region of interest may be a region of interest received by the instruction module 4200. Alternatively, the initial region of interest may be a region of interest on a previous surface of the sample 130 previously identified by the region of interest module 4300.

[0053] It should be understood that the identified one or more regions of interest correspond to the initial regions of interest. This means that on the respective surfaces of sample 130, these identified regions contain the same (or partial) features of interest as the initial regions of interest. In this invention, the same target features may mean the same physical entity. For example, if the slices are slices of the same organelle, then the identified one or more regions of interest correspond to one or more initial regions of interest, i.e., both contain partial structures of that organelle. Alternatively, in some embodiments, the same target features may represent different physical entities of the same type. For example, if the slices are slices of different organelles, then the identified one or more regions of interest correspond to one or more initial regions of interest, i.e., both contain partial structures of that organelle.

[0054] The region of interest (ROI) module 4300 can be configured to output one or more identified ROIs for later image acquisition of a given surface. Alternatively, the ROI module 4300 can be configured to trigger (or instruct or otherwise induce) imaging of a sample, utilizing one or more ROIs to define an automated functional region for applying automated functions used in the imaging. The image includes the application of one or more automated functions to at least one identified automated functional region defined in relation to the ROI, and image acquisition using the charged particle microscope 110 based on the output of said functions. It should be understood that the system 4000 can be configured to apply said functions to the identified one or more ROIs and configure the charged particle microscope 110 based on parameters generated by said automated functions.

[0055] As described above, automatic functions can be executed within the region of interest (ROI). However, typically, automatic functions will be executed within an automatic function region defined in relation to the ROI. Therefore, system 4000 may include an automatic function region module 4400. The automatic function region model is configured to define the automatic function region based on the ROI output by the ROI module 4300. The automatic function region module 4400 may store one or more predefined relative automatic function region definitions, specifying the relative position and / or extent of the automatic function region relative to the ROI. Examples of such relative definitions include vector offsets of the ROI.

[0056] The following is a further discussion of the region of interest module 4300 and its alternative embodiments.

[0057] Figure 4B An example embodiment of a system with a region of interest module 4300 is shown. Figure 4B In an example embodiment, the region of interest module 4300 includes a machine learning (ML) model 4310 and a processing module 4320. The processing module 4320 includes a cosine similarity module 4322 and a filtering module 4324.

[0058] The ML model 4310 includes an encoder 4315. The encoder 4315 of the machine learning model 4310 is configured to compute a deep embedding of the input received by the ML model 4310. The ML model 4310 receives a sample surface image 4025 and one or more initial regions of interest 4050 as input. A deep embedding is a mathematical representation of data, in this case, the sample surface image 4025 and one or more initial regions of interest 4050 as input. Therefore, it should be understood that a deep embedding is a specific type of data encoding (or coding) of the input data.

[0059] The machine learning model 4310 can be a multi-layer neural network that encodes any image as a vector in a deep embedding space.

[0060] The machine learning model 4310 can be (but is not limited to) a convolutional neural network or a Vision Transformer neural network. The ML model 4310 can be trained on public datasets. The machine learning model can also be trained on images in general rather than domain-specific formats, such as photos from a consumer camera. The encoder 4315 of the ML model 4310 typically computes a corresponding depth embedding for each sub-region of the received input image. Typically, a sub-region is a square array of pixels (usually 16x16 pixels). However, it should be understood that this is merely an example of a sub-region, and the ML model is not limited to 16x16 pixels.

[0061] The encoder 4315 of the ML model 4310 receives one or more initial regions of interest 4050 and calculates and stores reference depth embeddings of the one or more initial regions of interest 4050. As previously described, the one or more initial regions of interest 4050 may include all or part of the target features 130 of the initial surface of the sample 120. The reference depth embeddings of the one or more initial regions of interest 4050 may be stored in the ML model 4310 or in an external memory connected to the ML model 4310 via a suitable communication network (not in...). Figure 4B (as shown in the image).

[0062] Then, the encoder 4315 of the ML model 4310 receives the sample surface image 4025, in which similar regions of interest are to be identified. The encoder 4315 of the ML model 4310 calculates and stores the depth embedding of the sample surface image 4025. The depth embedding of the sample surface image 4025 can be stored in the ML model 4310 or in an external memory connected to the ML model 4310 via a suitable communication network (not in...). Figure 4B (As shown in the image).

[0063] Processing module 4320 includes a cosine similarity module 4322 and a filtering module 4324. Alternatively, reference deep embeddings and deep embeddings can be input from a connected external memory to the similarity module 4322, which is connected to the ML model 4310 (not in...) via a suitable communication network. Figure 4B (As shown in the image).

[0064] The similarity module 4322 is configured to determine a set of candidate regions of interest (ROIs) for the sample surface image 4025. Typically, these candidate ROIs are determined based on the deep embedding of the similarity (or similarity score) of the proposed ROIs and the embeddings of one or more initial ROIs that meet predefined criteria. These predefined criteria may include: predefined thresholds (e.g., any candidate region whose similarity exceeds the threshold), ranking criteria (e.g., selecting the candidate region with the highest similarity score), etc.

[0065] Typically, the similarity module utilizes cosine similarity. In this way, the similarity module can be configured to calculate the cosine similarity between the reference depth embeddings of one or more initial regions of interest (ROIs) and the depth embeddings of candidate ROIs. It should be understood that other similarity scores can also be used.

[0066] Then, the candidate regions of interest are typically fed into a filtering module 4324, which applies one or more filtering criteria to reduce the number of candidate regions of interest. The filtering module 4324 outputs the remaining regions of interest to determine the regions where automatic functions can be applied. The applied automatic functions can be... Figure 1 One or more automatic functions listed therein.

[0067] The filtering module 4324 filters the generated candidate regions to output the region of interest. The filtering module 4324 can apply one or more of the following criteria to generate the region of interest: distance metric to the initial region of interest; exclusion regions (or blocks) of the surface; distance metric to the center of the field of view, etc.

[0068] It should be understood that the filtering module can be omitted entirely. For example, the similarity module might be configured to output a single region of interest, such as the candidate region of interest with the highest similarity score.

[0069] The ML model 4310 described above can be composed of any one or more of many well-known ML algorithms. In a particular example, the Vision Transformer ML model can be utilized. Full details of the Vision Transformer model can be found in "A 16x16 Image: A Transformer for Large-Scale Image Recognition," by Alexey Dosovitskiy et al., https: / / doi.org / 10.48550 / arXiv.2010.11929. In this example, the trained encoder of the Vision Transformer model is used as encoder 4315. The trained encoder computes a depth embedding of the input image such that for each 16x16 pixel region in the image, there is one such embedding. Reference centers for one or more initial regions of interest 4050 are computed and stored as reference embeddings. The depth embedding of the sample surface 4025 is computed. The depth embedding is compared to the reference embedding using a cosine similarity metric. The most similar depth embedding is selected and mapped to coordinates in the sample surface image 4025 to compute a similarity score.

[0070] In this way, Figure 4B The region of interest module 4300 can be understood as being used to identify subsequent surfaces similar to the initial region of interest. Specifically, the identified region of interest may not trace specific physical entities (such as specific organelles) through the surface sequence. However, the identified region of interest includes target features of the same or similar type as the target feature type of the initial region.

[0071] While machine learning models have been utilized in the above discussion, it should be understood that the methods described can also be implemented using classical image processing techniques. Specifically, the machine learning model 4310 and the similarity module 4322 can be replaced by a single image processing similarity module. The image processing similarity module can receive a sample surface image 4025 as input and one or more initial regions of interest 4050. Then, the image processing similarity module can use known image comparison methods to determine candidate regions of interest.

[0072] This image comparison method can include any of the following: • Interrelated, where the interrelatedness between the initial region of interest 4050 and the candidate region of interest defines the similarity score between the initial region of interest 4050 and the candidate region of interest; • Histogram comparison, where image histograms of the initial region of interest (ROI) 4050 and candidate ROIs are calculated separately. Then, the Kullback Leibler divergence between the corresponding histograms can define the similarity score between the initial ROI 4050 and the candidate ROIs; or • Gabor filter similarity is calculated by encoding the initial region of interest (ROI) 4050 and candidate ROIs respectively. These encodings can effectively replace the deep embeddings described above. Therefore, the similarity score between the initial ROI 4050 and the candidate ROIs can be defined as the cosine similarity between the corresponding Gabor filter encodings.

[0073] Figure 4C An example embodiment of a system with a region of interest module 4300 is shown. Figure 4C In an example embodiment, the region of interest module 4300 includes an ML segmentation model 4330.

[0074] The ML model 4330 is configured to receive input from one or more initial regions of interest 4050 based on one or more target features 4055. In this example, the initial regions of interest 4050 typically specify the corresponding target features 4055. Therefore, the regions of interest 4050 in this embodiment can be considered as segmentation masks typically generated by the user. Figure 4C In the example image shown, the region of interest 4050 and the target feature 4055 are indicated in yellow. The ML model 4330 is also set to receive the sample surface image 4025 when the region of interest 4050 is being tracked.

[0075] The ML model is configured to trace an initial region of interest (ROI) 4050 from a sample surface image 4025. It should be understood that this tracing is similar to tracking an object through subsequent frames of a video. Specifically, the initial ROI can be considered as segmenting or identifying the object to be tracked (in this case, the target feature). The initial ROI can be considered as a reference segment, and the image corresponding to the initial ROI is a reference frame. The ML model 4330 is configured to generate a ROI 4055 on the sample surface image 4025 that corresponds to (or defines or includes) the initial ROI. Typically, the output of the ROI takes the form of a segmentation map (or mask) of the target feature.

[0076] Similar to the example above, the target output area can be used to perform automatic image acquisition functions. Specifically, the target output area can be used to define an automatic function area on which the aforementioned automatic functions can be performed.

[0077] It should be understood that any suitable video or image segmentation model (e.g., any video object segmentation model) can be used as an ML model. Examples of such models can be found in Gao, M., Zheng, F., and Yu, JJQ. et al. The book he authored, "Deep Learning for Video Object Segmentation: An Overview". Artif Intell Rev 56, 457–531 (2023). https: / / doi.org / 10.1007 / s10462-022-10176-7. The inventors discovered a particularly advantageous ML model: the Adaptive Feature Library and Uncertain Region Refinement (AFB-URR) model. Details of this model can be found in "Video Target Segmentation Based on Adaptive Feature Library and Uncertain Region Refinement" by Yongqing Liang, Xin Li, Navid Jafari, and Qin Chen, 2020, in the proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS '20). Curran Associates Inc., Red Hook, NY, USA, Article 289, 3430 3441. When using this model, an initial region of interest (ROI) is used instead of a reference segment, and the image corresponding to the ROI is used instead of the reference frame. The target frame is replaced by the sample surface image 4025.

[0078] Figure 5 A flowchart of a method 500 for imaging a subsequent sample surface (surface N) is shown, which can be implemented according to certain embodiments of the present invention. Figure 1 System 100 and Figure 4A The system 4000 is running.

[0079] In step 502, an initial image of the surface is obtained using a charged particle microscope 110 according to the system 100.

[0080] In step 504, instructions are received based on one or more initial regions of interest (GROUP) derived from one or more target features (by the instruction module according to...). Figure 4A System 4000).

[0081] In step 506, the surface of the sample to be imaged is exposed to a charged particle microscope (which can be determined by the receiving module according to...). Figure 4A (System 4000 receiver).

[0082] In step 508, one or more regions of interest to be imaged are determined by... Figure 4B and 4C The aforementioned system identification.

[0083] In step 510, one or more automatic functions (such as...) are applied based on one or more regions of interest determined on the sample surface. Figure 1 (as mentioned above).

[0084] In step 512, the sample surface is imaged by applying an automatic function.

[0085] The following is about Figure 5 Further discussion of various embodiments.

[0086] Figure 6A A flowchart of a method 610 for acquiring subsequent surface N is shown, which can be implemented according to certain embodiments of the present invention. Figure 1 System 100 and Figure 4A The system is executing error 4000. It should be understood that... Figure 6A In this method, the region of interest (ROI) is actually identified in the previously imaged surface (surface N-1). This identified ROI is then used to apply automatic functions to surface N, as the drift of the ROI from the previously imaged surface (surface N-1) to the surface to be imaged (surface N) will be minimal. This method is suitable for cases where the sample surface is sensitive. This will be discussed further below. As mentioned earlier, automatic functions can be applied to the ROI itself, or more commonly, to the automatic function regions associated with the ROI.

[0087] Figure 6B A flowchart of a method 620 for acquiring subsequent surface N is shown, which can be implemented according to certain embodiments of the present invention. Figure 1 System 100 and Figure 4A The system is executing error 4000. It should be understood that... Figure 6B In this process, surface N is imaged, and then the region of interest is identified on the imaged surface N. Once an automatic function is executed to obtain a better image, the region of interest on surface N is re-imaged. This method is useful when the sample is not easily affected during imaging and re-imagening. This will be discussed further below.

[0088] In some cases, users may expect to identify similar regions of interest in subsequent samples given one or more initial regions of interest. Examples may be provided by... Figure 4A and 4B The system is based on the following description Figure 5 , 6A Perform the steps listed in the flowchart in 6B.

[0089] In one embodiment, for imaging surface N, receiving module 4100 receives an image of surface N-1 4025, and instruction module 4200 receives instructions as input for one or more initial regions of interest 4050. As previously described, the one or more initial regions of interest 4050 may include all or part of the target features 130 of the initial surface of sample 120. The image of surface N-1 4025 is an image of the surface preceding surface N, which has been imaged in one embodiment of the invention. Figure 4A As described above, the receiving module 4100 obtains an image of surface N-1 4025. One or more initial regions of interest 4050 are input into the region of interest module 4300, where the encoder 4315 of the ML model 4310 calculates the depth embeddings of the one or more initial regions of interest 4050 and outputs reference depth embeddings of the one or more initial regions of interest 4050. The ML model 4310 can be as follows: Figure 4B The ML model is shown. The encoder 4315 of the ML model 4310 can calculate one or more reference depth embeddings for each 16x16 pixel region in the input of one or more initial regions of interest 4050. Then, an image of surface N-1 4025 is input to the encoder 4315 of the ML model 4310, which is located in the region of the target module 4300. The encoder 4315 of the ML model 4310 calculates the depth embedding of the sample surface image N-1 4025. The encoder 4315 of the ML model 4310 can calculate one or more depth embeddings for each 16x16 pixel region in the surface N-1 4025 image. The calculated reference depth embeddings and the depth embedding data results are input to the cosine similarity calculation module 4322 of the processing module 4320, where the cosine similarity between the reference depth embeddings and the depth embeddings is calculated to identify candidate regions of interest. The candidate regions of interest are input to the filtering module 4324. The filtering module 4324 outputs regions of interest on which surface N can be imaged using automatic functions. The filtering is... Figure 4B The method is performed. The imaging includes applying one or more of the automatic functions based on at least one identified region of interest, and acquiring an image of surface N using a charged particle microscope 110 based on the output of the automatic functions. It should be understood that the system 4000 can be configured to apply the automatic functions and configure the charged particle microscope 110 based on the parameters generated by the automatic functions and the image of surface N. Similarly, once surface N is imaged, surface N+1 can be imaged using the image of surface N. This embodiment is particularly useful when the sample surface to be imaged is sensitive and may be damaged by the charged particle microscope.

[0090] In one embodiment, to image surface N, instruction module 4200 receives instructions as input to one or more initial regions of interest 4050. As previously described, the one or more initial regions of interest 4050 may include all or part of the target features 130 of the initial surface of sample 120. The one or more initial regions of interest 4050 are input to region of interest module 4300, where encoder 4315 of ML model 4310 calculates depth embeddings of the one or more initial regions of interest 4050 and outputs reference depth embeddings of the one or more regions of interest 4050. Encoder 4315 of ML model 4310 may calculate one or more reference depth embeddings for each 16x16 pixel region from the input of the one or more initial regions of interest 4050. Figure 4B As shown, the ML model can be ML model 4310. Surface N is imaged by charged particle microscope 110 and input 4025 to receiving module 4100. Then, the image of surface N 4025 is input to encoder 4315 of ML model 4310, which is located in region of interest module 4300. Encoder 4315 of ML model 4310 calculates the depth embedding of sample surface image N-4025. Encoder 4315 of ML model 4310 can calculate one or more depth embeddings for each 16x16 pixel region in surface N-4025 image. The reference depth embedding and the depth embedding are respectively input to cosine similarity module 4322 of processing module 4320, where cosine similarity between reference depth embedding and depth embedding is calculated. The calculated cosine similarity is input to filtering module 4324. Filtering module 4324 outputs region of interest, on which surface N can be imaged using automatic functions. The filtering is... Figure 4B The method is performed. The imaging includes applying one or more automated functions and acquiring an image of surface N using a charged particle microscope 110 based on the output of the automated functions. It should be understood that the system 4000 can be configured to apply the functions to one or more identified regions of interest and configure the charged particle microscope 110 based on the parameters generated by the automated functions. This embodiment is particularly useful when the sample surface to be imaged is not very sensitive and / or may be damaged by the charged particle microscope.

[0091] Figure 7A An example of the above embodiment is shown, in which a similar region of interest is identified on a subsequent sample surface. Instructions for a region of interest 7110 of sample surface 7100 are provided. Following the instructions provided... Figure 4A and 4B The system is based on the following description Figure 5 , 6AThe steps listed in the flowchart of 6B were performed to identify similar regions of interest 7130, 7140 and 7150 on the subsequent sample surface 7120.

[0092] In some cases, users may expect to trace one or more regions of interest (ROIs) on the surface of subsequent samples, given an initial set of one or more ROIs. Examples can be found by... Figure 4A and 4C The system is based on the following description Figure 5 , 6A Perform the steps listed in the flowchart in 6B.

[0093] In one embodiment, to image surface N, receiving module 4100 receives an image of surface N-1 4025, and instruction module 4200 receives an instruction as input for an initial region of interest 4050. The initial region of interest specifies (or represents) target features. The image of surface N-1 4025 is an image of the surface preceding surface N, which has already been imaged in one embodiment of the invention. Figure 4A The receiving module 4100 obtains an image of surface N-1 4025. Typically, the initial region of interest 4050 is input into the region of interest module 4300 as a reference segmentation.

[0094] The ML model is configured to trace an initial region of interest 4050 through sample surface images N-1, N, N+1, etc. It should be understood that this tracing is similar to tracking an object through subsequent frames of a video. The ML model 4330 is configured to generate regions of interest (ROIs) on the sample surface image N-1 that correspond to (or define or include) the initial ROI 4055. Typically, the output of the ROI is in the form of a segmentation map (or mask) of the target features.

[0095] The region of interest (or segmentation mask) is output to the charged particle microscope 110, which uses it to estimate the location of one or more regions of interest on surface N and performs automatic functions to image them based on the estimated regions of interest. The image includes images acquired using the charged particle microscope 110 based on at least one of the identified regions of interest, and based on the output of the functions. It should be understood that the system 4000 can be configured to apply the functions to the identified regions of interest and configure the charged particle microscope 110 based on the parameters generated by the automatic functions. Similarly, once surface N is imaged, a segmentation mask can be generated using the image of surface N to image surface N+1. This embodiment is particularly useful when the sample surface to be imaged is sensitive and may be damaged by the charged particle microscope.

[0096] In another embodiment, to image surface N, surface N is first imaged by charged particle microscope 110 and the image is input into receiving module 4100. The image of surface N 4025 is then input into an ML model, and an initial region of interest 4050 is traced using the image of surface N. ML model 4330 is configured to generate regions of interest on the sample surface image N that correspond to (or define or include) the initial region of interest 4055. Typically, the output of the region of interest is in the form of a segmentation map (or mask) of target features.

[0097] The region of interest (or segmentation mask) is output to the charged particle microscope 110 and used by the charged particle microscope 110 to re-image the surface N. The re-imaging includes applying one or more automated functions based on at least the identified region of interest, and acquiring an image of the surface N using the charged particle microscope 110 based on the output of the automated functions. It should be understood that the system 4000 can be configured to apply the functions to the identified one or more regions of interest and configure the charged particle microscope 110 based on the parameters generated by the automated functions. This method can be used when the sample or sample surface is insensitive and will not be damaged by the initial imaging.

[0098] Figure 7A An example of the above embodiment is shown, in which a similar region of interest is identified in a subsequent sample surface. Instructions for a region of interest 7210 of sample surface 7200 are provided. Following the instructions provided... Figure 4A and 4C The system is based on the following description Figure 5 , 6A The steps listed in the flowchart of 6B were performed to find a similar region of interest (7210) on the subsequent sample surface 7220.

[0099] It should be understood that the described method has been presented as individual steps performed in a specific order. However, those skilled in the art should understand that these steps can be combined or performed in different orders while still achieving the desired results.

[0100] It should be understood that embodiments of the present invention can be implemented using a variety of different information processing systems. Specifically, although exemplary computing systems and methods are provided in the accompanying drawings and discussion, these are presented merely for the purpose of providing useful reference in discussing various aspects of the invention. Embodiments of the invention can be executed on any suitable data processing device, such as a personal computer, tablet computer, personal digital assistant, mobile phone, server computer, etc. Of course, for purposes of discussion, the description of systems and methods has been simplified, and they are merely one of many different types of systems and methods that can be used in embodiments of the invention. It should be understood that the boundaries between logical blocks are merely illustrative, and alternative embodiments may combine logical blocks or elements, or alternative functional decompositions may be applied to various logical blocks or elements.

[0101] It should be understood that the aforementioned functionality can be implemented as hardware and / or software as one or more corresponding modules. For example, the aforementioned functionality can be used in one or more software components executed by the system's processor. Furthermore, the aforementioned functionality can be used in one or more field-programmable gate arrays (FPGAs) and / or one or more application-specific integrated circuits (ASICs) and / or one or more digital signal processors (DSPs) and / or other hardware devices. Method steps implemented according to the flowcharts contained herein or as described above can be implemented by corresponding modules; multiple method steps implemented according to the flowcharts contained herein or as described above can be implemented jointly by a single module.

[0102] It should be understood that as long as embodiments of the present invention can be implemented by a computer program, the storage medium and transmission medium carrying the computer program constitute aspects of the present invention. A computer program may have one or more program instructions or program code that, when executed by a computer, perform embodiments of the present invention. As used herein, the term "program" can be a sequence of instructions designed to execute on a computer system and may include subroutines, functions, programs, modules, target methods, target implementations, executable applications, applets, server applets, source code, object code, shared libraries, dynamic link libraries, and / or other sequences of instructions designed to execute on a computer system. Storage media may be disks (such as hard disk drives or floppy disks), optical disks (such as CD-ROMs, DVD-ROMs, or Blu-ray discs), or memory (such as ROMs, RAMs, EEPROMs, EPROMs, flash memory, or portable / removable memory devices), etc. Transmission media may be communication signals, data broadcasts, communication links between two or more computers, etc.

Claims

1. A method for imaging a sample using a charged particle microscope, the method comprising: Initial images of the sample surface are acquired using a charged particle microscope; Receive instructions from a charged particle microscope to perform one or more automatic functions on one or more initial regions of interest; A series of subsequent images of the sample surface were acquired using a charged particle microscope. For each subsequent image, acquiring the subsequent image includes exposing the corresponding subsequent surface of the sample; Identify one or more regions of interest corresponding to one or more initial regions of interest; and Imaging the corresponding subsequent surface includes applying one or more automatic functions based on at least one region of the identified regions of interest corresponding to one or more initial regions of interest.

2. The method as described in claim 1, wherein, Identifying one or more surface regions of interest corresponding to one or more initial regions of interest includes: Identify one or more regions of interest in the previous image that correspond to one or more initial regions of interest; and Imaging the corresponding subsequent surface, wherein imaging the corresponding subsequent surface includes applying one or more automatic functions to at least one region of the identified region of interest based on the previous image.

3. The method of claim 1, wherein identifying one or more surface regions of interest corresponding to one or more initial regions of interest comprises: Image the surface; Identify one or more regions of interest that correspond to one or more initial regions of interest; as well as Imaging a surface, wherein imaging a corresponding subsequent surface includes applying one or more automatic functions to at least one region of an identified region of interest.

4. The method according to any one of claims 1 to 3, wherein the method further comprises: Calculate similarity scores between one or more initial regions of interest in the initial image and multiple sub-regions in the corresponding subsequent surface images of the sample; as well as Based on the corresponding similarity scores, one or more sub-regions are selected as candidate regions of interest from the corresponding subsequent surfaces of the sample.

5. The method of claim 4, wherein the method further comprises: Calculate the reference codes for one or more initial regions of interest in the initial image; Calculate the encoding of multiple sub-regions of the image of the corresponding subsequent surface of the sample; The similarity score for each pair of initial regions of interest and sub-regions of interest is the similarity score between the corresponding codes.

6. The method of claim 5, which is dependent on claim 4, wherein the reference encoding is a deep embedding computed by a trained machine learning algorithm, and the encoding of the plurality of sub-regions is a deep embedding of the plurality of sub-regions computed by a trained machine learning model.

7. The method of claim 6, wherein the trained machine learning model is a visual converter neural network.

8. The method of claim 4, wherein the similarity score is a correlation.

9. The method of claim 5, wherein the reference encoding and the encoding of the sub-region are corresponding image histograms, and the similarity score is the Kullback-Leibler divergence between the corresponding histograms.

10. The method of claim 5, wherein the reference encoding and the encoding of the sub-region are generated by a Gabor filter.

11. The method of any one of claims 4-7 or 10, wherein the similarity score is cosine similarity.

12. The method of any one of claims 4-11, further comprising: Select a region of interest from the candidate regions of interest based on one or more selection criteria.

13. The method of claim 12, wherein The selection criteria include one or more of the following: distance measure to the initial region of interest; exclusion region on the surface; distance measure to the center of the field of view.

14. The method according to any one of claims 1 to 3, wherein The step of identifying one or more regions of interest utilizes a trained machine learning model, where, One or more initial regions of interest define one or more features of interest. The identification described therein includes using a trained machine learning model to track one or more features of interest through the series of previous images.

15. The method of claim 14, wherein the trained machine learning model is a video object segmentation model.

16. The method of claim 14 or 15, wherein the trained machine learning model is configured to segment each subsequent image using the initial region of interest as a reference segmentation of the features of interest.

17. The method of claim 1, wherein receiving the instruction for the initial region comprises: Receive annotations from users indicating the initial region of interest.

18. The method of claim 1, wherein, Samples used for imaging can be 3D biological samples prepared by cryogenic freezing, 3D biological samples embedded in resin, materials science samples, or semiconductor samples.

19. The method of claim 1, wherein exposing the respective subsequent surfaces of the sample comprises milling, laser cutting, or diamond cutting with a focused ion beam or electron beam, or exposing each subsequent surface.

20. The method of claim 1, wherein the automatic functions include one or more of lens alignment function, astigmatism correction function, focusing function, contrast / brightness function, focus centering function, astigmatism correction centering function, or a combination of the previously listed basic automatic functions.

21. An apparatus configured to perform the method according to any one of claims 1 to 20.

22. One or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 20.

Images