Data acquisition in charged particle microscopy

Abstract

To improve a charged particle microscope (CPM) technique.SOLUTION: Disclosed herein are charged particle microscopy (CPM) support systems, as well as related methods, computing devices, and computer-readable media. For example, in some embodiments, a CPM support apparatus may include: first logic to cause a CPM to generate a single image of a first portion of a specimen; second logic to generate a first mask based on one or more regions-of-interest provided by user annotation of the single image; and third logic to train a machine-learning model using the single image and the one or more regions-of-interest. The first logic may cause the CPM to generate multiple images of corresponding multiple additional portions of the specimen, and the second logic may, after the machine-learning model is trained using the single image and the one or more regions-of-interest, generate multiple masks based on the corresponding images of the additional portions of the specimen using the machine-learning model without retraining.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Microscopy is a technical field for better viewing objects that are difficult to see with the naked eye using a microscope. Different fields of microscopy include, for example, optical microscopy, charged particle (electron and / or ion) microscopy, and scanning probe microscopy. Charged particle microscopy involves using a beam of accelerated charged particles as an illumination source. Types of charged particle microscopy include, for example, transmission electron microscopy, scanning electron microscopy, scanning transmission electron microscopy, and focused ion beam microscopy.

Brief Description of the Drawings

[0002] Embodiments will be readily understood by the following detailed description in conjunction with the accompanying drawings. For the sake of facilitating this description, like reference numerals refer to like structural elements. Embodiments are shown by way of example and not limitation in the figures of the accompanying drawings.

[0003] [Figure 1] A charged particle microscope (CPM) system configured for data acquisition according to embodiments disclosed herein is shown. [Figure 2] It is a block diagram of an exemplary CPM data acquisition module for performing data acquisition according to various embodiments. [Figure 3A] Stages of a CPM data acquisition process that can be executed by a CPM data acquisition module according to various embodiments are shown. [Figure 3B] Stages of a CPM data acquisition process that can be executed by a CPM data acquisition module according to various embodiments are shown. [Figure 3C] Stages of a CPM data acquisition process that can be executed by a CPM data acquisition module according to various embodiments are shown. [Figure 3D] Stages of a CPM data acquisition process that can be executed by a CPM data acquisition module according to various embodiments are shown. [Figure 3E]This shows the stages of the CPM data acquisition process that can be performed by a CPM data acquisition module in various embodiments. [Figure 4A] This describes various stages in the CPM data acquisition process that can be performed by a CPM data acquisition module in various embodiments. [Figure 4B] This describes various stages in the CPM data acquisition process that can be performed by a CPM data acquisition module in various embodiments. [Figure 5] This is a flowchart illustrating exemplary methods for performing CPM data acquisition using various embodiments. [Figure 6] This is a flowchart illustrating exemplary methods for performing CPM data acquisition using various embodiments. [Figure 7] This is a flowchart illustrating exemplary methods for performing CPM data acquisition using various embodiments. [Figure 8] Examples of graphical user interfaces that may be used to perform some or all of the CPM data acquisition methods disclosed herein, according to various embodiments. [Figure 9] This is a block diagram of an exemplary computing device capable of performing some or all of the CPM data acquisition methods disclosed herein, according to various embodiments. [Figure 10] This is a block diagram of an exemplary scientific instrument support system in which some or all of the CPM data acquisition methods disclosed herein can be performed according to various embodiments. [Modes for carrying out the invention]

[0004] Disclosed herein are charged particle microscope (CPM) assisted systems, as well as related apparatus, methods, computing devices, and computer-readable media. For example, in some embodiments, the CPM assisted system may include: a first logic for causing the CPM to generate one image of a first portion of a specimen; a second logic for generating a first mask based on one or more regions of interest provided by user annotations on the single image; and a third logic for training a machine learning model using the single image and one or more regions of interest. The first logic may cause the CPM to generate multiple images of corresponding additional portions of the specimen, and the second logic may, after the machine learning model has been trained using the single image and one or more regions of interest, generate multiple masks based on corresponding images of additional portions of the specimen using the machine learning model without retraining.

[0005] The embodiments of CPM data acquisition support disclosed herein can achieve improved performance compared to conventional approaches. For example, the CPM data acquisition techniques disclosed herein can dramatically improve imaging throughput without requiring skilled users to invest significant time in generating large training corpora. Thus, the embodiments disclosed herein offer improvements to CPM techniques (e.g., improvements to data acquisition techniques that support such scientific instruments, among other improvements).

[0006] The embodiments disclosed herein can achieve increased acquisition speed, reduced data storage requirements, and / or reduced radiation damage to specimens compared to conventional approaches. For example, conventional approaches typically utilize conventional image processing or machine learning techniques that require hundreds or thousands of input / output pairs for training. However, these approaches have many technical problems and limitations. For example, conventional image processing techniques often fail to accurately recognize features of interest (and therefore require sufficient supervision by experienced CPM users), and conventional machine learning techniques require a considerable upfront investment of time and energy to generate a sufficient training set (an intensive process that must be repeated for all features of interest).

[0007] Various embodiments disclosed herein can improve upon conventional approaches to achieve technical advantages such as reducing acquisition time and / or overall radiation dose by generating selective imaging masks based on a small number of training sets provided by the user. Such technical advantages cannot be achieved with routine conventional approaches, and all users of systems including such embodiments benefit from these advantages (e.g., by assisting the user in performing technical tasks, such as identifying features of interest in CPM specimens, through a guided human-machine interaction process). Accordingly, the technical features of the embodiments disclosed herein, as well as combinations of the features of the embodiments disclosed herein, are clearly not conventional in the field of CPM data acquisition. The computational and user interface features disclosed herein involve not only the collection and comparison of information but also the application of new analytical and technical techniques to modify the operation of the CPM system. Accordingly, this disclosure introduces functions that neither conventional computing devices nor humans could perform. Accordingly, embodiments of this disclosure can serve technical objectives such as controlling charged particle microscope systems and processes. In particular, this disclosure provides technical solutions to technical problems, including but not limited to faster data acquisition in CPM systems.

[0008] In the following detailed description, references are made to the accompanying drawings that form part of this specification, similar figures throughout designate similar parts, and examples of possible embodiments are shown. It should be understood that other embodiments may be utilized and structural or logical modifications may be made without departing from the scope of this disclosure. Therefore, the following detailed description should not be construed as restrictive.

[0009] Various operations may be described sequentially as multiple separate actions or operations in a manner that is most helpful in understanding the subject matter disclosed herein. However, the order of description should not be interpreted as suggesting that these operations are necessarily order-dependent. In particular, these operations may not be performed in the order presented. The operations described may be performed in a different order than in the embodiments described. Various additional operations may be performed, and / or the operations described may be omitted in additional embodiments.

[0010] For the purposes of this disclosure, the phrases "A and / or B" and "A or B" mean (A), (B), or (A and B). For the purposes of this disclosure, the phrases "A, B, and / or C" and "A, B, or C" mean (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C). Some elements may be referred to in the singular form (e.g., "processing device"), but any suitable element may be represented by multiple instances of that element, and vice versa. For example, a set of operations described as being performed by a processing device may be implemented using different operations performed by different processing devices.

[0011] This description uses the phrases “one embodiment,” “various embodiments,” and “several embodiments,” each of which may refer to one or more identical or different embodiments. Furthermore, terms such as “comprising,” “including,” and “having” as used in reference to embodiments of this disclosure are synonymous. When used to describe a range of dimensions, the phrase “between X and Y” refers to a range including X and Y. As used herein, “apparatus” may refer to any individual device, a collection of devices, a part of a device, or a collection of parts of a device. Drawings are not necessarily to scale.

[0012] Figure 1 shows an embodiment of a CPM system 100 including a CPM1 coupled to a display device 120. In some embodiments, the CPM system 100 may be a slice-and-view volume acquisition system or a tilt-series acquisition system. The CPM1 may include any preferred type of CPM such as a transmission electron microscope (TEM), scanning electron microscope (SEM), scanning transmission electron microscope (STEM), cryo-electron microscope (cryoEM), ion beam microscope, or dual-beam microscope (e.g., a focused ion beam scanning electron microscope (FIB-SEM)). The CPM1 may include an enclosure 2 having a charged particle source 4 therein. In some embodiments, the enclosure 2 may be a vacuum enclosure, but in other embodiments, a specific gaseous environment may be maintained within the enclosure 2 (e.g., for "environmental STEM" applications). The charged particle source 4 is The charged particle source 4 may be, for example, an electron source (e.g., a Schottky gun), a positive ion source (e.g., a gallium ion source or a helium ion source), a negative ion source, a proton source, or a positron source. The charged particle source 4 may generate a beam of charged particles that crosses an illuminator 6 which directs and / or focuses the particles onto a region of the sample S. The illuminator 6 may also perform aberration reduction, trimming, and / or filtering of charged particles output by the charged particle source 4. The illuminator 6 may have an axis 8 and may include one or more subcomponents such as an electrostatic lens, a magnetic lens, a scanning deflector, a corrector (e.g., an astigmatism corrector), and / or a capacitor system.

[0013] The specimen S may be held on a specimen holder 10 which can be positioned with multiple degrees of freedom by a positioning device 12. For example, the specimen holder 10 may include fingers that can translate in the xy plane and may be rotated about an axis in the xy plane to achieve different inclination angles of the specimen with respect to the axis 8 of the charged particle beam from the illuminator 6. Such movement may allow different areas of the specimen S to be illuminated, scanned, and / or inspected at different angles by the charged particle beam moving along the axis 8 (and / or may allow scanning motion to be performed as an alternative to beam scanning). A cooling device 14 may be in thermal contact with the specimen holder 10 and, if desired, may maintain the specimen holder 10 at a cryogenic temperature (e.g., using a circulating cryogenic coolant to achieve and maintain a desired low temperature).

[0014] A focused charged particle beam traveling along axis 8 may interact with the sample S in such a manner that it emits various types of radiation from the sample S. Such radiation may include secondary charged particles (e.g., secondary electrons), backscattered charged particles (e.g., backscattered electrons), X-rays, and / or optical radiation (e.g., cathode-ray luminescence). One or more of these radiation types, or other radiation types, may be detected by the detector 22. In some embodiments, the detector 22 may include, for example, a scintillator / photomultiplier tube or EDX detector combination. Alternatively or additionally, charged particles may traverse the sample S, emerge from the sample S, and continue propagating along axis 8 (with substantially, but generally, some deflection / scattering). Such transmitted electrons may enter an imaging system 24 that acts as a combined objective / projection lens and may, as appropriate, include various electrostatic and / or magnetic lenses, deflectors, correctors (e.g., astigmatism correctors), etc. In non-scanning mode, the imaging system 24 can focus transmitted electrons onto the fluorescent screen 26, which can be retracted (as schematically shown by arrow 28) or otherwise pulled back, if desired, so as to move away from axis 8. An image of a portion of the specimen S may be formed on the screen 26 by the imaging system 24, which can be viewed through an observation port 30 located in a preferred part of the enclosure 2 of the CPM 1. The retraction mechanism for the screen 26 may be, for example, essentially mechanical and / or electrical, and is not shown herein.

[0015] Alternatively or in addition to displaying an image on the screen 26, a charged particle detector D may be used. In such embodiments, the adjuster lens 24' can shift the focus of charged particles emerging from the imaging system 24 and redirect them towards the charged particle detector D (rather than on the retracted plane of the screen 26 as considered above). In the charged particle detector D, charged particles can form an image (e.g., a diffraction pattern), which can be processed by the controller 50 and displayed on the display device 120. In STEM mode, the output from the detector D can be recorded as a function of the (x, y) scanning beam position and the tilt angle of the sample S, and an image, which is a map of the detector output, can be constructed. Generally, the CPM1 may include one or more detectors arranged as desired. Examples of such detectors include, in particular, photomultiplier tubes (e.g., solid-state photomultiplier tubes), photodiodes, complementary metal-oxide-semiconductor (CMOS) detectors, charge-coupled device (CCD) detectors, and photocells used in combination with scintillator films. This disclosure uses the term “image” to refer to a set of data generated by one or more detectors of a CPM, and such image may include a scalar value at each pixel, a vector value at each pixel, or any other preferred arrangement of information.

[0016] The controller 50 may be connected to various illustrative components via control lines 50' (e.g., buses). The controller 50 may provide various functions such as synchronizing actions, providing setpoints, processing signals, performing calculations, and displaying messages / information on the message / information display device 120. While the controller 50 is shown in Figure 1 as being inside the enclosure 2 of the CPM 1, this is merely illustrative; the controller 50 may be located inside or outside the enclosure 2, or distributed among components both inside and outside the enclosure 2. For example, in some embodiments, some operations of the controller 50 may be performed by hardware located inside the enclosure 2, while other operations of the controller 50 may be performed by hardware located outside the enclosure 2 (e.g., a computing device such as a laptop or desktop computer).

[0017] Figure 2 is a block diagram of a CPM data acquisition module 1000 for performing data acquisition in various embodiments. The CPM data acquisition module 1000 may be part of the controller 50 of the CPM system 100 in Figure 1. The CPM data acquisition module 1000 may be implemented by circuitry (including, for example, electrical and / or optical components) such as a programmed computing device. The logic of the CPM data acquisition module 1000 may be contained in a single computing device or distributed across multiple computing devices communicating with each other as appropriate. Examples of computing devices that can implement the CPM data acquisition module 1000, either individually or in combination, are described with reference to the computing device 4000 in Figure 9, and examples of a system of interconnected computing devices in which the CPM data acquisition module 1000 can be implemented across one or more computing devices are described herein with reference to the scientific instrument support system 5000 in Figure 10.

[0018] The CPM data acquisition module 1000 may include an imaging logic 1002, a mask logic 1004, a training logic 1006, a user interface (UI) logic 1008, and a reconstruction logic 1010. As used herein, the term "logic" may include an apparatus that executes a set of operations associated with the logic. For example, any of the logic elements included in the CPM data acquisition module 1000 may be implemented by one or more computing devices programmed with instructions that cause one or more processing devices of the computing device to execute a series of operations. In certain embodiments, the logic elements may include one or more non-transitory computer-readable media, and the one or more non-transitory computer-readable media have instructions that cause one or more computing devices to execute a set of associated operations when executed by one or more processing devices of the one or more computing devices. As used herein, the term "module" may refer to a collection of one or more logic elements that together perform the functions associated with the module. Different ones of the logic elements within a module may take the same form or different forms. For example, some of the logic within a module may be implemented by a programmed general-purpose processing device, and other logic within the module may be implemented by an application-specific integrated circuit (ASIC). In another example, different logic elements of the logic elements within a module may be associated with different sets of instructions executed by one or more processing devices. A module may not include all of the logic elements depicted in the associated drawings. For example, a module may include a subset of the logic elements depicted in the associated drawings when the module executes a subset of the operations discussed herein with reference to that module.

[0019] The imaging logic 1002 may cause a CPM (e.g., CPM1 in Figure 1) to generate an image of a portion of a specimen (e.g., specimen S in Figure 1). For example, the imaging logic 1002 may generate images of different parts of the specimen at different depths (e.g., having different values ​​in the (z) direction). The imaging logic 1002 may be configured to capture different types of images. For example, in some embodiments, the imaging logic 1002 may be configured to capture low-resolution and high-resolution images. The terms “high-resolution” and “low-resolution” are used here in relative terms, indicating that the resolution or other informational content of a high-resolution image is greater than the resolution or other informational content of a low-resolution image. A low-resolution image of a specimen may require a lower radiation dose and / or acquisition time than a high-resolution image of a specimen. In some embodiments, the imaging logic 1002 may be configured to use different hardware (e.g., one or more different detectors) to capture higher-resolution images than lower-resolution images. In some embodiments, as will be further described below, the imaging logic 1002 causes the CPM to generate a low-resolution image of a portion of the specimen, which may then be used (e.g., by the mask logic 1004) to generate a mask, which is applied when the imaging logic 1002 causes the CPM to generate a high-resolution image of a portion of the specimen, thereby capturing only a subset of the CPM's field of view in high resolution. In some embodiments, the images generated by the CPM at the direction of the imaging logic 1002 may include bright-field images, annular bright-field (ABF) images, integrated differential phase-contrast (iDPC) images, or high-angle annular dark-field (HAADF) images. Figure 3A shows a graphic representation 102 of an image of a portion of the specimen (e.g., a "low-resolution" image) that may be generated by the imaging logic 1002.

[0020] The mask logic 1004 may receive an image of a portion of a sample (e.g., sample S in Figure 1) (e.g., from the imaging logic 1002) and, based on the received image, may generate a relevant mask for subsequent imaging by the CPM (e.g., CPM1 in Figure 1). As used herein, the “mask” can be a dataset that indicates to the CPM (e.g., CPM1 in Figure 1) which areas within the full field of view of the portion of the sample will be imaged in the subsequent imaging operation. For example, the mask may indicate which squares within the “full-frame” grid of the field of view will be imaged in the subsequent imaging operation. Thus, the mask may correspond to a subset of the full-frame field of view of the CPM when imaging a portion of the sample. In some embodiments, the mask logic 1004 may identify a region of interest in the received low-resolution image (generated by the imaging logic 1002) of the portion of the sample and generate a mask that identifies the region of interest as the area to be imaged by the CPM in subsequent high-resolution imaging of the portion of the sample. Figure 3B shows a graphic representation 104 of an exemplary set of regions of interest (bordered in white) that the mask logic 1004 can identify in the graphic representation 102 of Figure 3A. Regions of interest may include features of interest in the graphic representation 102, and the mask logic 1004 can identify regions of interest through initial manual input by the user, followed by the application of machine learning techniques, as will be further described below. The graphic representation 104 may correspond to a mask for subsequent imaging of the portion of the sample imaged in the graphic representation 102, and can be generated by the mask logic 1004. In particular, in the graphic representation 104, the white-bordered regions indicate areas in the field of view of the CPM that are imaged in later rounds of imaging of the portion of the sample (e.g., high-resolution imaging), and the black regions indicate areas in the field of view of the CPM that are not imaged in later rounds of imaging. In graphic representation 104, the region of interest may precisely correspond to the imaged area of ​​the mask; however, in other embodiments, the mask associated with a set of regions of interest may be larger than the combined region of interest (for example, by a fixed number or percentage of pixels around each individual region of interest).Note that the graphic representations 102 and 104 in FIGS. 3A and 3B are each associated with a particular portion of the specimen. Graphic representations such as graphic representations 102 and 104 can be generated for each of a plurality of portions of the specimen (e.g., at different depths of the specimen).

[0021] The mask generated by the mask logic 1004 may indicate to the CPM that the portion of the field of view corresponding to the region of interest is imaged in an imaging operation after a portion of the specimen, and the portion of the field of view that does not correspond to the region of interest is not imaged in a subsequent imaging operation. Reducing the area of the field of view to be imaged can reduce the radiation to which the specimen is exposed and can reduce the acquisition time of subsequent imaging operations compared to an imaging operation in which the entire field of view is imaged. In some embodiments, the mask generated by the mask logic may have a higher resolution than the low-resolution image used by the mask logic 1004 to generate the mask.

[0022] The mask logic 1004 may identify a region of interest in a received image using machine learning techniques that use images of a portion of a sample and user identification of the region of interest within the image as training data. In particular, at the start of imaging a particular sample, the mask logic 1004 may perform a first low-resolution imaging round of a first portion of the sample and then provide a graphic representation of the low-resolution image corresponding to the user (e.g., via the UI logic 1008, which will be described further below). The user may then annotate the graphic representation of the low-resolution image of the first portion of the sample with instructions indicating which region of the low-resolution image is the region of interest (e.g., by "drawing" the white border of the graphic representation 104 in Figure 3B on the graphic representation 102 in Figure 3A). The mask logic 1004 may provide the training logic 1006 (further discussed below) with a low-resolution image of a first portion of the sample and the identification of regions of interest in the low-resolution image of the first portion of the sample, which the training logic 1006 may train a machine learning computational model (e.g., a neural network model previously trained on images that may not be CPM images, as further discussed below) to identify regions of interest in subsequently acquired low-resolution images of other portions of the sample (e.g., portions of the sample at different depths (z)). The mask logic 1004 may use the regions of interest generated by the machine learning computational model to generate a mask associated with the portion of the sample (similar to that of the graphic representation 104 in Figure 3B). The regions of interest identified by the machine learning computational model may precisely correspond to the imaged regions of the mask generated by the mask logic 1004, although in other embodiments, the mask associated with the set of regions of interest generated by the machine learning computational model may be larger than the combined region of interest (e.g., by a fixed number or percentage of pixels around individual regions of interest). In some embodiments, if the sample contains multiple different types of features of interest (e.g., different components of living cells), the mask logic 1004 may use multiple corresponding machine learning computational models to recognize each of the features of interest in a low-resolution image of the sample.In other embodiments, a single machine learning computation model may be trained to recognize multiple different features of interest. For ease of consideration, a single “machine learning computation model” may be considered herein as part of the mask logic 1004, but the mask logic 1004 may implement any desired number of machine learning computation models (e.g., corresponding to different features of interest in the sample).

[0023] In some embodiments, the user may manually identify regions of interest (corresponding to features of interest) only in a single low-resolution image of a portion of the sample, or in only a few images (e.g., fewer than 10) of different parts of the sample, and this manual identification is provided to the training logic 1006 to train the machine learning computational model before the mask logic 1004 can successfully use the trained machine learning computational model to identify similar features of interest in other parts of the sample. This may be in contrast to conventional machine learning approaches, which require hundreds or thousands of manually annotated, or otherwise previously annotated, images of a particular sample or feature of interest in order to successfully perform an image segmentation task with that sample or feature of interest. Instead, the mask logic 1004 may use a machine learning computational model pre-trained for general-purpose image recognition using a training corpus that does not contain images of the sample. For example, in some embodiments, where the sample contains a particular biological sample, the training corpus may not contain images of that biological sample or similar biological samples, but instead may contain substantially different images (e.g., images of traffic lights, images of bicycles, etc.). Training a machine learning computation model previously trained for general image recognition and segmentation (e.g., using a publicly available image dataset entirely different from CPM images) on a single or a few CPM images annotated with regions of interest can yield acceptable performance in identifying similar regions of interest in other CPM images of a sample, especially if the machine learning computation model is trained with an error function that prioritizes over-identification of regions of interest rather than under-identification (i.e., priority for "false positives" rather than "false negatives"). Such prioritization can be advantageous even in settings where, for example, the previous portion of a sample may be milled or otherwise removed to image the next portion of the sample, and therefore there may be no opportunity to re-image the previous portion (e.g., in an automated slice-and-view volume acquisition setting).Training conventional machine learning computational models represents a significant burden for users and slows the adoption of machine learning techniques. Therefore, a mask logic 1004 that includes a machine learning computational model that generates an overly inclusive mask (i.e., a mask indicating which regions have desired features and which do not) can bring about an overall technical improvement in CPM imaging, significantly reducing the user burden (annotating one or a few low-resolution images) while achieving many of the advantages of selective high-resolution imaging (e.g., reduced radiation dose and acquisition time, as described above).

[0024] After the mask logic 1004 generates a mask associated with a portion of the sample (based on a low-resolution image of the portion of the sample, as described above), the imaging logic 1002 may use the mask to perform a high-resolution imaging round of the portion of the sample. During this high-resolution imaging round, only a subset of the field of view of the portion of the sample may be imaged. Figure 3C shows a graphic representation 106 of a high-resolution image of the portion of the sample imaged (at low resolution) in Figure 3A, corresponding to the mask of the graphic representation 104 in Figure 3B. In some embodiments, this high-resolution imaging round may include features of interest in the portion of the sample, as well as regions of the portion of the sample that do not include features of interest (for example, because the machine learning computation model implemented by the mask logic 1004 is trained to preferentially generate false positives rather than false negatives). In some embodiments, the imaging logic 1002 (or UI logic 1008) may create a combined image for a portion of a sample by rescaling (e.g., resampling) a low-resolution image, subtracting masked portions (e.g., as shown in graphic representation 108 of Figure 3D), and then superimposing a high-resolution image onto the result to form a combined image (e.g., as shown in graphic representation 110 of Figure 3E) containing low-resolution and high-resolution information about the portion of the sample.

[0025] If two different parts of a sample are "adjacent" to each other (for example, adjacent milled or mechanically sliced ​​parts, or adjacent angle captures in a tilted series), then the low-resolution images of these parts are expected to be similar, such as the mask generated by the mask logic 1004. If the low-resolution images of these parts, or the associated masks, are substantially different, the mask logic 1004 may use this condition to determine that it is unlikely that a trained machine learning computation model would produce an acceptablely correct output for both parts, and therefore the mask logic 1004 may perform additional or alternative actions to generate a mask for one or more of the parts. In particular, in some embodiments, the mask logic 1004 may compare the low-resolution images generated by the imaging logic 1002, or the masks generated by the mask logic 1004 based on the output of a machine learning computation model, for adjacent or otherwise physically close parts of the sample, and determine whether the differences between the low-resolution images (or corresponding masks) meet one or more difference criteria. When comparing masks of different parts of a sample, the difference criteria may include the difference in the percentage area of ​​the regions of interest of the different parts above a threshold, the amount of overlap of the regions of interest of the different parts below a threshold, and / or other preferred difference criteria. When comparing low-resolution images of different parts of a sample, the difference criteria may include comparing any preferred image similarity metric to a threshold (e.g., if the mean squared intensity difference between the low-resolution images exceeds a threshold). If the difference between low-resolution images (or corresponding masks) of two physically adjacent parts of a sample satisfies one or more such difference criteria, the mask logic 1004 may perform one or more modification actions, such as prompting the user to annotate one or both of the low-resolution images (and then providing the newly annotated images to the training logic 1006 for retraining the machine learning computation model), increasing the size of the regions of interest in one or more of the masks to increase the likelihood of capturing features of interest, evaluating one or more of the proposed masks and accepting, rejecting, or modifying the proposed masks, or prompting the user for any other preferred modification or mitigation action.

[0026] As described above, the training logic 1006 may be configured to train the machine learning computation model of the mask logic 1004 on a set of training data. The training logic 1006 may also be configured to retrain the machine learning computation model when additional training data is received. As is known in the art, the training data may include a set of input / output pairs (e.g., one or more pairs of input low-resolution images of a portion of a sample and corresponding regions of interest in the low-resolution images of a portion of the sample), and the training logic 1006 may use this training data to train the machine learning computation model according to any preferred technique (e.g., by adjusting the weights and other parameters of the machine learning computation model). Any preferred machine learning computation model may be used, such as a neural network model. For example, the mask logic 1004 may implement a multilayer neural network model, such as a convolutional neural network model (e.g., a ResNet model such as ResNet-50). In some embodiments, Mask Logic 1004 may implement a video object segmentation model that includes a multilayer neural network model, such as the video object segmentation model described by Liang et al., “Video Object Segmentation with Adaptive Feature Bank and Uncertain-Region Refinement,” 34th Conference on Neural Information Processing Systems (NeurlPS2020), Vancouver, Canada. The machine learning computation model of Mask Logic 1004 may be trained for object recognition / segmentation using a general corpus of images, such as the DAVIS17 dataset and / or the YouTube®-VOS18 dataset, prior to imaging specific samples by CPM.

[0027] As described above, in some embodiments, the user may manually identify regions of interest (corresponding to features of interest) only in a single low-resolution image of the sample, or only in a small number of images of the sample (e.g., fewer than 10), and this manual identification is provided to the training logic 1006 to train the machine learning computation model before the mask logic 1004 can successfully use the trained machine learning computation model to identify similar features of interest in other parts of the sample. This is especially true when different parts of a sample are physically close to each other, such that the features of one part are very similar to the features of an adjacent part (similar to adjacent frames in video object segmentation). For example, a first part of a sample may represent a first plane passing through the sample (e.g., a plane formed by mechanically slicing or milling the sample), and a second part of the sample may represent a second plane passing through the sample (e.g., another plane formed by mechanically slicing or milling the sample). In some embodiments, such parallel planes may be spaced apart by a distance of 1 to 30 microns (e.g., 1 to 10 microns if the plane is formed by milling, or 10 to 30 microns if the plane is formed by mechanical slicing), and may represent planes spaced apart in the (z) direction. For example, Figure 4A shows a graphic representation 112 of a low-resolution image of a portion of a sample having two different types of features of interest manually identified by a user (e.g., via UI logic 1008), and Figure 4B shows a graphic representation 114 of a low-resolution image of adjacent portions of a sample having two different types of regions of interest (corresponding to the two different types of features of interest in Figure 4A) generated by the machine learning computation model of the mask logic 1004 after the training logic 1006 has trained the machine learning computation model in the manually annotated low-resolution image of Figure 4A (e.g., planes of the sample parallel to the plane of graphic representation 112).The features of the sample portion shown in Figure 4B may be very similar to the features of the sample portion shown in Figure 4A due to their proximity in the sample (e.g., 1 to 30 microns apart), and the over-inclusion error of the machine learning computation model of the mask logic 1004 may result in the identification of a region of interest in the graphic representation 114 of Figure 4B, which substantially includes the features of interest identified in Figure 4A and additional regions of the sample portion associated with Figure 4B.

[0028] In some embodiments, the training logic 1006 may retrain the machine learning computation model of the mask logic 1004 when a retraining condition is met. For example, in some embodiments, the training logic 1006 may retrain the machine learning computation model of the mask logic 1004 when a threshold number of "new" training datasets (e.g., 20 training datasets) are accumulated. In some embodiments, the training logic 1006 may retrain the machine learning computation model using available retraining datasets when it receives a retraining command from a user (e.g., via a GUI such as GUI3000 in Figure 8). In some embodiments, the training logic 1006 may retrain the machine learning computation model when one or more performance metrics of the machine learning computation model (e.g., errors on one or more validation datasets) meet one or more retraining criteria (e.g., errors increase the threshold amount from the previous training round). In some embodiments, the retraining condition may include any one or more of these conditions. New retraining data can be generated by providing the user with one or more additional low-resolution images of a portion of the sample (e.g., via UI logic 1008) and asking the user to identify regions of interest in the low-resolution images. In some embodiments, retraining of the machine learning computation model can be done on a calendar schedule (e.g., daily or weekly) and / or after processing a certain number of low-resolution images (e.g., every 100 images). Note that in some embodiments, the mask may be a "negative mask" in which areas of the field of view should not be imaged. In some such embodiments, the machine learning computation model of the mask logic 1004 is trained to recognize features of no interest, and the "negative" mask corresponds to the regions of those features.

[0029] The UI logic 1008 can provide information to the user and receive input from the user (for example, via a GUI such as GUI 3000, which is discussed below with reference to Figure 8). In some embodiments, the UI logic 1008 can cause at least some graphic representations of low-resolution images related to a portion of a sample to be displayed on a display device (for example, any of the display devices discussed herein). For example, Figures 3A and 3B are graphic representations 102 and 104, respectively, which, according to various embodiments, may be provided to the user via a display device (and a GUI such as GUI 3000, which is discussed below with reference to Figure 8) as part of a data acquisition technique disclosed herein.

[0030] In some embodiments, the UI logic 1008 may cause at least several graphic representations of high-resolution images related to a portion of the sample (e.g., graphic representation 106 in Figure 3C) to be displayed on a display device (e.g., any of the display devices considered herein). In some embodiments, the UI logic 1008 may cause one or more combined graphic representations of low-resolution / high-resolution images (e.g., graphic representation 110 in Figure 3E) to be displayed on the display device.

[0031] The UI logic 1008 may request and receive user input, such as user annotations on low-resolution images, as described herein. In some embodiments, the UI logic 1008 may display one or more performance metrics of the machine learning computation model of the mask logic 1004 (e.g., a plot of the rate of user annotation requests over time, the number of low-resolution images processed by the mask logic 1004 without requesting additional user annotations, etc.). Other preferred methods for displaying performance metrics of the machine learning computation model may be used.

[0032] The imaging logic 1002 may provide low-resolution and high-resolution images generated by CPM for different parts of the sample for further processing by the reconstruction logic 1010, which can generate a three-dimensional reconstruction of part or all of the sample based on the images. In some embodiments, the imaging logic 1002 may provide the images directly to the reconstruction logic 1010 (for example, when the imaging logic 1002 and the reconstruction logic 1010 are implemented as part of a common software package and / or run on a common computing device), while in other embodiments, the imaging logic 1002 may provide the images in an intermediate form that can later be provided to the reconstruction logic 1010. An example of this latter embodiment may include the imaging logic 1002 exporting the images to a storage device (for example, a networked storage device or a physical storage device such as a Universal Serial Bus (USB) stick) that can later be accessed by the reconstruction logic 1010. In some embodiments, the imaging logic 1002 may be contained in a separate software package from the software package containing the reconstruction logic 1010. In some embodiments, the CPM data acquisition module 1000 may provide an automated slice-and-view volume acquisition tool for CPM or a tilted series volume acquisition tool for CPM.

[0033] As described above, the reconstruction logic 1010 can generate a three-dimensional reconstruction of the specimen using the image generated by the imaging logic 1002. The reconstruction logic 1010 can use any suitable known technique for this reconstruction. For example, in various embodiments, the reconstruction logic 1010 can use the image to perform tomography reconstruction, weighted back projection (WBP), simultaneous iterative reconstruction technique (SIRT), HAADF energy-dispersive spectroscopy (EDS) bimodal tomography (HEBT) technique, conjugate gradient least squares (CGLS) technique, expectation value maximization (EM) technique, contemporary numerical reconstruction technique (SART), diffraction tomography technique, or a combination thereof.

[0034] Figures 5, 6, and 7 are flowcharts of methods 2000, 2100, and 2200, respectively, for performing CPM data acquisition according to various embodiments. The operation of methods 2000, 2100, and 2200 can be illustrated by referring to specific embodiments disclosed herein (e.g., the CPM data acquisition module 1000 discussed herein with reference to Figure 2, the GUI 3000 discussed herein with reference to Figure 8, the computing device 4000 discussed herein with reference to Figure 9, and / or the scientific instrument support system 5000 discussed herein with reference to Figure 10), but methods 2000, 2100, and 2200 can be used in any preferred configuration to perform any preferred data acquisition or other support operation. The operations are shown once each in a specific order in Figures 5-7, but the operations can be appropriately rearranged and / or repeated as desired (e.g., different operations to be performed can be performed in parallel as appropriate).

[0035] In method 2000 of Figure 5, in 2002, the CPM can be made to generate a single image of the first portion of the sample. For example, the imaging logic 1002 of the CPM data acquisition module 1000 can perform the operation in 2002 (e.g., to generate a low-resolution image of a portion of the sample).

[0036] In 2004, a first mask may be generated based on one or more regions of interest provided by user annotations on a single image. For example, the mask logic 1004 of the CPM data acquisition module 1000 may perform the operation of 2004 in response to user annotations received via the UI logic 1008.

[0037] In 2006, a machine learning computation model may be trained using a single image and one or more regions of interest. For example, the training logic 1006 of the CPM data acquisition module 1000 may perform the operation of 2006 to train a machine learning computation model on a single image and its associated annotations. As described above, in some embodiments, the machine learning computation model may be pre-trained on images that are not of a sample.

[0038] In 2008, multiple images of corresponding parts of a sample can be generated. For example, the imaging logic 1002 of the CPM data acquisition module 1000 can perform the operation of 2008 (e.g., to acquire images of consecutively "adjacent" parts of a sample, such as milled or mechanically sliced ​​planes or adjacent angles in a series of inclinations).

[0039] In 2010, multiple corresponding masks based on multiple images of multiple parts of a sample can be generated using a trained machine learning computation model without retraining. For example, the mask logic 1004 of the CPM data acquisition module 1000 can perform the operation of 2010 (e.g., to generate masks related to different parts of a sample without retraining the machine learning computation model).

[0040] In method 2100 of Figure 6, 2102 can cause the charged particle microscope to generate a single image of a first portion of the specimen. For example, the imaging logic 1002 of the CPM data acquisition module 1000 can perform the operation of 2102 (e.g., to generate a low-resolution image of a portion of the specimen).

[0041] In 2104, a first mask may be generated based on one or more regions of interest indicated by user annotations on a single image, where the regions of interest include features of interest within the sample. For example, the mask logic 1004 of the CPM data acquisition module 1000 may perform the operation of 2004 in response to user annotations received via the UI logic 1008.

[0042] In 2106, the machine learning computation model may be trained using a single image and one or more regions of interest. For example, the training logic 1006 of the CPM data acquisition module 1000 may perform the operation of 2106 to train the machine learning computation model on a single image and its associated annotations. As described above, in some embodiments, the machine learning computation model may be pre-trained on images that are not of a sample.

[0043] In 2108, the charged particle microscope may be made to generate an image of a second portion of the specimen, the second portion of the specimen being adjacent to the first portion of the specimen. For example, the imaging logic 1002 of the CPM data acquisition module 1000 may perform the operation of 2108 (e.g., to acquire images of "adjacent" portions of the specimen, such as adjacent angles in a milled or mechanically sliced ​​plane or a series of inclinations).

[0044] In 2110, a second mask may be generated based on the image of a second part of the sample using a trained machine learning computation model, where the second mask images the region of the second part of the sample containing the feature of interest and the region of the second part of the sample not containing the feature of interest. For example, the mask logic 1004 of the CPM data acquisition module 1000 may perform the operation of 2110 (e.g., to generate a mask related to another part of the sample without retraining the machine learning computation model, such a mask would be "over-inclusive" as described above).

[0045] In method 2200 shown in Figure 7, a first dataset related to a first portion of a specimen may be generated in 2202 by processing data from a first imaging round of the first portion using a charged particle microscope. For example, the imaging logic 1002 of the CPM data acquisition module 1000 may perform the operation of 2202 (e.g., to generate a low-resolution image of a portion of the specimen).

[0046] In 2204, a first mask may be generated relating to a first portion of the sample, and the generation of the first mask is based on user identification of one or more first regions of interest in a first dataset relating to the first portion of the sample. For example, the mask logic 1004 of the CPM data acquisition module 1000 may perform the operation of 2204 in response to user annotations received via the UI logic 1008.

[0047] In 2206, a machine learning computation model can be trained using a first portion of a sample and a first dataset associated with one or more first regions of interest. For example, the training logic 1006 of the CPM data acquisition module 1000 can perform the operation of 2206 to train a machine learning computation model. In some embodiments, training a machine learning computation model in 2206 may involve a single input / output pair (e.g., based on a single annotated low-resolution image) or a small number of input / output pairs (e.g., based on 10 or fewer annotated low-resolution images).

[0048] In 2208, a first dataset related to the second part of the specimen can be generated by processing data from the first imaging round of the second part using a charged particle microscope. For example, the imaging logic 1002 of the CPM data acquisition module 1000 can perform the operation of 2208 (e.g., to generate a low-resolution image of another part of the specimen).

[0049] In 2210, a second mask associated with the second part of the sample can be generated using a trained machine learning computation model and a first dataset associated with the second part of the sample. For example, the mask logic 1004 of the CPM data acquisition module 1000 can perform the operation of 2210.

[0050] In 2212, a second dataset related to the first portion of a specimen can be generated by processing data from a second imaging round of the first portion according to a first mask, using a charged particle microscope. For example, the imaging logic 1002 of the CPM data acquisition module 1000 can perform the operation of 2212 (e.g., to generate a high-resolution image of the first portion of the specimen according to the first mask).

[0051] In 2214, if the difference between the first mask and the second mask, or the difference between the first dataset associated with the first part of the sample and the first dataset associated with the second part of the sample, satisfies one or more difference criteria, the second mask may be adjusted before the second dataset associated with the second part of the sample is generated according to the second mask. For example, the mask logic 1004 of the CPM data acquisition module 1000 may perform the operation of 2214 (e.g., to evaluate the difference between the mask and / or the low-resolution images of the sample parts and adjust one or more of the masks).

[0052] In 2216, a second dataset related to the second part of the specimen can be generated by processing data from a second imaging round according to a second mask of the second part using a charged particle microscope. For example, the imaging logic 1002 of the CPM data acquisition module 1000 can perform the operation of 2216 (e.g., to generate a high-resolution image of the second part of the specimen according to a second mask).

[0053] The CPM data acquisition methods disclosed herein may include interaction with a human user (e.g., via a user-local computing device 5020, as discussed herein with reference to Figure 10). These interactions may include providing the user with information (e.g., information about a CPM such as the CPM 5010 in Figure 10, a specimen being analyzed, or information about other tests or measurements performed by the CPM or other scientific instruments, information retrieved from local or remote databases, or other information), or providing the user with options to input commands (e.g., to control the operation of a scientific instrument such as the CPM 5010 in Figure 10, or to control the analysis of data generated by the CPM or other scientific instruments), queries (e.g., to local or remote databases), or other information. In some embodiments, these interactions may be performed via a graphical user interface (GUI) that includes a visual display on a display device (e.g., display device 4010, discussed herein with reference to Figure 9) which prompts the user to provide outputs and / or inputs (e.g., via one or more input devices such as a keyboard, mouse, trackpad, or touchscreen, included in other I / O devices 4012, discussed herein with reference to Figure 9). The CPM data acquisition systems, and more generally scientific instrument support systems, disclosed herein may include any preferred GUI for interacting with the user.

[0054] Figure 8 shows an exemplary GUI 3000 that may be used to perform some or all of the CPM data acquisition methods disclosed herein, according to various embodiments. As described above, the GUI 3000 may be provided on a display device (e.g., display device 4010, as discussed herein with reference to Figure 9) of a computing device (e.g., computing device 4000, as discussed herein with reference to Figure 9) of a scientific instrument support system (e.g., scientific instrument support system 5000, as discussed herein with reference to Figure 10), and a user may interact with the GUI 3000 using any suitable input device (e.g., any of the input devices included in other I / O device 4012, as discussed herein with reference to Figure 9), and input techniques (e.g., cursor movement, motion capture, face recognition, gesture detection, voice recognition, button activation, etc.).

[0055] GUI3000 may include a data display area 3002, a data analysis area 3004, a CPM control area 3006, and a setting area 3008. The specific number and arrangement of areas shown in Figure 8 are merely illustrative, and any number and arrangement of areas containing any desired features may be included in GUI3000.

[0056] The data display area 3002 may display data generated by a CPM (for example, a CPM 5010 discussed herein with reference to Figure 10). For example, the data display area 3002 may display any of the low-resolution images, high-resolution images, or combined images disclosed herein (for example, any of the graphic representations 102 to 110 in Figures 3A to 3E).

[0057] The data analysis area 3004 may display the results of data analysis (e.g., the results of analyzing the data illustrated in the data display area 3002 and / or other data). For example, the data analysis area 3004 may display a region of interest indicated by the user, a region of interest generated by the machine learning computation model of the mask logic 1004, and / or a mask generated by the mask logic 1004 (e.g., one of the graphic representations 112 and 114 in Figures 4A-4B). In some embodiments, the data display area 3002 and the data analysis area 3004 may be combined in the GUI 3000 (e.g., to include data output from scientific instruments and some analysis of the data in a common graph or area).

[0058] The CPM control region 3006 may include options that allow the user to control the scientific instrument (e.g., the CPM 5010 discussed herein with reference to Figure 10). For example, the CPM control region 3006 may include user-selectable options to perform other control functions, such as manually annotating low-resolution images to serve as training data; retraining a machine learning computation model; accepting, rejecting, or modifying proposed masks generated by the mask logic 1004 to control areas to be imaged by the high-resolution imaging round.

[0059] The configuration area 3008 may include options that allow the user to control the features and functions of GUI 3000 (and / or other GUIs) and / or perform common computing operations relating to the data display area 3002 and the data analysis area (3004) (for example, saving data on a storage device such as the storage device 4004 discussed herein with respect to Figure 9, sending data to another user, labeling data, etc.).

[0060] As described above, the CPM data acquisition module 1000 can be implemented by one or more computing devices. Figure 9 is a block diagram of a computing device 4000 that can perform some or all of the CPM data acquisition methods and other scientific instrument-assisted methods disclosed herein, according to various embodiments. In some embodiments, the CPM data acquisition module 1000 can be implemented by a single computing device 4000 or multiple computing devices 4000. Furthermore, as described below, the computing device 4000 (or multiple computing devices 4000) implementing the CPM data acquisition module 1000 may be part of one or more of the CPM 5010, the user-local computing device 5020, the service-local computing device 5030, or the remote computing device 5040 in Figure 10.

[0061] Although the computing device 4000 in Figure 9 is shown having several components, one or more of these components may be omitted or duplicated to suit the application and configuration. In some embodiments, some or all of the components included in the computing device 4000 may be mounted on one or more motherboards and enclosed in a housing (e.g., including plastic, metal, and / or other materials). In some embodiments, some of these components may be manufactured on a single system-on-a-chip (SoC) (e.g., the SoC may include one or more processing devices 4002 and one or more storage devices 4004). Furthermore, in various embodiments, the computing device 4000 may not include one or more components shown in Figure 9, but may include interface circuits (not shown) for coupling with one or more components using any preferred interface (e.g., a Universal Serial Bus (USB) interface, a High Definition Multimedia Interface (HDMI®) interface, a Controller Area Network (CAN) interface, a Serial Peripheral Interface (SPI) interface, an Ethernet interface, a wireless interface, or any other preferred interface). For example, the computing device 4000 may not include the display device 4010, but may include a display device interface circuit (e.g., a connector and driver circuit) to which the display device 4010 can be coupled.

[0062] The computing device 4000 may include processing devices 4002 (e.g., one or more processing devices). As used herein, the term “processing device” may mean any device or part of a device that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. The processing device 4002 may include one or more digital signal processors (DSPs), application-specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptographic processors (dedicated processors that execute cryptographic algorithms in hardware), server processors, or any other suitable processing devices.

[0063] The computing device 4000 may include a storage device 4004 (e.g., one or more storage devices). The storage device 4004 may include one or more memory devices such as random access memory (RAM) (e.g., static RAM (SRAM) devices, magnetic RAM (MRAM) devices, dynamic RAM (DRAM) devices, resistive RAM (RRAM) devices, or conductive bridge RAM (CBRAM) devices), hard drive-based memory devices, solid-state memory devices, network drives, cloud drives, or any combination of memory devices. In some embodiments, the storage device 4004 may include memory that shares a die with the processing device 4002. In such embodiments, the memory may be used as cache memory and may include, for example, embedded dynamic random access memory (eDRAM) or spin-transfer torque magnetic random access memory (STT-MRAM). In some embodiments, the storage device 4004 may include a non-temporary computer-readable medium having instructions that, when executed by one or more processing devices (e.g., processing device 4002), cause the computing device 4000 to execute any suitable method or part thereof from those disclosed herein.

[0064] The computing device 4000 may include an interface device 4006 (for example, one or more interface devices 4006). The interface device 4006 may include one or more communication chips, connectors, and / or other hardware and software to manage communication between the computing device 4000 and other computing devices. For example, the interface device 4006 may include circuitry that manages wireless communication for transferring data to and from the computing device 4000. The term “wireless” and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communication channels, etc., that can communicate data through the use of modulated electromagnetic radiation over a non-solid medium. This term does not mean that the devices in question are wire-free, although in some embodiments they may be wire-free. The circuitry included in interface device 4006 for managing wireless communication may implement any of several wireless standards or protocols, including, but not limited to, Institute of Electrical and Electronics Engineers (IEEE) standards, including Wi-Fi (IEEE 802.11 family), IEEE 802.16 standards (e.g., IEEE 802.16-2005 amendment), and Long Term Evolutionary (LTE) projects with any modifications, updates, and / or revisions (e.g., Advanced LTE project, Ultra-Mobile Broadband (UMB) project (also known as "3GPP®2")). In some embodiments, the circuitry included in interface device 4006 for managing wireless communication may operate according to a Global System for Mobile Communications (GSM), General Purpose Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE network. In some embodiments, the circuitry included in the interface device 4006 for managing wireless communication may operate according to GSM Evolutionary High-Speed ​​Data (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolutionary UTRAN (E-UTRAN).In some embodiments, the circuitry included in the interface device 4006 for managing wireless communication may operate in accordance with Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Extended Cordless Communications (DECT), Evolutionary Data Optimization (EV-DO), and their derivatives, and any other wireless protocols designated as 3G, 4G, 5G, and later. In some embodiments, the interface device 4006 may include one or more antennas (e.g., one or more antenna arrays) for receiving and / or transmitting wireless communications.

[0065] In some embodiments, the interface device 4006 may include circuitry for managing wired communications, such as electrical, optical, or any other preferred communication protocol. For example, the interface device 4006 may include circuitry to assist communications according to Ethernet technology. In some embodiments, the interface device 4006 may support both wireless and wired communications, and / or support multiple wired communication protocols and / or multiple wireless communication protocols. For example, a first set of circuitry in the interface device 4006 may be dedicated to short-range wireless communications such as Wi-Fi or Bluetooth, and a second set of circuitry in the interface device 4006 may be dedicated to long-range wireless communications such as Global Positioning System (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, or others. In some embodiments, a first set of circuitry in the interface device 4006 may be dedicated to wireless communications, and a second set of circuitry in the interface device 4006 may be dedicated to wired communications.

[0066] The computing device 4000 may include a battery / power circuit 4008. The battery / power circuit 4008 may include one or more energy storage devices (e.g., batteries or capacitors) and / or circuits for coupling components of the computing device 4000 to an energy source separate from the computing device 4000 (e.g., AC line power).

[0067] The computing device 4000 may include a display device 4010 (for example, multiple display devices). The display device 4010 may include any visual indicator, such as a head-up display, computer monitor, projector, touchscreen display, liquid crystal display (LCD), light-emitting diode display, or flat panel display.

[0068] The computing device 4000 may include other input / output (I / O) devices 4012. Other I / O devices 4012 may include, for example, one or more audio output devices (e.g., speakers, headsets, earphones, alarms, etc.), one or more audio input devices (e.g., microphones or microphone arrays), location devices (e.g., GPS devices that communicate with satellite-based systems to receive the location of the computing device 4000, as is known in the Art), audio codecs, video codecs, printers, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors, vibration sensors, accelerometers, gyroscopes, etc.), image capture devices such as cameras, cursor control devices such as keyboards, mice, styluses, trackballs, or touchpads, barcode readers, quick response (QR) code readers, or radio frequency identification (RFID) readers.

[0069] The computing device 4000 may have any form factor suitable for its application and configuration, such as a handheld or mobile computing device (e.g., a mobile phone, smartphone, mobile internet device, tablet computer, laptop computer, netbook computer, ultrabook computer, personal digital assistant (PDA), ultramobile personal computer, etc.), a desktop computing device, or a server computing device or other networked computing component.

[0070] One or more computing devices implementing any of the CPM data acquisition modules or methods disclosed herein may be part of a scientific instrument support system. Figure 10 is a block diagram of an exemplary scientific instrument support system 5000 in which some or all of the scientific instrument support methods (e.g., CPM data acquisition methods) disclosed herein may be performed according to various embodiments. The CPM data acquisition modules and methods disclosed herein (e.g., CPM data acquisition module 1000 in Figure 2 and methods 2000, 2100, and 2200 in Figures 5, 6, and 7) may be implemented by one or more CPMs 5010, user-local computing devices 5020, service-local computing devices 5030, or remote computing devices 5040 of the scientific instrument support system 5000.

[0071] Any of the CPM5010, user-local computing device 5020, service-local computing device 5030, or remote computing device 5040 may include any of the embodiments of the computing device 4000 discussed herein with reference to Figure 9, and any of the CPM5010, user-local computing device 5020, service-local computing device 5030, or remote computing device 5040 may take any suitable form of the embodiments of the computing device 4000 discussed herein with reference to Figure 9.

[0072] The CPM5010, user-local computing device 5020, service-local computing device 5030, or remote computing device 5040 may each include a processing device 5002, a storage device 5004, and an interface device 5006. The processing device 5002 may take any preferred form, including any form of the processing device 4002 considered herein with reference to Figure 4, and the processing device 5002 included in different versions of the CPM5010, user-local computing device 5020, service-local computing device 5030, or remote computing device 5040 may take the same or different forms. The storage device 5004 may take any preferred form, including any form of the storage device 5004 considered herein with reference to Figure 4, and the storage device 5004 included in different versions of the CPM5010, user-local computing device 5020, service-local computing device 5030, or remote computing device 5040 may take the same or different forms. Interface device 5006 may take any preferred form, including any form of interface device 4006 considered herein with reference to Figure 4, and interface device 5006 included in different CPM 5010, user local computing device 5020, service local computing device 5030, or remote computing device 5040 may take the same or different forms.

[0073] The CPM 5010, user-local computing device 5020, service-local computing device 5030, and remote computing device 5040 may communicate with other elements of the scientific instrument support system 5000 via a communication path 5008. The communication path 5008 may communicatively connect interface devices 5006 of different elements of the scientific instrument support system 5000, as shown, and may be a wired or wireless communication path (for example, according to one of the communication techniques considered herein with reference to interface device 4006 of computing device 4000 in Figure 9). The particular scientific instrument support system 5000 shown in Figure 10 includes communication paths between each pair of CPM 5010, user-local computing device 5020, service-local computing device 5030, and remote computing device 5040, but this “fully connected” implementation is merely illustrative, and in various embodiments, various versions of the communication path 5008 may not exist. For example, in some embodiments, the service local computing device 5030 may not have a direct communication path 5008 between its interface device 5006 and the interface device 5006 of the CPM 5010. Instead, it may communicate with the CPM 5010 via the communication path 5008 between the service local computing device 5030 and the user local computing device 5020, and the communication path 5008 between the user local computing device 5020 and the CPM 5010. The CPM 5010 may include any suitable CPM such as an SEM, TEM, STEM, or ion beam microscope.

[0074] The user-local computing device 5020 may be a computing device that is local to the user of the CPM 5010 (for example, according to any embodiment of the computing device 4000 considered herein). In some embodiments, the user-local computing device 5020 may also be, but is not required to be, local to the CPM 5010. For example, the user-local computing device 5020 located at the user's home or office is remote from the CPM 5010 but can communicate with the CPM 5010, and as a result, the user can use the user-local computing device 5020 to control and / or access data from the CPM 5010. In some embodiments, the user-local computing device 5020 may be a laptop, smartphone, or tablet device. In some embodiments, the user-local computing device 5020 may be a portable computing device.

[0075] The service local computing device 5030 may be a computing device that is local to an entity servicing the CPM 5010 (for example, according to any embodiment of the computing device 4000 considered herein). For example, the service local computing device 5030 may be local to the manufacturer of the CPM 5010 or to a third-party service company. In some embodiments, the service local computing device 5030 may communicate with the CPM 5010, the user local computing device 5020, and / or the remote computing device 5040 (for example, via a direct communication path 5008 or via a plurality of “indirect” communication paths 5008, as considered above), and receive data relating to the operation of the CPM 5010, the user local computing device 5020, and / or the remote computing device 5040 (for example, the results of a self-test of the CPM 5010, the calibration coefficients used by the CPM 5010, the measurements of sensors associated with the CPM 5010, etc.). In some embodiments, the service local computing device 5030 may communicate with the CPM 5010, the user local computing device 5020, and / or the remote computing device 5040 (for example, via a direct communication path 5008 or via multiple “indirect” communication paths 5008, as considered above), and transmit data to the CPM 5010, the user local computing device 5020, and / or the remote computing device 5040 (for example, updating programmed instructions such as firmware in the CPM 5010, initiating the execution of a test or calibration sequence in the CPM 5010, updating programmed instructions such as software in the user's local computing device 5020 or the remote computing device 5040, etc.).Users of the CPM5010 may communicate with the service local computing device 5030 to report problems with the CPM5010 or user local computing device 5020, request a visit from a technician to improve the operation of the CPM5010, order consumables or replacement parts associated with the CPM5010, or utilize the CPM5010 or user local computing device 5020 for other purposes.

[0076] The remote computing device 5040 may be a computing device located away from the CPM 5010 and / or the user-local computing device 5020 (for example, according to any embodiment of the computing device 4000 considered herein). In some embodiments, the remote computing device 5040 may be included in a data center or other large-scale server environment. In some embodiments, the remote computing device 5040 may include network-attached storage (for example, as part of storage device 5004). The remote computing device 5040 may store data generated by the CPM 5010, perform analysis of the data generated by the CPM 5010 (for example, according to programmed instructions), facilitate communication between the user-local computing device 5020 and the CPM 5010, and / or facilitate communication between the service-local computing device 5030 and the CPM 5010.

[0077] In some embodiments, one or more of the elements of the scientific instrument support system 5000 shown in Figure 10 may be absent. Furthermore, in some embodiments, multiple of the various elements of the scientific instrument support system 5000 in Figure 10 may be present. For example, the scientific instrument support system 5000 may include multiple user local computing devices 5020 (e.g., different user local computing devices 5020 associated with different users or in different locations). In another example, the scientific instrument support system 5000 may include multiple CPMs 5010 all communicating with a service local computing device 5030 and / or remote computing devices 5040. In such embodiments, the service local computing device 5030 may monitor these multiple CPMs 5010, trigger updates, or "broadcast" other information to the multiple scientific instruments 5010 simultaneously. Different CPM 5010s or other scientific instruments within the scientific instrument support system 5000 may be located close to each other (e.g., in the same room) or far apart from each other (e.g., on different floors of a building, in different buildings, in different cities, etc.). In some embodiments, the CPM 5010 may be connected to an Internet-of-Things (IoT) stack that enables command and control of the CPM 5010 via web-based applications, virtual or augmented reality applications, mobile applications, and / or desktop applications. Any of these applications may be accessed by a user who communicates with the CPM 5010 and operates a user-local computing device 5020 via an intervening remote computing device 5040. In some embodiments, the CPM 5010 may be sold by the manufacturer as part of a local scientific instrument computing unit 5012, together with one or more associated user-local computing devices 5020.

[0078] The following paragraphs provide various examples of embodiments disclosed herein. Embodiment 1 is a charged particle microscope support device comprising: a first logic for causing a charged particle microscope to generate a single image of a first part of a specimen; a second logic for generating a first mask based on one or more regions of interest provided by user annotations on the single image; and a third logic for training a machine learning computation model using the single image and one or more regions of interest, wherein the first logic causes the charged particle microscope to generate multiple images of corresponding additional parts of the specimen; and the second logic is for generating multiple masks based on corresponding images of additional parts of the specimen using the machine learning computation model without retraining after the machine learning computation model has been trained using the single image and one or more regions of interest. Example 2 includes the subject matter described in Example 1 and further specifies that the charged particle microscope includes an electron microscope, an ion microscope, or a dual-beam microscope. Example 3 includes the subject matter described in any of Examples 1-2, and further specifies that the first logic causes a charged particle microscope to generate other images of the first portion of the specimen according to the first mask after the generation of the first mask. Example 4 includes the subject described in Example 3, and further specifies that the resolution of a single image of the first part of the specimen is less than the resolution of other images of the first part of the specimen. Example 5 includes a subject as described in any of Examples 3-4, and further specifies that the acquisition time for a single image of the first part is less than the acquisition time for other images of the first part of the sample. Example 6 includes the subject matter described in any of Examples 3 to 5, and further specifies that the detector associated with a single image of the first part is different from the detectors associated with other images of the first part of the specimen. Example 7 includes a subject as described in any of Examples 3 to 6, and further specifies that the radiation dose to the specimen associated with a single image of the first part is less than the radiation dose to the specimen associated with other images of the first part of the specimen. Example 8 includes a subject described in any of Examples 3 to 7, further specifying that the other images include electron backscatter diffraction (EBD) data or energy-dispersive X-ray spectroscopy (EDS) data. Example 9 includes a subject as described in any of Examples 3 to 7, and further includes a fourth logic for combining a single image and other images into a combined image representing a first part of the specimen, and for providing the combined image for display by a display device. Example 10 includes the subject matter described in any of Examples 3 to 9, and further specifies that the first logic causes a charged particle microscope to generate multiple other images of the corresponding additional parts of the specimen according to the corresponding multiple masks after the generation of multiple masks. Example 11 includes the subject matter described in any of Examples 1 to 10, and further specifies that 1) a first part of the specimen represents a plane passing through the specimen, and several additional parts of the specimen represent several parallel planes passing through the specimen, or 2) a first part of the specimen represents a plane passing through the specimen, and several additional parts of the specimen represent several planes passing through the specimen at different angles. Example 12 includes a subject as described in any of Examples 1 to 11 and further includes a fourth logic for generating a three-dimensional image of the specimen using images captured according to a mask. Example 13 includes the subject matter described in any of Examples 1 to 12, and further specifies that the second logic provides a single image of the first part of the specimen for display on a display device and receives instructions from the user for one or more regions of interest of the displayed single image by user annotations. Example 14 includes the subject matter described in any of Examples 1 to 13, and further specifies that several additional parts of the specimen are adjacent planes passing through at least a portion of the specimen. Example 15 includes the subject matter described in any of Examples 1 to 14, further specifying that the first part of the specimen is adjacent to the nearest of several additional parts of the specimen. Example 16 includes a subject as described in any of Examples 1 to 15, and further specifies that the first portion of the specimen is spaced at a distance of 1 to 30 microns from the nearest of several additional portions of the specimen. Example 17 includes the subject matter described in Example 16, further specifying that the first part of the specimen and the nearest of several additional parts of the specimen are spaced apart in the (z) direction. Example 18 includes the subject described in any of Examples 16-17, further specifying that the first portion of the specimen is spaced at a distance of 1 to 10 microns from the nearest of several additional portions of the specimen. Example 19 includes the subject matter described in Example 18 and further identifies the first part of the specimen and the nearest of several additional parts of the specimen as different milled parts of the specimen. Example 20 includes the subject described in any of Examples 16-17, further specifying that the first portion of the specimen is spaced 10 to 30 microns apart from the nearest of several additional portions of the specimen. Example 21 includes the subject matter described in Example 20 and further identifies the first part of the specimen and the nearest of several additional parts of the specimen as different mechanically sliced ​​parts of the specimen. Example 22 includes a subject as described in any of Examples 1-21 and further specifies that no additional user annotations for regions of interest in the image are received before multiple masks are generated. Example 23 includes the subject matter described in any of Examples 1 to 22, and further specifies that the machine learning computation model is trained using a training corpus that does not contain images of the samples before the machine learning computation model is trained using a single image and one or more regions of interest. Example 24 includes the subject matter described in Example 23, further specifying that the specimen includes a biological sample, and the training corpus does not include images of similar biological samples. Example 25 further specifies that it includes a subject described in any of Examples 23-24, the region of interest includes a feature of interest, and the training corpus does not include any examples of the feature of interest. Example 26 includes the subject matter described in any of Examples 1 to 25, and further specifies that the second logic is to compare masks related to adjacent parts of a sample and adjust one or more of the masks when the differences between the masks satisfy one or more difference criteria. Example 27 includes the subject matter described in Example 26 and further specifies that the adjustment of one or more of the masks includes increasing the size of the region imaged according to one or more of the masks. Example 28 includes the subject matter described in any of Examples 26-27, and further specifies that the second logic is to prompt the user to annotate one or more regions of interest in other images of the sample if the differences between the masks satisfy one or more difference criteria. Example 29 includes the subject matter described in Example 28 and further identifies a third logic for training a machine learning computation model using other images and one or more regions of interest of those images. Example 30 includes the subject matter described in any of Examples 26-29, and further specifies that the second logic is to prompt the user to accept or reject one or more of the masks if the differences between the masks meet one or more difference criteria. Example 31 includes the subject matter described in any of Examples 1 to 30, and further specifies that the machine learning computation model includes a multilayer neural network model.

[0079] Example 32 includes the subject matter described in any of Examples 1 to 31, and further specifies that the first mask is instructed to image the region of interest.

[0080] Example 33 includes the subject described in Example 32 and further specifies that the first mask is instructed to image the region of interest and the region around the region of interest. Example 34 includes the subject matter described in any of Examples 1 to 33, and further includes a charged particle microscope. Example 35 includes a subject described in any of Examples 1 to 34, and further specifies that at least one of the first part of the specimen and the additional part of the specimen has the same set of (x,y) coordinates and different (z) coordinates. Example 36 is a charged particle microscope support device comprising: a first logic for causing a charged particle microscope to generate a single image of a first portion of a specimen; a second logic for generating a first mask based on one or more regions of interest indicated by user annotations on the single image, wherein the regions of interest include features of interest in the specimen; and a third logic for training a machine learning computational model using the single image and one or more regions of interest, wherein the first logic causes the charged particle microscope to generate an image of a second portion of the specimen, the second portion of the specimen being adjacent to the first portion of the specimen; and the second logic for generating a second mask based on the image of the second portion of the specimen using the machine learning computational model after the machine learning computational model has been trained using the single image and one or more regions of interest, wherein the second mask instructs to image regions of the second portion of the specimen that include features of interest and regions of the second portion of the specimen that do not include features of interest. Example 37 includes the subject matter described in Example 36 and further specifies that the charged particle microscope includes an electron microscope, an ion microscope, or a dual-beam microscope. Example 38 includes the subject matter described in any of Examples 36-37, and further specifies that the first logic causes a charged particle microscope to generate other images of the first portion of the specimen according to the first mask after the generation of the first mask. Example 39 includes the subject matter described in Example 38 and further specifies that the resolution of a single image of the first part of the specimen is less than the resolution of other images of the first part of the specimen. Example 40 includes a subject described in any of Examples 38-39, and further specifies that the acquisition time for a single image of the first part is less than the acquisition time for other images of the first part of the sample. Example 41 includes the subject matter described in any of Examples 38-40, and further specifies that the detector associated with a single image of the first part is different from the detectors associated with other images of the first part of the specimen. Example 42 includes a subject as described in any of Examples 38 to 41, and further specifies that the radiation dose to the specimen associated with a single image of the first part is less than the radiation dose to the specimen associated with other images of the first part of the specimen. Example 43 includes a subject described in any of Examples 38 to 42, further specifying that other images include electron backscatter diffraction (EBD) data or energy-dispersive X-ray spectroscopy (EDS) data. Example 44 includes a subject as described in any of Examples 38 to 43, and further includes a fourth logic for combining a single image and other images into a combined image representing a first part of the specimen, and for providing the combined image for display by a display device. Example 45 further specifies that the subject described in any of Examples 38-44 is the other image, the other image is the first other image, and the first logic causes the charged particle microscope to generate the second other image of the second part of the specimen according to the second mask after the generation of the second mask. Example 46 includes the subject matter described in any of Examples 36 to 45, and further specifies that 1) a first part of the specimen represents a plane passing through the specimen and a second part of the specimen represents a parallel plane passing through the specimen, or 2) a first part of the specimen represents a plane passing through the specimen and a second part of the specimen represents a plane passing through the specimen at a different angle. Example 47 includes the subject matter described in any of Example 46, further specifying that the first and second portions of the specimen are adjacent to the milled or mechanically sliced ​​portions of the specimen. Example 48 includes a subject described in any of Examples 36-47 and further includes a fourth logic for generating a three-dimensional image of the specimen using images captured according to a mask. Example 49 includes a subject as described in any of Examples 36-48, and further specifies that the second logic provides a single image of a first portion of the specimen for display on a display device and receives instructions from the user for one or more regions of interest of the displayed single image by user annotations. Example 50 includes a subject described in any of Examples 36 to 49, further specifying that the first portion of the specimen is separated from the second portion of the specimen by a distance of 1 to 30 microns. Example 51 includes the subject matter described in Example 50, further specifying that the first part of the specimen and the second part of the specimen are spaced apart in the (z) direction. Example 52 includes any of Examples 50 to 51, further specifying that the first portion of the specimen is separated from the second portion of the specimen by a distance of 1 to 10 microns. Example 53 includes the subject matter described in Example 52, further specifying that the first and second parts of the specimen are different milled parts of the specimen. Example 54 includes the subject described in any of Examples 50 to 51, and further specifies that the first portion of the specimen is separated from the second portion of the specimen by a distance of 10 to 30 microns. Example 55 includes the subject matter described in Example 54, further specifying that the first and second parts of the specimen are different mechanically sliced ​​parts of the specimen. Example 56 includes a subject as described in any of Examples 36-55, and further specifies that no additional user annotations are received for the region of interest in the image before the second mask is generated. Example 57 includes the subject matter described in any of Examples 36 to 56, and further specifies that the machine learning computation model is trained using a training corpus that does not contain images of the samples before the machine learning computation model is trained using a single image and one or more regions of interest. Example 58 includes the subject matter described in Example 57, further specifying that the specimen includes a biological sample and the training corpus does not include images of similar biological samples. Example 59 includes a subject described in any of Examples 57-58, and further specifies that the training corpus does not include any examples of the interest features. Example 60 includes the subject matter described in any of Examples 36 to 59, and further specifies that the second logic compares the first mask with the second mask and adjusts the second mask if the difference between the masks satisfies one or more difference criteria. Example 61 includes the subject matter described in Example 60 and further specifies that one or more of the mask adjustments include increasing the size of the region imaged according to the second mask. Example 62 includes the subject matter described in any of Examples 60-61, and further specifies that the second logic is to prompt the user to annotate one or more regions of interest in other images of the sample if the differences between the masks satisfy one or more difference criteria. Example 63 includes the subject matter described in Example 62 and further identifies that the third logic is for training a machine learning computation model using other images and one or more regions of interest of those images. Example 64 includes the subject matter described in any of Examples 60-63, and further specifies that the second logic is to prompt the user to accept or reject the second mask if the difference between the first and second masks satisfies one or more difference criteria. Example 65 includes the subject matter described in any of Examples 36 to 64, and further specifies that the machine learning computation model includes a multilayer neural network model. Example 66 includes a subject as described in any of Examples 36-65, and further specifies that the first mask instructs to image the region of interest. Example 67 includes the subject described in Example 66 and further specifies that the first mask is instructed to image the region of interest and the region around the region of interest. Example 68 comprises the subject matter described in any of Examples 36 to 67, and further comprises a charged particle microscope. Example 69 includes a subject described in any of Examples 36 to 68, and further specifies that the first part of the specimen and the second part of the specimen have the same set of (x,y) coordinates and different (z) coordinates. Example 70 is a charged particle microscope support device comprising: first logic for generating a first dataset related to a first part of a specimen by processing data from a first imaging round of a first part of a specimen by a charged particle microscope; second logic for generating a first mask related to a first part of a specimen based on user identification of one or more first regions of interest in the first dataset related to a first part of a specimen; and third logic for training a machine learning computation model using the first dataset related to a first part of a specimen and one or more first regions of interest, wherein the first logic is for generating a first dataset related to a second part of a specimen by processing data from a first imaging round of a second part of a specimen by a charged particle microscope, and the second logic is for training a machine learning computation model using the first dataset related to a first part of a specimen and one or more first regions of interest, after the machine learning computation model and specimen This is a charged particle microscope support device for generating a second mask related to a second part of a specimen using a first dataset related to a second part of a specimen, the first logic for generating a second dataset related to a first part of a specimen by processing data from a second imaging round of the first part of the specimen according to a first mask using a charged particle microscope, the second logic for adjusting the second mask before generating a second dataset related to a second part of a specimen according to a second mask if the difference between the first mask and the second mask, or the difference between the first dataset related to a first part of a specimen and the first dataset related to a second part of a specimen, satisfies one or more difference criteria, and the first logic for generating a second dataset related to a second part of a specimen by processing data from a second imaging round of the second part of a specimen according to a second mask using a charged particle microscope. Example 71 includes the subject matter described in Example 70 and further specifies that the charged particle microscope includes an electron microscope, an ion microscope, or a dual-beam microscope. Example 72 includes the subject matter described in Examples 70-71 and further specifies that the resolution of the first imaging round of the first portion of the specimen is less than that of the second imaging round of the first portion of the specimen. Example 73 includes a subject described in any of Examples 70-72, and further specifies that the acquisition time of the first imaging round of the first part is less than the acquisition time of the second imaging round of the first part of the specimen. Example 74 includes the subject matter described in any of Examples 70-73 and further specifies that the detector associated with the first imaging round of the first part is different from the detector associated with the second imaging round of the first part of the specimen. Example 75 includes a subject described in any of Examples 70-74, and further specifies that the radiation dose to the specimen during the first imaging round of the first part of the specimen is less than the radiation dose to the specimen during the second imaging round of the first part of the specimen. Example 76 includes a subject described in any of Examples 70-75 and further specifies that a second dataset related to the first portion of the specimen includes electron backscatter diffraction (EBD) data or energy-dispersive X-ray spectroscopy (EDS) data. Example 77 includes a subject as described in any of Examples 70 to 76, and further includes a fourth logic for combining a first dataset related to a second part of a sample and a second dataset related to a second part of a sample into a combined image representing the second part of a sample, and for providing the combined image for display by a display device. Example 78 includes the subject matter described in any of Examples 70 to 77, and further specifies that 1) a first part of the specimen represents a plane passing through the specimen and a second part of the specimen represents a parallel plane passing through the specimen, or 2) a first part of the specimen represents a plane passing through the specimen and a second part of the specimen represents a plane passing through the specimen at a different angle. Example 79 includes the subject matter described in any of Examples 70-78 and further includes a fourth logic for generating a three-dimensional image of the sample using a second dataset. Example 80 includes the subject matter described in any of Examples 70-79, and further specifies that the second logic provides a first dataset related to a first portion of the sample for display on a display device and receives instructions from the user for one or more regions of interest of the displayed first dataset by user annotations. Example 81 includes the subject matter described in any of Examples 70 to 80, further specifying that the first portion of the specimen is separated from the second portion of the specimen by a distance of 1 to 30 microns. Example 82 includes the subject matter described in Example 81, further specifying that the first part of the specimen and the second part of the specimen are spaced apart in the (z) direction. Example 83 includes the subject matter described in any of Examples 81-82, further specifying that the first portion of the specimen is separated from the second portion of the specimen by a distance of 1 to 10 microns. Example 84 includes the subject matter described in Example 83, further specifying that the first and second parts of the specimen are different milled parts of the specimen. Example 85 includes the subject matter described in any of Examples 81 to 82, further specifying that the first portion of the specimen is separated from the second portion of the specimen by a distance of 10 to 30 microns. Example 86 includes the subject matter described in Example 85, further specifying that the first and second parts of the specimen are different mechanically sliced ​​parts of the specimen. Example 87 includes a subject as described in any of Examples 70-86 and further specifies that no additional user annotations for the region of interest in the dataset associated with the sample are received before the second mask is generated. Example 88 includes the subject matter described in any of Examples 70-87 and further specifies that the machine learning computation model is trained using a training corpus that does not contain images of the sample before the machine learning computation model is trained using a first dataset related to a first portion of the sample and one or more first regions of interest. Example 89 includes the subject matter described in Example 88, further specifying that the specimen includes a biological sample and the training corpus does not include images of similar biological samples. Example 90 further specifies that it includes a subject described in any of Examples 88-89, the region of interest includes a feature of interest, and the training corpus does not include any examples of the feature of interest. Example 91 includes the subject matter described in any of Examples 70 to 90, and further specifies that the adjustment of the second mask includes increasing the size of the region to be imaged according to the second mask. Example 92 includes the subject matter described in any of Examples 70 to 91, and further specifies that the second logic is to prompt the user to accept or reject the second mask if the difference between the first mask and the second mask satisfies one or more difference criteria. Example 93 includes the subject matter described in any of Examples 70 to 92, and further specifies that the machine learning computation model includes a multilayer neural network model. Example 94 includes a subject described in any of Examples 70-93, and further specifies that the first mask instructs to image the region of interest. Example 95 includes a subject as described in any of Example 94, and further specifies that the first mask is instructed to image the region of interest and the region around the region of interest. Example 96 comprises the subject matter described in any of Examples 70 to 95, and further comprises a charged particle microscope. Example 97 includes the subject matter described in any of Examples 70 to 96, and further specifies that the first part of the specimen and the second part of the specimen have the same set of (x,y) coordinates and different (z) coordinates. Example 98 includes one of the CPM data acquisition modules disclosed herein. Example 99 includes any of the methods disclosed herein. Example 100 includes one of the GUIs disclosed herein. Example 101 includes any of the scientific instrument-assisted computing devices and systems disclosed herein.

Claims

1. A charged particle microscope support device, first logic for causing the charged particle microscope to generate a single image of a first portion of the specimen; second logic for generating a first mask based on one or more regions of interest provided by user annotations of the single image; and third logic for training a machine learning computational model using the single image and the one or more regions of interest; the first logic causes the charged particle microscope to generate a plurality of images of a corresponding plurality of additional portions of the specimen; The charged particle microscope support device, wherein the second logic is for generating a plurality of masks based on corresponding images of the additional portions of the specimen using the machine learning computational model without retraining after the machine learning computational model has been trained using the single image and the one or more regions of interest.

2. 2. The charged particle microscope support device of claim 1, wherein the first logic, after generating the first mask, causes the charged particle microscope to generate another image of the first portion of the specimen according to the first mask.

3. 3. The charged particle microscope support device of claim 2, wherein the resolution of the single image of the first portion of the specimen is less than the resolution of the other images of the first portion of the specimen.

4. 3. The charged particle microscope support device of claim 2, wherein the acquisition time of the single image of the first portion is less than the acquisition time of the other images of the first portion of the specimen.

5. 3. The charged particle microscope support apparatus of claim 2, further comprising fourth logic for combining the single image and the other image into a combined image representing the first portion of the specimen and for providing the combined image for display by a display device.

6. 2. The charged particle microscope support device of claim 1, wherein: 1) the first portion of the specimen represents a plane through the specimen and the multiple additional portions of the specimen represent multiple parallel planes through the specimen; or 2) the first portion of the specimen represents a plane through the specimen and the multiple additional portions of the specimen represent multiple planes through the specimen at different angles.

7. the second logic comprising: providing the single image of the first portion of the specimen for display on a display device; and 10. The charged particle microscope support device of claim 1, for receiving from a user an indication of one or more regions of interest by user annotation of the displayed single image.

8. 2. The charged particle microscope support system of claim 1, wherein the second logic is for comparing masks associated with adjacent portions of the specimen and for adjusting one or more of the masks if differences between the masks satisfy one or more difference criteria.

9. A charged particle microscope support device, first logic for causing the charged particle microscope to generate a single image of a first portion of the specimen; second logic for generating a first mask based on one or more regions of interest indicated by user annotations of the single image, the regions of interest including features of interest within the specimen; and and third logic for training a machine learning computational model using the single image and the one or more regions of interest; the first logic causes the charged particle microscope to generate an image of a second portion of the specimen, the second portion of the specimen being proximate to the first portion of the specimen; The second logic is for training a machine learning computational model using the single image and the one or more regions of interest, and then using the machine learning computational model to generate a second mask based on an image of the second portion of the specimen, the second mask instructing imaging of regions of the second portion of the specimen that include the feature of interest and regions of the second portion of the specimen that do not include the feature of interest.

10. 10. The charged particle microscope support device of claim 9, further comprising fourth logic for generating a three-dimensional image of the specimen using images captured according to the first and second masks.

11. 10. The charged particle microscope support device of claim 9, wherein the first portion of the specimen is spaced from the second portion of the specimen by a distance between 1 micron and 30 microns.

12. The charged particle microscope support device of claim 11 , wherein the first portion of the specimen and the second portion of the specimen are spaced apart in the (z) direction.

13. 10. The charged particle microscope assistance device of claim 9, wherein before the machine learning computational model is trained using the single image and the one or more regions of interest, the machine learning computational model is trained using a training corpus that does not include images of the specimen.

14. The apparatus of claim 13 , wherein the training corpus does not include any examples of the feature of interest.

15. 10. The charged particle microscope support system of claim 9, wherein the second logic is for comparing the first and second masks and adjusting the second mask if differences between the first and second masks satisfy one or more difference criteria.

16. A charged particle microscope support device, first logic for generating a first data set associated with a first portion of the specimen by processing data from a first round of imaging of the first portion with the charged particle microscope; second logic for generating a first mask associated with the first portion of the specimen based on a user identification of one or more first regions of interest in the first dataset associated with the first portion of the specimen; and third logic for training a machine learning computational model using a first dataset associated with the first portion of the specimen and the one or more first regions of interest; the first logic is for generating a first data set associated with the second portion of the specimen by processing data from a first round of imaging of the second portion with the charged particle microscope; the second logic is for, after the machine learning computational model has been trained using the first dataset associated with the first portion of the specimen and the one or more first regions of interest, generating a second mask associated with the second portion of the specimen using the machine learning computational model and the first dataset associated with the second portion of the specimen; the first logic is for generating a second data set associated with the first portion of the specimen by processing data from a second round of imaging of the first portion with the charged particle microscope according to the first mask; the second logic is for preparing the second mask before a second data set associated with the second portion of the specimen is generated according to the second mask when a difference between the first mask and the second mask or a difference between the first data set associated with the first portion of the specimen and the first data set associated with the second portion of the specimen meets one or more difference criteria; the first logic is for generating a second data set relating to the second portion of the specimen by processing data from a second imaging round according to the second mask of the second portion by the charged particle microscope.

17. 17. The charged particle microscope support apparatus of claim 16, wherein adjusting the second mask includes increasing a size of an area imaged according to the second mask.

18. 17. The charged particle microscope support device of claim 16, wherein the second logic is for prompting a user to accept or reject the second mask when differences between the first mask and the second mask meet one or more difference criteria.

19. The charged particle microscope support device of claim 16 , wherein the machine learning computational model includes a multi-layer neural network model.

20. 17. The charged particle microscope support device of claim 16, further comprising a charged particle microscope.

Images