Area selection in charged particle microscope imaging

Abstract

To provide a device, a system, a method, and a computer readable medium, that are associated with area selections in charged particle microscope (CPM) imaging.SOLUTION: A CPM support device includes: a first logic for generating a first dataset associated with an area of a sample as a result of processing data from a first imaged round of the image by a CPM; a second logic for generating a predictive parameter of the area; and a third logic for determining whether or not a second imaged round of the area is executed by the CPM based on the predictive parameter of the area. In response to the determination of the third logic to execute the second imaged round of the area, the first logic generates a second dataset including a measurement parameter associated with the area as a result of processing data from the second imaged round of the area by the CPM.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates generally to charged particle microscopy, and more particularly to region selection in charged particle microscopy imaging. [Background technology]

[0002] Microscopy is a technical field that uses microscopes to better view objects that are difficult to see with the naked eye. Different fields of microscopy include, for example, optical microscopy, charged particle (electron and / or ion) microscopy, and scanning probe microscopy. Charged particle microscopy involves using a beam of accelerated charged particles as an illumination source. Types of charged particle microscopy include, for example, transmission electron microscopy, scanning electron microscopy, scanning transmission electron microscopy, and focused ion beam microscopy. [Brief explanation of the drawings]

[0003]

[0013] The embodiments will be readily understood by the following detailed description taken in conjunction with the accompanying drawings, in which:

[0014] To facilitate this description, like reference numerals refer to like structural elements;

[0015] The embodiments are illustrated in the figures of the accompanying drawings, by way of example, and not by way of limitation.

[0004] [Figure 1] FIG. 1 is a block diagram of an exemplary charged particle microscope (CPM) assistance module for performing CPM imaging assistance operations, according to various embodiments. [Figure 2] 1 illustrates an exemplary specimen that may be imaged by CPM with the region selection techniques disclosed herein, according to various embodiments. [Figure 3A] 1 is a graphical representation that may be provided to a user via a display device as part of the region selection techniques disclosed herein, according to various embodiments. [Figure 3B] 1 is a graphical representation that may be provided to a user via a display device as part of the region selection techniques disclosed herein, according to various embodiments. [Figure 4] 1 is a flow diagram of an exemplary method for region selection in CPM imaging, according to various embodiments. [Figure 5] 1 is a flow diagram of an exemplary method for region selection in CPM imaging, according to various embodiments. [Figure 6] 1 is a flow diagram of an exemplary method for generating a machine learning computational model for region selection in CPM imaging, according to various embodiments. [Figure 7] 1 is an example of a graphical user interface that may be used to perform some or all of the assistance methods disclosed herein, according to various embodiments. [Figure 8] FIG. 1 is a block diagram of an example computing device that may perform some or all of the assistance methods disclosed herein, according to various embodiments. [Figure 9] FIG. 1 is a block diagram of an example CPM assistance system in which some or all of the assistance methods disclosed herein may be implemented, according to various embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0005] Disclosed herein are devices, systems, methods, and computer-readable media related to region selection in charged particle microscope (CPM) imaging. For example, in some embodiments, a CPM-assisted device may include: first logic for generating a first data set associated with a region of a specimen by processing data from a first imaging round of the region by CPM; second logic for generating predicted parameters for the region; and third logic for determining whether a second imaging round of the region will be performed by CPM based on the predicted parameters for the region; wherein the first logic is for generating a second data set including measured parameters associated with the region by processing data from the second imaging round of the region by CPM in response to a determination by the third logic that a second imaging round of the region will be performed.

[0006] CPM-assisted embodiments disclosed herein may achieve improved performance compared to conventional approaches. For example, conventional CPM requires a significant amount of manual intervention by an expert user to select regions of interest for detailed imaging. Thus, despite advances in CPM technology, the overall throughput of CPM systems remains stagnant. CPM-assisted embodiments disclosed herein may predict "high-resolution" information about regions of a specimen based on "low-resolution" imaging rounds, thus increasing CPM imaging throughput compared to conventional approaches and also reducing radiation damage to the specimen under study. The embodiments disclosed herein may be readily applied to many imaging applications, such as cryo-electron microscopy (cryo-EM), microcrystal electron diffraction (MED), and tomography. Thus, the embodiments disclosed herein provide improvements to CPM technology (e.g., improvements to the computer technology that assists CPM, among other improvements).

[0007] In the following detailed description, reference is made to the accompanying drawings that form a part hereof, wherein like numerals refer to like parts throughout, and in which are shown, by way of illustration, embodiments that may be implemented. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present disclosure. Therefore, the following detailed description is not to be taken in a limiting sense.

[0008] Various operations may be described as multiple discrete actions or operations in a sequence that is most helpful in understanding the subject matter disclosed herein. However, the order of description should not be construed to imply that these operations are necessarily order dependent. In particular, these operations may not be performed in the order presented. The operations described may be performed in a different order than in the described embodiment. In additional embodiments, various additional operations may be performed and / or described operations may be omitted.

[0009] For purposes of this disclosure, the phrases "A and / or B" and "A or B" mean (A), (B), or (A and B). For purposes of this disclosure, the phrases "A, B, and / or C" and "A, B, or C" mean (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C). While some elements may be referred to in the singular (e.g., "processing device"), any suitable element may be represented by multiple instances of that element, and vice versa. For example, a sequence of operations described as being performed by a processing device may be implemented with different operations being performed by different processing devices.

[0010] This specification uses the phrases "embodiments," "various embodiments," and "some embodiments," each of which may refer to one or more of the same or different embodiments. Furthermore, terms such as "comprises," "including," and "having" when used with respect to embodiments of the present disclosure are synonymous. When used to describe a range of dimensions, the phrase "between X and Y" represents a range that includes X and Y. As used herein, "apparatus" may refer to any individual device, a collection of devices, a portion of a device, or a collection of portions of devices. The drawings are not necessarily drawn to scale.

[0011] FIG. 1 is a block diagram of a CPM assistance module 1000 for performing assistance operations, according to various embodiments. The CPM assistance module 1000 may be implemented by a circuit (e.g., including electrical and / or optical components) such as a programmed computing device. The logic of the CPM assistance module 1000 may be included in a single computing device or may be distributed across multiple computing devices in communication with each other as needed. An example of a computing device that may implement the CPM assistance module 1000, alone or in combination, is discussed with reference to the computing device 4000 of FIG. 8, and an example of a system of interconnected computing devices in which the CPM assistance module 1000 may be implemented across one or more computing devices is discussed herein with reference to the CPM assistance system 5000 of FIG. 9. The CPM whose operation is assisted by the CPM assistance module 1000 may include any suitable type of CPM, such as a scanning electron microscope (SEM), a transmission electron microscope (TEM), a scanning transmission electron microscope (STEM), or an ion beam microscope.

[0012] CPM assistance module 1000 may include imaging logic 1002, parameter prediction logic 1004, region selection logic 1006, user interface logic 1008, and training logic 1010. As used herein, the term "logic" may include an apparatus for performing a series of operations associated with logic. For example, any of the logic elements included in CPM assistance module 1000 may be implemented by one or more computing devices programmed with instructions that cause one or more processing devices of the computing devices to perform a series of operations associated with the computing devices. In particular embodiments, a logic element may include one or more non-transitory computer-readable media having instructions that, when executed by one or more processing devices of one or more computing devices, cause one or more computing devices to perform a series of operations associated with the computing devices. As used herein, the term "module" may refer to a collection of one or more logic elements that together perform the function associated with the module. Different logic elements within a module may take the same form or different forms. For example, some logic within a module may be implemented by a programmed general-purpose processing device, while other logic within the module may be implemented by an application-specific integrated circuit (ASIC). In another example, different ones of the logic elements in a module may be associated with different sets of instructions executed by one or more processing devices. A module may not include all of the logic elements shown in the associated figures. For example, a module may include a subset of the logic elements shown in the associated figures if that module performs a subset of the operations discussed herein with respect to that module. In some embodiments, different ones of the logic elements in a module may utilize shared elements (e.g., shared programmed instructions and / or shared circuitry).

[0013] The imaging logic 1002 may be configured to generate a data set associated with a region of the specimen by processing data from a round of imaging of the region by a CPM (e.g., CPM 5010, discussed below with reference to FIG. 9). In some embodiments, the imaging logic 1002 may cause the CPM to perform one or more rounds of imaging of the region of the specimen.

[0014] In some embodiments, the imaging logic 1002 may be configured for cryo-electron microscopy (cryo-EM), and the specimen may be a cryo-EM specimen, such as the cryo-EM specimen 100 shown in FIG. 2. The cryo-EM specimen 100 of FIG. 2 may include a copper mesh grid (e.g., having a diameter between 1 millimeter and 10 millimeters) having square patches of carbon 102 thereon. The carbon in the patches 102 may include regular holes 104, e.g., having a diameter between 1 micron and 5 microns, and the holes 104 may have a thin layer of supercooled ice 108 therein, within which an element of interest 106 (e.g., a particle such as a protein molecule or other biomolecule) is embedded. In some embodiments, each of the holes 104 may serve as a different region to be analyzed by the CPM assistance module 1000 (e.g., to select the “best” one or more holes 104 for further investigation of the element of interest 106, as discussed below). This particular example of a specimen is merely illustrative, and any specimen suitable for a particular CPM may be used.

[0015] In some embodiments, the imaging logic 1002 may be configured to generate a first dataset associated with a region of the specimen by processing data from a first round of imaging of the region by CPM, and may also generate a second dataset associated with the region of the specimen by processing data from a second round of imaging of the region by CPM (e.g., if the region is selected for multiple imaging rounds, as discussed below with reference to the parameter prediction logic 1004 and the region selection logic 1006). For ease of explanation, the “initial” dataset generated by the imaging logic 1002 for use by the parameter prediction logic 1004 may be referred to herein as the “first dataset,” and the “subsequent” dataset generated by the imaging logic 1002 (after the region has been selected for additional imaging by the region selection logic 1006) may be referred to herein as the “second dataset.” For a particular region of the specimen, the first data set may have a lower resolution than the second data set, the acquisition time of the first imaging round associated with the first data set may be less than the acquisition time of the second imaging round associated with the second data set, and / or the radiation dose delivered to the specimen in the first imaging round associated with the first data set may be less than the radiation dose delivered to the specimen in the second imaging round associated with the second data set. The imaging logic 1002 may generate first data sets for multiple different regions of the specimen, and for some of these multiple different regions, may also generate second data sets.

[0016] In some embodiments, the imaging logic 1002 may generate datasets (e.g., first dataset and / or second dataset) by processing the output of the CPM from an imaging round by performing any of several processing operations, such as filtering, alignment, or other known operations. A second dataset generated by the imaging logic 1002 for a particular region of the specimen may include measurements of a set of parameters for that region. In some embodiments, a “set of parameters” may include a single parameter, while in other embodiments, a “set of parameters” may include multiple parameters. The set of parameters for a region may include at least one data quality indicator, a measurement indicative of the likelihood of obtaining high-quality information about particles or other elements of interest of the specimen in that region by further investigating the region. Thus, the data quality indicator may indicate to a user of the CPM the likelihood that further investigation of the relevant region of the specimen will yield “good” results. In some embodiments where the specimen includes particles or other elements of interest in ice (e.g., in cryo-electron microscopy applications), one of the parameters of the region (e.g., data quality indicator) may include the thickness of the ice in that region. In some embodiments, the parameters of a region may include the number of features of interest in the region, the level of contamination in the region, the amount of specimen movement in the region, the amount of specimen degradation in the region, or the orientation distribution of features of interest within the region, any of which may be measured by performing a second imaging round with the CPM, and the measurements of these parameters may be included in a second data set associated with the region.

[0017] The parameter prediction logic 1004 may provide a first data set associated with a region of the specimen (e.g., resulting from a "low resolution," "low acquisition time," and / or "low radiation dose" imaging round) generated by the imaging logic 1002 to a machine learning computational model that outputs predicted values ​​for a set of parameters of the region of the specimen. The set of parameters for which predicted values ​​may be generated by the machine learning computational model of the parameter prediction logic 1004 may be the same set of parameters whose measured values ​​may be generated by the imaging logic 1002 in a second data set (e.g., resulting from a "high resolution," "high acquisition time," and / or "high radiation dose" imaging round).

[0018] Thus, the machine learning computational model of the parameter prediction logic 1004 can predict "high-resolution" information about regions of the specimen based on "low-resolution" imaging rounds. If the machine learning computational model is accurate, a user may be able to easily evaluate parameters of various regions of the specimen and evaluate which regions to further investigate (e.g., by one or more additional imaging rounds with different characteristics) without incurring the additional acquisition time and / or radiation dose associated with a second "high-resolution" imaging round for regions of no further interest. This can dramatically increase the throughput of CPM imaging, which is typically bottlenecked by the need for users of conventional CPM systems to perform high-resolution imaging of all regions to even make an initial assessment of which regions of the specimen are promising for further study. Furthermore, even "expert" CPM users cannot accurately assess which regions of the specimen are likely candidates for further investigation from "low-resolution" imaging rounds. Traditional "rules of thumb" or "intuitive" user guesses, or even traditional analytical approaches (such as those discussed herein with respect to "bootstrapping" machine learning computational models), are typically inaccurate and thus reduce throughput.

[0019] In some embodiments, the machine learning computational model of the parameter prediction logic 1004 may be a multi-layer neural network model. For example, the machine learning computational model included in the parameter prediction logic 1004 may have a residual network (ResNet) architecture that includes skip connections through one or more neural network layers. The training data (e.g., input images and parameter values) may be normalized in any suitable manner (e.g., using histogram equalization and mapping parameters to an interval such as [0, 1]). Other machine learning computational models, such as other neural network models (e.g., dense convolutional neural network models or other deep convolutional neural network models, etc.).

[0020] In some embodiments, the parameter prediction logic 1004 may be configured to initially generate a machine learning computational model by inputting initial values ​​for some or all of the parameters of the machine learning computational model (e.g., weights associated with a neural network model). In some embodiments, some or all of the initial values ​​may be randomly generated, while in some embodiments, some or all of the initial values ​​may be adopted from another machine learning computational model. For example, a first machine learning computational model may be pre-trained to generate predicted values ​​of parameters for specimens containing a first type of element of interest. Then, if a user wants to investigate specimens containing a second, different type of element of interest, some or all of the parameters of the second machine learning computational model (for generating predicted values ​​of parameters for specimens containing the second type of element of interest) may be the same as the parameters of the first machine learning computational model. The second machine learning computational model may be further trained on specimens containing the second type of element of interest; therefore, the parameters of the second machine learning computational model may not remain the same as those of the first machine learning computational model after such training. In some embodiments, the selection of the first machine learning computational model on which the second machine learning computational model is "based" may be performed by a user (e.g., a user may select a stored first machine learning computational model to use as a perspective for the second machine learning computational model via the graphical user interface (GUI) 3000 of FIG. 7, discussed below), or may be performed automatically by the parameter prediction logic 1004 (e.g., based on the similarity between the first type of element of interest and the second type of element of interest).For example, if the first type of element of interest is a first protein and the second type of element of interest is a second, different protein, the parameter prediction logic 1004 may use a lookup table or database query to determine the similarity of the protein sequences of the first and second proteins and may use the resulting similarity to initialize a second machine learning computational model for the second protein (e.g., by "copying" the first machine learning computational model to serve as the initial second machine learning computational model if the similarity exceeds a threshold, or by "using" the number of weights of the first machine learning computational model when adding weights of the initial second machine learning computational model proportional to the similarity between the first and second proteins). In other embodiments, the similarity between the first type of element of interest and the second type of element of interest may be a shape similarity between the elements of interest (e.g., using a machine learning computational model trained on spherical elements of interest as a starting point for a new machine learning computational model to be applied to elliptical elements of interest, or using a machine learning computational model trained on a type of particle in a hexagonal grid as a starting point for a new machine learning computational model to be applied to the same type of particle in a standard, non-hexagonal grid).

[0021] In some embodiments, the parameter prediction logic 1004 may deploy the machine learning computational model only after appropriate training of the machine learning computational model (e.g., using training data including multiple first and second data sets for different regions of the specimen). Before the machine learning computational model is trained (e.g., before appropriate training data is generated), the parameter prediction logic 1004 may use an image processing computational model different from the machine learning computational model to generate predicted values ​​for a set of parameters of a region of the specimen. Examples of such image processing computational models may include linear regression, histogram binarization, or any other suitable computational model. While predicted values ​​generated by such an image processing computational model may be less accurate than predicted values ​​generated by a trained machine learning computational model, the use of such an image processing computational model may assist a user during an initial "bootstrap" period during which training data is generated.

[0022] The region selection logic 1006 may determine whether a second imaging round of a region of the specimen is to be performed by the CPM based on predicted values ​​of the set of parameters for the region (generated by the machine learning computational model of the parameter prediction logic 1004). If the region selection logic 1006 determines that a second imaging round of a region of the specimen should not be performed, the second imaging round may not be performed (and thus a second data set including measurements of the set of parameters may not be generated for the region). The region selection logic 1006 may apply any one or more criteria to the predicted values. For example, in some embodiments, the region selection logic 1006 may determine that a second imaging round of a region of the specimen is to be performed when the predicted values ​​of one or more of the parameters meet threshold criteria for those parameters. For example, in embodiments in which the machine learning computational model of the parameter prediction logic 1004 outputs a predicted value of ice thickness in a region, the region selection logic 1006 may select a region for the second imaging round when the predicted value of ice thickness is between a lower threshold and an upper threshold (the values ​​may be set, for example, by a user). In addition to, or instead of, a threshold criterion, the region selection logic 1006 may determine that a second round of imaging of the specimen regions is to be performed for all regions of the specimen having one or more predicted parameter values ​​in the “best” percentage of all predicted parameter values ​​(e.g., indicating that these regions are likely to have good data quality), and that a second round of imaging of the specimen regions is not to be performed for the remaining regions (e.g., indicating that these regions are likely to have poor data quality). For example, in an embodiment in which the machine learning computational model of the parameter prediction logic 1004 outputs a predicted value of a data quality indicator, the region selection logic 1006 may determine that a second round of imaging is to be performed for all regions whose predicted data quality indicator is in the top 20%. This particular number is merely an example, and any suitable similar criterion may be used.

[0023] In some embodiments, the criteria applied by the region selection logic 1006 to the predictions generated by the machine learning computational model may include probabilistic criteria. For example, for a particular region of a specimen, the region selection logic 1006 may select the region for a second round of imaging with a given probability. The probability may be independent of the predicted value of the set of parameters for the region, or may be weighted to increase as the predicted value of the set of parameters becomes "better" and / or weighted to increase as the predicted value of the set of parameters becomes "worse" (e.g., to generate data that can be used to retrain the machine learning computational model, as discussed below with reference to the training logic 1010). Any suitable probabilistic selection criteria may also be implemented by the region selection logic 1006.

[0024] The user interface logic 1008 may provide information to and receive input from a user (e.g., via a GUI such as GUI 3000 discussed below with reference to FIG. 7). In some embodiments, the user interface logic 1008 may cause a graphical representation of at least some of the first dataset associated with a region of the specimen to be displayed on a display device (e.g., any of the display devices discussed herein). For example, FIGS. 3A and 3B are graphical representations 160 and 170, respectively, that may be provided to a user via a display device (and a GUI such as GUI 3000 discussed below with reference to FIG. 7) as part of the region selection techniques disclosed herein, according to various embodiments. The graphical representation 160 may include at least some of the first dataset associated with one or more regions of the specimen. For example, the graphical representation 160 of FIG. 3A shows a patch 102 of a cryo-EM sample (e.g., the cryo-EM sample 100 of FIG. 2) having multiple holes 104 therein. The individual holes 104 may correspond to different individual regions that may be analyzed according to the region selection techniques disclosed herein, and the graphical representation 160 of Figure 3A illustrates an embodiment in which a first "low-resolution" data set of the specimen holes 104 is obtained by capturing a "low-resolution" image of the entire patch 102. The portion of the image of the entire patch 102 that corresponds to a particular hole 104 may serve as the first data set associated with the hole 104.

[0025] In some embodiments, the user interface logic 1008 may cause a graphical representation of at least some of the predicted values ​​(generated by the parameter prediction logic 1004) of the set of parameters associated with regions of the specimen to be displayed on a display device (e.g., any of the display devices discussed herein). For example, the graphical representation 170 of FIG. 3B may include the graphical representation 160 of FIG. 3A (e.g., a “raw” image) with a left dot and a right dot superimposed on each hole 104. The shading (or color, or other characteristic) of the left dot may indicate the predicted value of the parameter (e.g., ice thickness) associated with the corresponding hole 104 (e.g., darker dots indicate thicker ice, and vice versa). This particular example is merely illustrative, and the predicted values ​​of the parameters associated with regions of the specimen may be displayed in any suitable manner. The user interface logic 1008 may cause the predicted values ​​of the parameters to be displayed simultaneously with the display of the first data set (e.g., as shown in the graphical representation 170 of FIG. 3B), or may cause the predicted values ​​of the parameters to be displayed without the simultaneous display of the first data set.

[0026] In some embodiments, the user interface logic 1008 may cause a graphical representation of at least some of the measurements (generated by the measurement logic 1002) of a set of parameters associated with regions of the specimen to be displayed on a display device (e.g., any of the display devices discussed herein). For example, in the graphical representation 170 of FIG. 3B , the shading (or color, or other characteristic) of the right dot may indicate the measurement of the parameter (e.g., ice thickness) associated with the corresponding hole 104 (e.g., darker dots indicate thicker ice, and vice versa). This particular example is merely illustrative, and the measurements of the parameters associated with regions of the specimen may be shown in any suitable manner. Furthermore, although the graphical representation 170 of FIG. 3B shows both the predicted value (left dot) and the measured value (right dot) for each hole 104 (indicating that each hole 104 was imaged in the first and second imaging rounds), this is merely illustrative, and in some embodiments, less than all of the region imaged in the first imaging round is imaged in the second imaging round. The user interface logic 1008 may display the measured values ​​of the parameters simultaneously with the display of the first data set (e.g., as shown in the graphical representation 170 of FIG. 3B ), or may display the measured values ​​of the parameters without the simultaneous display of the first data set. In some embodiments, only predicted values ​​(e.g., left dots) of one or more of the parameters associated with the region of the specimen are displayed by the user interface logic 1008. In other embodiments, only measured values ​​(e.g., right dots) of one or more of the parameters associated with the region of the specimen are displayed by the user interface logic 1008, or both predicted values ​​(e.g., left dots) and measured values ​​(e.g., right dots) of one or more of the parameters associated with the region of the specimen may be displayed simultaneously.

[0027] In some embodiments, the user interface logic 1008 may display one or more performance metrics of the machine learning computational model. For example, the graphical representation 170 of FIG. 3B communicates the performance metrics of the machine learning computational model by depicting left and right dots side by side. If the predicted value (generated by the machine learning computational model) is the same as or close to the measured value, the left and right dots associated with a particular region (e.g., hole 104) have the same or similar shading. If the predicted value and the measured value differ significantly, the left and right dots associated with a particular region (e.g., hole 104) have different shadings to visually indicate the discrepancy. Other suitable ways of displaying the performance metrics of the machine learning computational model may be used. For example, the user interface logic 1008 may display a plot of error over time to show how the error of the machine learning computational model (e.g., generated by the training logic 1010 during validation of the machine learning computational model) changed as additional rounds of training were performed.

[0028] The training logic 1010 may be configured to train the machine learning computational model of the parameter prediction logic 1004 on a set of training data and to retrain the machine learning computational model when additional training data is received. As is known in the art, the training data may include a set of input-output pairs (e.g., an input "low-resolution" first image of a region of the specimen and a pair of corresponding measurements of parameters of the region of the specimen), and the training logic 1010 may use this training data to train the machine learning computational model (e.g., by adjusting weights and other parameters of the machine learning computational model) according to any suitable technique (e.g., a gradient descent algorithm). In some embodiments, the training logic 1010 may save a portion of the available training data for use as validation data, as is known in the art.

[0029] When a retraining condition is met, the training logic 1010 may retrain the machine learning computational model of the parameter prediction logic 1004. For example, in some embodiments, the training logic 1010 may retrain the machine learning computational model of the parameter prediction logic 1004 upon accumulation of a threshold number of “new” training datasets (e.g., 20 training datasets). In some embodiments, the training logic 1010 may retrain the machine learning computational model using available retraining datasets upon receiving a retraining command from a user (e.g., via a GUI such as GUI 3000 of FIG. 7). In some embodiments, the training logic 1010 may retrain the machine learning computational model when one or more performance metrics of the machine learning computational model (e.g., errors of one or more validation datasets) meet one or more retraining criteria (e.g., error increase threshold from the previous training round). In some embodiments, the retraining condition may include any one or more of these conditions.

[0030] 4 is a flow diagram of a method 2000 of region selection in charged particle microscope imaging, according to various embodiments. While the operations of method 2000 may be described with reference to particular embodiments disclosed herein (e.g., CPM assistance module 1000 discussed herein with reference to FIG. 1 , GUI 3000 discussed herein with reference to FIG. 7 , computing device 4000 discussed herein with reference to FIG. 8 , and / or CPM assistance system 5000 discussed herein with reference to FIG. 9 ), method 2000 may be used in any suitable configuration to perform any suitable assistance operations. While operations are shown in FIG. 4 in a particular order, each once, the operations may be appropriately reordered and / or repeated as desired (e.g., different operations performed may be performed in parallel as desired).

[0031] At 2002, a first data set associated with a region of a specimen may be generated by processing data from a first round of imaging of the region with CPM. For example, imaging logic 1002 of CPM assistance module 1000 may perform the operations of 2002 according to any of the embodiments disclosed herein.

[0032] At 2004, a first dataset associated with the region may be provided to a machine learning computational model to generate predicted values ​​for a set of parameters for the region. For example, parameter prediction logic 1004 of CPM assistance module 1000 may perform the operations of 2004 according to any of the embodiments disclosed herein.

[0033] In 2006, based on the predicted values ​​of the set of parameters of the region, it may be determined whether a second imaging round of the region is to be performed by the charged particle microscope. For example, region selection logic 1006 of CMP assist module 1000 may perform the operations of 2006 according to any of the embodiments disclosed herein.

[0034] In 2008, in response to determining that the second round of imaging of the region is to be performed, a second data set associated with the region may be generated by processing data from the second round of imaging of the region by the charged particle microscope, the second data set including measurements of a set of parameters of the region. For example, imaging logic 1002 of CPM assistance module 1000 may perform the operations of 2008 according to any of the embodiments disclosed herein.

[0035] At 2010, a display may be caused on a display device of an indication of the difference between the predicted values ​​of the set of parameters of the region and the measured values ​​of the set of parameters of the region. For example, the user interface logic 1008 may perform the operations of 2010 according to any of the embodiments disclosed herein.

[0036] In 2012, the machine learning computational model may be retrained using the first data set and the second data set. For example, the training logic 1010 may perform the operations of 2012 according to any of the embodiments disclosed herein.

[0037] FIG. 5 is a flow diagram of a method 2050 of region selection in charged particle microscope imaging, according to various embodiments. The operations of method 2050 of FIG. 5 may be part of method 2000 of FIG. 4, as discussed further below. While the operations of method 2050 may be described with reference to particular embodiments disclosed herein (e.g., CPM assistance module 1000 discussed herein with reference to FIG. 1, GUI 3000 discussed herein with reference to FIG. 7, computing device 4000 discussed herein with reference to FIG. 8, and / or CPM assistance system 5000 discussed herein with reference to FIG. 9), method 2050 may be used in any suitable setting to perform any suitable assistance operations. Although operations are shown in FIG. 5 in a particular order, each once, the operations may be appropriately reordered and / or repeated as desired (e.g., different operations performed may be performed in parallel as desired).

[0038] At 120, a region counter variable i may be initialized to an initial value. For example, imaging logic 1002 of CPM assistance module 1000 may perform the operations of 120. While Figure 5 shows this initial value to be "1," any suitable region counter variable may be used and suitably initialized.

[0039] At 122, a first imaging round of region i may be performed. For example, imaging logic 1002 of CPM assistance module 1000 may cause CPM to perform a first "low-resolution" imaging round of region i, and imaging logic 1002 may generate a first data set for region i based on this first imaging round (e.g., according to any of the embodiments disclosed herein).

[0040] At 124, predicted parameter values ​​for region i may be generated. For example, parameter prediction logic 1004 of CPM assistance module 1000 may generate predicted values ​​for the set of parameters for region i based on a first imaging round (performed at 122) of the region (e.g., based on a first data set for region i generated by imaging logic 1002 according to any of the embodiments disclosed herein).

[0041] At 126, it may be determined whether region i is selected for the second imaging round. For example, region selection logic 1006 of CPM assistance module 1000 may determine whether region i is selected for the second imaging round according to any of the embodiments disclosed herein.

[0042] If it is determined at 126 that region i is selected for a second imaging round, method 2050 may proceed to 128, where the second imaging round of region i may be performed. For example, imaging logic 1002 of CPM assistance module 1000 may cause CPM to perform a second "high resolution" imaging round of region i, and imaging logic 1002 may generate a second dataset for region i based on the second imaging round (e.g., according to any of the embodiments disclosed herein).

[0043] At 130, measured parameter values ​​for region i may be generated. For example, imaging logic 1002 of CPM assistance module 1000 may generate measurements of a set of parameters for region i (e.g., included in a second data set for region i generated by imaging logic 1002 according to any of the embodiments disclosed herein) based on a second imaging round of the region (performed at 128).

[0044] At 132, it may be determined whether to retrain the parameter prediction logic. For example, the training logic 1010 may determine whether a retraining condition for the machine learning computational model of the parameter prediction logic 1004 has been met, according to any of the embodiments disclosed herein.

[0045] When it is determined at 132 that the parameter prediction logic should be retrained, the method 2050 may proceed to 134, where the parameter prediction logic may be retrained. Data generated in the first and second imaging rounds of region i may be used in the retraining. For example, the training logic 1010 may retrain the machine learning computational model of the parameter prediction logic according to any of the embodiments disclosed herein.

[0046] When region i is determined not to be selected for the second imaging round at 126, when a decision is made not to retrain the parameter prediction logic, or following 134, method 2050 may proceed to 136 where region counter variable i is incremented. For example, imaging logic 1002 may increment region counter variable i. Method 2050 may then return to 122 for a different region i, and operations 122-136 may be repeated until all regions have been imaged.

[0047] FIG. 6 is a flow diagram of a method 2100 for generating a machine learning computational model (e.g., for use by parameter prediction logic 1004) for region selection in CPM imaging, according to various embodiments. The operations of method 2100 of FIG. 6 may be used, for example, to generate an initial machine learning computational model for any suitable of the other methods disclosed herein. While the operations of method 2100 may be described with reference to particular embodiments disclosed herein (e.g., CPM assistance module 1000 discussed herein with reference to FIG. 1 , GUI 3000 discussed herein with reference to FIG. 7 , computing device 4000 discussed herein with reference to FIG. 8 , and / or CPM assistance system 5000 discussed herein with reference to FIG. 9 ), method 2100 may be used in any suitable setting to perform any suitable assistance operations. While the operations are shown in FIG. 6 in a particular order, each once, the operations may be appropriately reordered and / or repeated as needed (e.g., different operations performed may be performed in parallel as needed).

[0048] At 2102, a first machine learning computational model may be stored in association with a first element of interest. The first machine learning computational model may be for receiving as input a low-resolution charged particle microscope dataset representing a region of the specimen that includes the first element of interest and for outputting predicted values ​​of a set of parameters of the region of the specimen that includes the first element of interest. For example, the parameter prediction logic 1004 may perform the operations of 2102 (e.g., according to any of the embodiments disclosed herein).

[0049] At 2104, a user indication of a second element of interest different from the first element of interest may be received. For example, the user interface logic 1008 may perform the operations of 2104 (e.g., according to GUI 3000 of FIG. 7 or any other embodiment disclosed herein).

[0050] At 2106, the first machine learning computational model may be used to generate an initial second machine learning computational model associated with a second element of interest. The second machine learning computational model may be for receiving as input a low-resolution charged particle microscope dataset representing a region of the specimen that includes the second element of interest and for outputting predicted values ​​of a set of parameters of the region of the specimen that includes the second element of interest. For example, the parameter prediction logic 1004 may perform the operations of 2106 (e.g., according to any of the embodiments disclosed herein).

[0051] The CPM-assisted methods disclosed herein may involve interactions with a human user (e.g., via a user local computing device 5020 discussed herein with reference to FIG. 9). These interactions may include providing information to the user (e.g., information regarding the operation of a CPM such as CPM 5010 of FIG. 9, information regarding the sample being analyzed, or information regarding other tests or measurements performed by the CPM, information retrieved from a local or remote database, or other information), or providing the user with options for entering commands (e.g., to control the operation of a CPM such as CPM 5010 of FIG. 9 or to control the analysis of data generated by the CPM), queries (e.g., to a local or remote database), or other information. In some embodiments, these interactions may be performed via a graphical user interface (GUI) including a visual display on a display device (e.g., display device 4010 discussed herein with reference to FIG. 8) that provides output to the user and / or prompts the user to provide input (e.g., via one or more input devices, such as a keyboard, mouse, trackpad, or touchscreen, included in other I / O devices 4012 discussed herein with reference to FIG. 8). The CPM assistance system disclosed herein may include any suitable GUI for interaction with a user.

[0052] 7 shows an exemplary GUI 3000 that may be used to perform some or all of the assistance methods disclosed herein, according to various embodiments. As noted above, the GUI 3000 may be provided on a display device (e.g., display device 4010 discussed herein with reference to FIG. 8) of a computing device (e.g., computing device 4000 discussed herein with reference to FIG. 8) of a CPM assistance system (e.g., CPM assistance system 5000 discussed herein with reference to FIG. 9), and a user may interact with the GUI 3000 using any suitable input device (e.g., any of the input devices included in other I / O devices 4012 discussed herein with reference to FIG. 8) and input technology (e.g., cursor movement, motion capture, facial recognition, gesture detection, voice recognition, button activation, etc.).

[0053] GUI 3000 may include a data display area 3002, a data analysis area 3004, a CPM control area 3006, and a settings area 3008. The particular number and arrangement of areas shown in Figure 7 is merely illustrative, and any number and arrangement of areas, including any desired features, may be included in UI 3000.

[0054] The data display area 3002 may display data generated by a CPM (e.g., the CPM 5010 discussed herein with reference to FIG. 9). For example, the data display area 3002 may display a first data set and / or a second data set generated by the imaging logic 1002 for different regions of the specimen (e.g., the graphical representation 160 of FIG. 3A).

[0055] The data analysis area 3004 may display the results of the data analysis (e.g., the results of analyzing the data illustrated in the data display area 3002 and / or other data). For example, the data analysis area 3004 may display predicted values ​​of a set of parameters associated with a region of the specimen (e.g., as generated by the parameter prediction logic 1004) and / or measured values ​​of a set of parameters associated with a region of the specimen (e.g., as generated by the imaging logic 1002). For example, the data analysis area 3004 may display a graphical representation such as the graphical representation 170 of FIG. 3B (e.g., including a first data set associated with the region, predicted values ​​of the set of parameters associated with the region, and / or measured values ​​of a set of parameters associated with the region, as discussed above). In some embodiments, the data display area 3002 and the data analysis area 3004 may be combined within the GUI 3000 (e.g., to include data output from the CPM and some analysis of the data in a common graph or area).

[0056] The CPM control area 3006 may include options that allow a user to control the CPM (e.g., CPM 5010 discussed herein with reference to FIG. 9) or the analysis or processing of data generated by the CPM. For example, the CPM control area 3006 may include user-selectable options to retrain a machine learning computational model, generate a new machine learning computational model from a previous machine learning computational model (e.g., discussed above with reference to method 2100 of FIG. 6), or perform other control functions (e.g., confirm or update the output of the region selection logic 1006 to control the region imaged by the second imaging round).

[0057] The settings area 3008 may include options that enable the user to control features and functionality of the GUI 3000 (and / or other GUIs) and / or perform common computing operations related to the data display area 3002 and the data analysis area (3004) (e.g., saving data on a storage device such as the storage device 4004 discussed herein with respect to FIG. 8, sending data to another user, labeling data, etc.).

[0058] As noted above, CPM assistance module 1000 may be implemented by one or more computing devices. FIG. 8 is a block diagram of a computing device 4000 that may perform some or all of the CPM assistance methods disclosed herein, according to various embodiments. In some embodiments, CPM assistance module 1000 may be implemented by a single computing device 4000 or multiple computing devices 4000. Additionally, as discussed below, a computing device 4000 (or multiple computing devices 4000) implementing CPM assistance module 1000 may be part of one or more of CPM 5010, user local computing device 5020, service local computing device 5030, or remote computing device 5040 of FIG. 9.

[0059] Although the computing device 4000 of FIG. 8 is illustrated as having several components, any one or more of these components may be omitted or duplicated as suitable for the application and configuration. In some embodiments, some or all of the components included in the computing device 4000 may be mounted on one or more motherboards and enclosed in a housing (e.g., comprising plastic, metal, and / or other materials). In some embodiments, some of these components may be fabricated on a single system-on-chip (SoC) (e.g., an SoC may include one or more processing devices 4002 and one or more storage devices 4004). Furthermore, in various embodiments, the computing device 4000 may not include one or more components shown in FIG. 8, but may include interface circuitry (not shown) for coupling to one or more components using any suitable interface (e.g., a universal serial bus (USB) interface, a high-definition multimedia interface (HDMI®) interface, a controller area network (CAN) interface, a serial peripheral interface (SPI) interface, an Ethernet interface, a wireless interface, or any other suitable interface). For example, the computing device 4000 may not include a display device 4010, but may include display device interface circuitry (e.g., connectors and driver circuitry) to which the display device 4010 may be coupled.

[0060] The computing device 4000 may include a processing device 4002 (e.g., one or more processing devices). As used herein, the term “processing device” may refer to any device or portion of a device that processes electronic data from registers and / or memory and converts the electronic data into other electronic data that may be stored in registers and / or memory. The processing device 4002 may include one or more digital signal processors (DSPs), application specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptographic processors (specialized processors that execute cryptographic algorithms in hardware), server processors, or any other suitable processing devices.

[0061] The computing device 4000 may include a storage device 4004 (e.g., one or more storage devices). The storage device 4004 may include one or more memory devices, such as random access memory (RAM) (e.g., static RAM (SRAM) devices, magnetoresistive RAM (MRAM) devices, dynamic RAM (DRAM) devices, resistive RAM (RRAM) devices), or conductive bridging RAM (CBRAM) devices, hard drive-based memory devices, solid state memory devices, network drives, cloud drives, or any combination of memory devices. In some embodiments, the storage device 4004 may include memory that shares a die with the processing device 4002. In such embodiments, the memory may be used as cache memory and may include, for example, embedded dynamic random access memory (eDRAM) or spin transfer torque magnetic random access memory (STT-MRAM). In some embodiments, storage device 4004 may include a non-transitory computer-readable medium having instructions that, when executed by one or more processing devices (e.g., processing device 4002), cause computing device 4000 to perform any suitable one or portion of the methods disclosed herein.

[0062] The computing device 4000 may include an interface device 4006 (e.g., one or more interface devices 4006). The interface device 4006 may include one or more communication chips, connectors, and / or other hardware and software to manage communications between the computing device 4000 and other computing devices. For example, the interface device 4006 may include circuitry for managing wireless communications for transferring data to and from the computing device 4000. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communication channels, etc. that may communicate data through the use of modulated electromagnetic radiation over a non-solid medium. This term does not imply that the associated devices do not include any wires, although in some embodiments they may not. The circuitry included in interface device 4006 for managing wireless communications may implement any of several wireless standards or protocols, including, but not limited to, Institute of Electrical and Electronics Engineers (IEEE) standards including Wi-Fi (IEEE 802.11 family), the IEEE 802.16 standard (e.g., IEEE 802.16-2005 amendment), the Long Term Evolution (LTE) project with any amendments, updates, and / or revisions (e.g., also referred to as the Advanced LTE project, the Ultra Mobile Broadband (UMB) project ("3GPP®2"), etc.). In some embodiments, the circuitry included in interface device 4006 for managing wireless communications may operate in accordance with a Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE network.In some embodiments, the circuitry included in the interface device 4006 for managing wireless communications may operate in accordance with GSM Evolution High-Speed ​​Data (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). In some embodiments, the circuitry included in the interface device 4006 for managing wireless communications may operate in accordance with Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Communications (DECT), Evolution Data Optimized (EV-DO), and derivatives thereof, and any other wireless protocols designated as 3G, 4G, 5G, and beyond. In some embodiments, the interface device 4006 may include one or more antennas (e.g., one or more antenna arrays) for receiving and / or transmitting wireless communications.

[0063] In some embodiments, the interface device 4006 may include circuitry for managing wired communications, such as electrical, optical, or any other suitable communications protocol. For example, the interface device 4006 may include circuitry for supporting communications via Ethernet technology. In some embodiments, the interface device 4006 may support both wireless and wired communications, and / or may support multiple wired and / or wireless communications protocols. For example, a first set of circuits in the interface device 4006 may be dedicated to shorter-range wireless communications, such as Wi-Fi or Bluetooth, while a second set of circuits in the interface device 4006 may be dedicated to longer-range wireless communications, such as Global Positioning System (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, etc. In some embodiments, the first set of circuits in the interface device 4006 may be dedicated to wireless communications, and the second set of circuits in the interface device 4006 may be dedicated to wired communications.

[0064] Computing device 4000 may include battery / power circuitry 4008. Battery / power circuitry 4008 may include one or more energy storage devices (e.g., batteries or capacitors) and / or circuitry for coupling components of computing device 4000 to an energy source separate from computing device 4000 (e.g., AC line power).

[0065] The computing device 4000 may include a display device 4010 (e.g., multiple display devices). The display device 4010 may include any visual indicator, such as a heads-up display, a computer monitor, a projector, a touchscreen display, a liquid crystal display (LCD), a light-emitting diode display, or a flat panel display.

[0066] The computing device 4000 may include other input / output (I / O) devices 4012. The other I / O devices 4012 may include, for example, one or more audio output devices (e.g., speakers, headsets, earphones, alarms, etc.), one or more audio input devices (e.g., a microphone or microphone array, as known in the art), a location device (e.g., a GPS device that communicates with a satellite-based system to receive the location of the computing device 4000), an audio codec, a video codec, a printer, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors), vibration sensors, accelerometers, gyroscopes, etc.), an image capture device such as a camera, a keyboard, a cursor control device such as a mouse, stylus, trackball, touchpad, a barcode reader, a quick response (QR) code reader, or a radio frequency identification (RFID) reader, etc.

[0067] The computing device 4000 may have any form factor suitable for its use and configuration, such as a handheld or mobile computing device (e.g., a mobile phone, smartphone, mobile internet device, tablet computer, laptop computer, netbook computer, ultrabook computer, personal digital assistant (PDA), ultra-mobile personal computer, etc.), desktop computing device, or server computing device or other networked computing component).

[0068] One or more computing devices implementing any of the CPM assistance modules or methods disclosed herein may be part of a CPM assistance system. Figure 9 is a block diagram of an example CPM assistance system 5000 in which some or all of the CPM assistance methods disclosed herein may be performed, according to various embodiments. The CPM assistance modules and methods disclosed herein (e.g., CPM assistance module 1000 of Figure 1 and method 2000 of Figure 2) may be implemented by one or more CPMs 5010, user local computing devices 5020, service local computing devices 5030, or remote computing devices 5040 of the CPM assistance system 5000.

[0069] Any of the CPM 5010, the user local computing device 5020, the service local computing device 5030, or the remote computing device 5040 may include any of the embodiments of the computing device 4000 discussed in this specification with reference to FIG. 8, and any of the CPM 5010, the user local computing device 5020, the service local computing device 5030, or the remote computing device 5040 may take the form of any suitable one of the embodiments of the computing device 4000 discussed in this specification with reference to FIG. 8.

[0070] The CPM 5010, the user local computing device 5020, the service local computing device 5030, or the remote computing device 5040 may each include a processing device 5002, a storage device 5004, and an interface device 5006. The processing device 5002 may take any suitable form, including any form of the processing device 4002 discussed herein with reference to FIG. 4, and the processing devices 5002 included in different ones of the CPM 5010, the user local computing device 5020, the service local computing device 5030, or the remote computing device 5040 may take the same form or different forms. The storage device 5004 may take any suitable form, including any form of the storage device 5004 discussed herein with reference to FIG. 4, and the storage devices 5004 included in different ones of the CPM 5010, the user local computing device 5020, the service local computing device 5030, or the remote computing device 5040 may take the same form or different forms. The interface device 5006 may take any suitable form, including any of the forms of the interface device 4006 discussed herein with reference to FIG. 4, and the interface devices 5006 included in different ones of the CPM 5010, the user local computing device 5020, the service local computing device 5030, or the remote computing device 5040 may take the same or different forms.

[0071] The CPM 5010, the user local computing device 5020, the service local computing device 5030, and the remote computing device 5040 may communicate with other elements of the CPM assistance system 5000 via communication paths 5008. The communication paths 5008 may communicatively couple the interface devices 5006 of different ones of the elements of the CPM assistance system 5000, as shown, and may be wired or wireless communication paths (e.g., in accordance with any of the communication techniques discussed herein with reference to the interface device 4006 of the computing device 4000 of FIG. 8). While the particular CPM assistance system 5000 shown in FIG. 9 includes communication paths between each pair of the CPM 5010, the user local computing device 5020, the service local computing device 5030, and the remote computing device 5040, this “fully connected” implementation is merely illustrative, and in various embodiments, various ones of the communication paths 5008 may not be present. For example, in some embodiments, the service local computing device 5030 may not have a direct communication path 5008 between its interface device 5006 and the interface device 5006 of the CPM 5010, and instead may communicate with the CPM 5010 via a communication path 5008 between the service local computing device 5030 and the user local computing device 5020, and a communication path 5008 between the user local computing device 5020 and the CPM 5010. The CPM 5010 may include any suitable CPM, such as a SEM, TEM, STEM, or ion beam microscope.

[0072] The user local computing device 5020 may be a computing device that is local to the user of the CPM 5010 (e.g., according to any of the embodiments of the computing device 4000 discussed herein). In some embodiments, the user local computing device 5020 may also be local to the CPM 5010, although this need not be the case. For example, a user local computing device 5020 in a user's home or office may be remote from the CPM 5010 but communicate with the CPM 5010, such that the user may use the user local computing device 5020 to control and / or access data from the CPM 5010. In some embodiments, the user local computing device 5020 may be a laptop, smartphone, or tablet device. In some embodiments, the user local computing device 5020 may be a portable computing device.

[0073] The service local computing device 5030 may be a computing device that is local to an entity that services the CPM 5010 (e.g., according to any of the embodiments of computing device 4000 discussed herein). For example, the service local computing device 5030 may be local to the manufacturer of the CPM 5010 or a third-party service company. In some embodiments, the service local computing device 5030 may communicate with the CPM 5010, the user local computing device 5020, and / or the remote computing device 5040 (e.g., via a direct communication path 5008 or via multiple “indirect” communication paths 5008, as discussed above) and receive data regarding the operation of the CPM 5010, the user local computing device 5020, and / or the remote computing device 5040 (e.g., results of self-tests of the CPM 5010, calibration coefficients used by the CPM 5010, measurements of sensors associated with the CPM 5010, etc.). In some embodiments, the service local computing device 5030 may communicate with the CPM 5010, the user local computing device 5020, and / or the remote computing device 5040 (e.g., via a direct communication path 5008 or via multiple "indirect" communication paths 5008, as discussed above), and send data to the CPM 5010, the user local computing device 5020, and / or the remote computing device 5040 (e.g., to update programmed instructions such as firmware within the CPM 5010, initiate the execution of a test or calibration sequence for the CPM 5010, update programmed instructions such as software in the user's local computing device 5020 or the remote computing device 5040, etc.).A user of the CPM 5010 can utilize the CPM 5010 or the user local computing device 5020 to communicate with the service local computing device 5030 to report problems with the CPM 5010 or the user local computing device 5020, to request a technician visit to improve the operation of the CPM 5010, to order consumables or replacement parts related to the CPM 5010, or for other purposes.

[0074] The remote computing device 5040 may be a computing device (e.g., according to any of the embodiments of computing device 4000 discussed herein) that is remote from the CPM 5010 and / or the user local computing device 5020. In some embodiments, the remote computing device 5040 may be included in a data center or other large-scale server environment. In some embodiments, the remote computing device 5040 may include network-attached storage (e.g., as part of the storage device 5004). The remote computing device 5040 may store data generated by the CPM 5010, perform analysis of data generated by the CPM 5010 (e.g., according to programmed instructions), facilitate communications between the user local computing device 5020 and the CPM 5010, and / or facilitate communications between the service local computing device 5030 and the CPM 5010.

[0075] In some embodiments, one or more elements of the CPM assistance system 5000 illustrated in FIG. 9 may not be present. Furthermore, in some embodiments, multiple ones of various ones of the elements of the CPM assistance system 5000 of FIG. 9 may be present. For example, the CPM assistance system 5000 may include multiple user local computing devices 5020 (e.g., different user local computing devices 5020 associated with different users or in different locations). In another example, the CPM assistance system 5000 may include multiple CPMs 5010 all in communication with a service local computing device 5030 and / or a remote computing device 5040. In such an embodiment, the service local computing device 5030 may monitor these multiple CPMs 5010, and the service local computing device 5030 may trigger updates or “broadcast” other information to the multiple CPMs 5010 simultaneously. Different CPMs 5010 in the CPM assistance system 5000 may be located near each other (e.g., in the same room) or far from each other (e.g., on different floors of a building, in different buildings, in different cities, etc.). In some embodiments, the CPM 5010 may be connected to an Internet-of-Things (IoT) stack that allows command and control of the CPM 5010 via web-based applications, virtual or augmented reality applications, mobile applications, and mobile and / or desktop applications. Any of these applications may be accessed by a user operating a user local computing device 5020 in communication with the CPM 5010 by way of an intervening remote computing device 5040. In some embodiments, the CPM 5010 may be sold by a manufacturer as part of a local CPM computing unit 5012 along with one or more associated user local computing devices 5020.

[0076] The following paragraphs provide various examples of the embodiments disclosed herein.

[0077] Example 1 is a charged particle microscope support device including: first logic for generating a first data set associated with a region of a specimen by processing data from a first round of imaging of the region with the charged particle microscope; second logic for providing the first data set associated with the region to a machine learning computational model to generate predicted values ​​of a set of parameters for the region; third logic for determining whether a second round of imaging of the region will be performed with the charged particle microscope based on the predicted values ​​of the set of parameters for the region, wherein the first logic is for generating a second data set associated with the region by processing data from the second round of imaging of the region with the charged particle microscope in response to a determination by the third logic that a second round of imaging of the region will be performed, the second data set including measured values ​​of the set of parameters for the region; and fourth logic for retraining the machine learning computational model using the first data set and the second data set.

[0078] Example 2 includes the subject matter described in Example 1 and further includes a charged particle microscope.

[0079] Example 3 includes the subject matter of Example 1 or 2, further specifying that the acquisition time of the first imaging round is less than the acquisition time of the second imaging round.

[0080] Example 4 includes the subject matter of any of Examples 1-3, further specifying that the radiation dose of the first imaging round is less than the radiation dose of the second imaging round.

[0081] Example 5 includes the subject matter of any of Examples 1-4, further specifying that the machine learning computational model includes a multi-layer neural network model.

[0082] Example 6 includes the subject matter of any of Examples 1-5, further specifying that the second logic is for generating predicted values ​​of the set of parameters for other regions of the specimen by providing other datasets associated with the other regions to an image processing computational model that is different from the machine learning computational model before providing the first dataset associated with the region to the machine learning computational model to generate predicted values ​​of the set of parameters for the region; and for generating predicted values ​​of the sets of parameters for the other regions of the specimen; and the first logic is for generating other datasets associated with the other regions of the specimen by processing data from imaging rounds of the other regions with a charged particle microscope before generating the first dataset.

[0083] Example 7 includes the subject matter of example 6, and further specifies that the image processing computational model includes linear regression or histogram binarization.

[0084] Example 8 includes the subject matter of any of examples 1-7, further specifying that the set of parameters includes a data quality indicator.

[0085] Example 9 includes the subject matter of any of Examples 1-8, further specifying that the set of parameters includes ice thickness.

[0086] Example 10 includes the subject matter of any of Examples 1-9, further specifying that the set of parameters includes a number of elements of interest, a level of contamination, an amount of specimen movement, an amount of specimen degradation, or a directional distribution of the elements of interest.

[0087] Example 11 includes the subject matter of any of Examples 1-10, further specifying that the second logic first generates the machine learning computational model before using the machine learning computational model.

[0088] Example 12 includes the subject matter of example 11, further specifying that the second logic is for initially generating the machine learning computational model by selecting random values ​​for at least some parameters of the machine learning computational model.

[0089] Example 13 includes the subject matter of example 11 or 12, further specifying that the second logic is for initially generating the machine learning computational model by adopting at least some parameters from another machine learning computational model.

[0090] Example 14 includes the subject matter of example 13, further specifying that the specimen includes a first element of interest, and the other machine learning computational model is associated with the specimen including a second element of interest, and the second logic is for initially generating the machine learning computational model using similarity between the first element of interest and the second element of interest by employing at least some parameters from the other machine learning computational model.

[0091] Example 15 includes the subject matter of Example 14, and further specifies that the first element of interest is a first protein, the second element of interest is a second protein, and the similarity between the first protein and the second protein is a protein sequence similarity between the first protein and the second protein.

[0092] Example 16 includes the subject matter of any of Examples 1-14, further specifying that the region of the specimen is a first region of the specimen; the first logic is for generating a first data set associated with a second region of the specimen different from the first region of the specimen by processing data from a first imaging round of the second region with the charged particle microscope; the second logic is for providing the first data set associated with the second region to a machine learning computational model to generate predicted values ​​of the set of parameters for the second region; and the third logic is for determining whether a second imaging round of the second region should be performed based on the predicted values ​​of the set of parameters of the second region; and the first logic is for not generating the second data set including measured values ​​of the set of parameters for the second region in response to a determination by the third logic that the second imaging round of the second region should not be performed.

[0093] Example 17 includes the subject matter of any of Examples 1-16, further specifying that the third logic is for determining, based on the probabilistic selection criteria, whether a second round of imaging of the region is performed by the charged particle microscope.

[0094] Example 18 includes the subject matter of any of Examples 1-17, further specifying that the fourth logic is for retraining the machine learning computational model upon accumulation of a threshold number of retraining datasets.

[0095] Example 19 includes the subject matter of any of Examples 1-18, further specifying that the fourth logic is for retraining the machine learning computational model in response to a retrain command from a user.

[0096] Example 20 includes the subject matter of any of Examples 1-19, further specifying that the fourth logic is for retraining the machine learning computational model when one or more performance metrics of the machine learning computational model meet one or more retraining criteria.

[0097] Example 21 includes the subject matter of any of Examples 1-20, further specifying that the first data set includes images having a lower resolution than the images of the second data set.

[0098] Example 22 includes the subject matter of any of Examples 1-21, further specifying that the set of parameters for the region is a single parameter.

[0099] Example 23 includes the subject matter of any of Examples 1-22, further including fifth logic for displaying on a display device a graphical representation of at least some of the first data set associated with the region.

[0100] Example 24 includes the subject matter of any of Examples 1-23, further including fifth logic for displaying on a display device a graphical representation of at least some of the predicted values ​​of the set of parameters of the region.

[0101] Example 25 includes the subject matter of any of Examples 1-24, further including fifth logic for displaying a graphical representation of at least some of the measurements of the set of parameters of the region on a display device.

[0102] Example 26 includes the subject matter of any of Examples 1-25, further including fifth logic for displaying, on a display device, a graphical representation of at least some of the first data set associated with the region simultaneously with a graphical representation of at least some of the predicted values ​​of the parameter set for the region.

[0103] Example 27 includes the subject matter of any of Examples 1-26, further including fifth logic for displaying, on a display device, a graphical representation of at least some of the first data set associated with the region concurrently with a graphical representation of at least some of the predicted values ​​of the set of parameters for the region and a graphical representation of at least some of the measured values ​​of the set of parameters for the region.

[0104] Example 28 includes the subject matter of any of Examples 1-27, further including fifth logic for displaying, on the display device, a graphical representation of at least some of the first data set associated with the region simultaneously with a graphical representation of at least some of the measurements of the parameter set for the region.

[0105] Example 29 includes the subject matter of any of Examples 1-28, further including fifth logic for displaying, on the display device, a graphical representation of at least some of the predicted values ​​of the set of parameters of the region simultaneously with a graphical representation of at least some of the measured values ​​of the set of parameters of the region.

[0106] Example 30 includes the subject matter of any of Examples 1-29, further including fifth logic for displaying, on a display device, one or more indicators of one or more performance metrics of the machine learning computational model.

[0107] Example 31 includes the subject matter of any of Examples 1-30, further specifying that the third logic is for determining whether a second round of imaging of the region is performed by the charged particle microscope based on the predicted values ​​of the set of parameters of the region by comparing at least some of the predicted values ​​of the set of parameters of the region to one or more selection criteria.

[0108] Example 32 includes the subject matter of Example 31, and further specifies that the third logic is for determining whether a second round of imaging of the region is to be performed by the charged particle microscope based on the predicted values ​​of the set of parameters of the region, by determining that a second round of imaging of the region is to be performed when the predicted values ​​of the set of parameters of the region indicate that the region is likely to have good data quality.

[0109] Example 33 includes the subject matter of example 31 or 32, and further specifies that the third logic is for determining whether to perform a second round of imaging of the region by the charged particle microscope based on the predicted values ​​of the set of parameters of the region, by determining that a second round of imaging of the region is performed when the predicted values ​​of the set of parameters of the region indicate that the region is likely to have insufficient data quality.

[0110] Example 34 is a charged particle microscope support device including: first logic for generating a first data set associated with a region of a specimen by processing data from a first round of imaging of the region by the charged particle microscope; second logic for providing the first data set associated with the region to a machine learning computational model to generate predicted values ​​of a set of parameters of the region, the first logic for generating a second data set associated with the region by processing data from a second round of imaging of the region by the charged particle microscope, the second data set including measured values ​​of the set of parameters of the region; and third logic for displaying on a display device an indication of the difference between the predicted values ​​of the set of parameters of the region and the measured values ​​of the set of parameters of the region.

[0111] Example 35 includes the subject matter of example 34, further including a charged particle microscope.

[0112] Example 36 includes the subject matter of Example 34 or 35, further specifying that the acquisition time of the first imaging round is less than the acquisition time of the second imaging round.

[0113] Example 37 includes the subject matter of any of Examples 34-36, further specifying that the radiation dose of the first imaging round is less than the radiation dose of the second imaging round.

[0114] Example 38 includes the subject matter of any of Examples 34-37, further specifying that the machine learning computational model includes a multilayer neural network model.

[0115] Example 39 includes the subject matter of any of Examples 34-38, further specifying that the second logic is for generating predicted values ​​for the set of parameters of other regions of the specimen by passing other datasets to a non-machine learning computational model before providing the first dataset associated with the region to the machine learning computational model to generate predicted values ​​for the set of parameters of the region; and the first logic is for generating other datasets associated with other regions of the specimen by processing data from imaging rounds of the other regions with a charged particle microscope before generating the first dataset.

[0116] Example 40 includes the subject matter of example 39, further specifying that the non-machine learning computational model includes linear regression or histogram binarization.

[0117] Example 41 includes the subject matter of any of Examples 34-40, further specifying that the set of parameters includes a data quality indicator.

[0118] Example 42 includes the subject matter of any of examples 34-41, further specifying that the set of parameters includes ice thickness.

[0119] Example 43 includes the subject matter of any of examples 34-42, further specifying that the set of parameters includes a number of elements of interest, a level of contamination, an amount of specimen movement, an amount of specimen degradation, or a directional distribution of the elements of interest.

[0120] Example 44 includes the subject matter of any of Examples 34-43, further specifying that the second logic first generates the machine learning computational model before using the machine learning computational model.

[0121] Example 45 includes the subject matter of example 44, further specifying that the second logic is for initially generating the machine learning computational model by selecting random values ​​for at least some parameters of the machine learning computational model.

[0122] Example 46 includes the subject matter of example 44 or 45, further specifying that the second logic is for initially generating the machine learning computational model by adopting at least some parameters from other machine learning computational models.

[0123] Example 47 includes the subject matter of Example 46, further specifying that the specimen includes a first element of interest, and the other machine learning computational model is associated with the specimen including a second element of interest, and the second logic is for generating the initial machine learning computational model using similarity between the first element of interest and the second element of interest by employing at least some parameters from the other machine learning computational model.

[0124] Example 48 includes the subject matter of Example 47, and further specifies that the first element of interest is a first protein, the second element of interest is a second protein, and the similarity between the first protein and the second protein is a protein sequence similarity between the first protein and the second protein.

[0125] Example 49 includes the subject matter of any of Examples 34-48, further specifying that the charged particle microscope comprises a scanning electron microscope (SEM), a transmission electron microscope (TEM), a scanning transmission electron microscope (STEM), or an ion beam microscope.

[0126] Example 50 includes the subject matter of any of Examples 34-49, further specifying that the region of the specimen is a first region of the specimen, the first logic is for generating a first data set associated with a second region of the specimen that is different from the first region of the specimen by processing data from a first round of imaging of the second region with the charged particle microscope, the second logic is for providing the first data set associated with the second region to a machine learning computational model to generate predicted values ​​of the set of parameters for the second region, and the first logic is for not generating a second data set including measured values ​​of the set of parameters for the second region.

[0127] Example 51 includes the subject matter of any of Examples 34-50, further including fourth logic for retraining the machine learning computational model using the first dataset and the second dataset.

[0128] Example 52 includes the subject matter described in Example 51, and further specifies that the fourth logic is for retraining the machine learning computational model upon accumulation of a threshold number of retraining datasets.

[0129] Example 53 includes the subject matter of example 51 or 52, and further specifies that the fourth logic is for retraining the machine learning computational model in response to a retraining command from a user.

[0130] Example 54 includes the subject matter of any of Examples 51-53, further specifying that the fourth logic is for retraining the machine learning computational model when one or more performance metrics of the machine learning computational model meet one or more retraining criteria.

[0131] Example 55 includes the subject matter of any of examples 34-54, further specifying that the first data set includes images having a lower resolution than the images of the second data set.

[0132] Example 56 includes the subject matter of any of Examples 34-55, further specifying that the set of parameters for the region is a single parameter.

[0133] Example 57 includes the subject matter of any of Examples 34-56, and further specifies that the third logic is for displaying on a display device a graphical representation of at least some of the first dataset associated with the region.

[0134] Example 58 includes the subject matter of any of Examples 34-57, and further specifies that the third logic is for displaying a graphical representation of at least some of the predicted values ​​of the set of parameters of the region on a display device.

[0135] Example 59 includes the subject matter of any of Examples 34-58, and further specifies that the third logic is for displaying a graphical representation of at least some of the measurements of the set of parameters of the region on a display device.

[0136] Example 60 includes the subject matter of any of Examples 34-59, and further specifies that the third logic is for displaying, on the display device, a graphical representation of at least some of the first dataset associated with the region simultaneously with a graphical representation of at least some of the predicted values ​​of the parameter set for the region.

[0137] Example 61 includes the subject matter of any of Examples 34-60, and further specifies that the third logic is for causing, on the display device, a graphical representation of at least some of the first data set associated with the region simultaneously with a graphical representation of at least some of the predicted values ​​of the set of parameters for the region and a graphical representation of at least some of the measured values ​​of the set of parameters for the region.

[0138] Example 62 includes the subject matter of any of Examples 34-61, and further specifies that the third logic is for displaying, on the display device, a graphical representation of at least some of the first data set associated with the region simultaneously with a graphical representation of at least some of the measurements of the parameter set for the region.

[0139] Example 63 includes the subject matter of any of Examples 34-62, further specifying that the indication of the difference between the predicted values ​​of the set of parameters of the region and the measured values ​​of the set of parameters of the region includes a graphical representation of at least some of the predicted values ​​of the set of parameters of the region simultaneously with a graphical representation of at least some of the measured values ​​of the set of parameters of the region.

[0140] Example 64 includes the subject matter of any of Examples 34-63, and further specifies that the third logic is for displaying, on the display device, one or more indicators of one or more performance metrics of the machine learning computational model.

[0141] Example 65 includes the subject matter of any of Examples 34 to 64, further including fifth logic for determining whether a second imaging round of the region is to be performed by the charged particle microscope based on predicted values ​​of the set of parameters of the region, and third logic for generating a second dataset associated with the region in response to a determination by the fifth logic that a second imaging round of the region is to be performed.

[0142] Example 66 includes the subject matter of example 65, and further specifies that the fifth logic is also for determining, based on the probabilistic selection criteria, whether a second imaging round of the region is performed by the charged particle microscope.

[0143] Example 67 includes the subject matter of any of Examples 65-66, further specifying that the fifth logic is for determining whether a second round of imaging of the region is performed by the charged particle microscope based on the predicted values ​​of the set of parameters of the region by comparing at least some of the predicted values ​​of the set of parameters of the region to one or more selection criteria.

[0144] Example 68 includes the subject matter described in Example 67, further specifying that the fifth logic is for determining whether a second round of imaging of the region is performed by the charged particle microscope based on the predicted values ​​of the set of parameters of the region, by determining that a second round of imaging of the region is performed when the predicted values ​​of the set of parameters of the region indicate that the region is likely to have good data quality.

[0145] Example 69 includes the subject matter of example 67 or 68, further specifying that the fifth logic is for determining whether to perform a second round of imaging of the region by the charged particle microscope based on the predicted value of the set of parameters of the region, by determining that a second round of imaging of the region is performed when the predicted value of the set of parameters of the region indicates that the region is likely to have insufficient data quality.

[0146] Example 70 is a charged particle microscope support device, comprising: first logic for storing a first machine learning computational model associated with a first element of interest, the first machine learning computational model receiving as input a low-resolution charged particle microscope dataset representing a region of the specimen including the first element of interest and for outputting predicted values ​​of a set of parameters for the region of the specimen including the first element of interest; and second logic for receiving a user indication of a second element of interest different from the first element of interest, the first logic for using the first machine learning computational model to generate an initial second machine learning computational model associated with the second element of interest, the second machine learning computational model receiving as input a low-resolution charged particle microscope dataset representing a region of the specimen including the second element of interest and for outputting predicted values ​​of a set of parameters for the region of the specimen including the second element of interest.

[0147] Example 71 includes the subject matter of Example 70, and further specifies that the first logic is for generating an initial second machine learning computational model by selecting random values ​​for at least some parameters of the first machine learning computational model.

[0148] Example 72 includes the subject matter of example 70 or 71, and further specifies that the first logic is for generating an initial second machine learning computational model by adopting at least some parameters from the first machine learning computational model.

[0149] Example 73 includes the subject matter of Example 72, and further specifies that the first logic is for generating an initial second machine learning computational model using similarity between the first element of interest and the second element of interest.

[0150] Example 74 includes the subject matter of Example 73, and further specifies that the first element of interest is a first protein, the second element of interest is a second protein, and the similarity between the first protein and the second protein is a protein sequence similarity between the first protein and the second protein.

[0151] Example 75 includes the subject matter of any of Examples 70-74, further including: third logic for generating a first dataset associated with a specific region of the specimen including a second element of interest by processing data from a first imaging round of the specific region by a charged particle microscope, the first dataset including a low-resolution charged particle microscope dataset representing the specific region; and the first logic for providing the first dataset associated with the specific region to a second machine learning computational model to generate predicted values ​​of a set of parameters for the specific region based on the predicted values ​​of the set of parameters for the specific region. The system further includes fourth logic for determining whether a second imaging round should be performed by a charged particle microscope, wherein the third logic, in response to a determination by the fourth logic that a second imaging round of the specific region should be performed, processes data from the second imaging round of the specific region by the charged particle microscope to generate second data associated with the specific region, the second dataset including measurements of a set of parameters of the specific region; and fifth logic for retraining a second machine learning computational model using the first dataset and the second dataset.

[0152] Example 76 includes the subject matter of Example 75, further specifying that the acquisition time of the first imaging round is less than the acquisition time of the second imaging round.

[0153] Example 77 includes the subject matter of Example 75 or 76, further specifying that the radiation dose of the first imaging round is less than the radiation dose of the second imaging round.

[0154] Example 78 includes the subject matter of any of Examples 75-77, further specifying that the first logic is for providing other datasets to an image processing computational model different from the machine learning computational model to generate predicted values ​​for the set of parameters of other regions of the specimen before providing the first dataset associated with the particular region to a second machine learning computational model to generate predicted values ​​for the set of parameters of the particular region; and the third logic is for generating other datasets associated with other regions of the specimen by processing data from imaging rounds of the other regions with a charged particle microscope before generating the first dataset.

[0155] Example 79 includes the subject matter of example 78, further specifying that the image processing computational model includes linear regression or histogram binarization.

[0156] Example 80 includes the subject matter of any of Examples 75-79, further specifying that the specific region of the specimen is a first region of the specimen; the third logic is for generating a first dataset associated with a second region of the specimen different from the first region of the specimen by processing data from a first imaging round of the second region with the charged particle microscope; the first logic is for providing the first dataset associated with the second region to a second machine learning computational model to generate predicted values ​​of the set of parameters for the second region; the fourth logic is for determining whether a second imaging round of the second region should be performed based on the predicted values ​​of the set of parameters of the second region; and the third logic is for not generating the second dataset including measured values ​​of the set of parameters of the second region in response to a determination by the fourth logic that a second imaging round of the second region should not be performed.

[0157] Example 81 includes the subject matter of any of Examples 75-80, further specifying that the fourth logic is for determining, based on the probabilistic selection criteria, whether a second imaging round of the second region is performed by the charged particle microscope.

[0158] Example 82 includes the subject matter of any of Examples 75-81, and further specifies that the fifth logic is for retraining the second machine learning computational model upon accumulation of a threshold number of retraining datasets.

[0159] Example 83 includes the subject matter of any of Examples 75 to 82, and further specifies that the fifth logic is for retraining the second machine learning computational model in response to a retraining command from a user.

[0160] Example 84 includes the subject matter of any of Examples 75-83, further specifying that the fifth logic is for retraining the second machine learning computational model when one or more performance metrics of the second machine learning computational model satisfy one or more retraining criteria.

[0161] Example 85 includes the subject matter of any of Examples 74-83, further specifying that the first dataset includes images having a lower resolution than the images of the second dataset.

[0162] Example 86 includes the subject matter of any of Examples 75-85, further specifying that the set of parameters for the region is a single parameter.

[0163] Example 87 includes the subject matter of any of Examples 75-86, and further specifies that the second logic is for displaying on a display device a graphical representation of at least some of the first data set associated with the particular region.

[0164] Example 88 includes the subject matter of any of Examples 75-87, and further specifies that the second logic is for displaying on a display device a graphical representation of at least some of the predicted values ​​of the set of parameters for the particular region.

[0165] Example 89 includes the subject matter of any of Examples 75-88, and further specifies that the second logic is for causing a graphical representation of at least some of the measurements of the set of parameters for the particular region to be displayed on the display device.

[0166] Example 90 includes the subject matter of any of Examples 75-89, and further specifies that the second logic is for displaying, on the display device, a graphical representation of at least some of the first data set associated with the particular region simultaneously with a graphical representation of at least some of the predicted values ​​of the parameter set for the particular region.

[0167] Example 91 includes the subject matter of any of Examples 75-90, and further specifies that the second logic is for causing, on the display device, a graphical representation of at least some of the first data set associated with the particular region simultaneously with a graphical representation of at least some of the predicted values ​​of the set of parameters for the particular region and a graphical representation of at least some of the measured values ​​of the set of parameters for the particular region.

[0168] Example 92 includes the subject matter of any of Examples 75-91, and further specifies that the second logic is for causing a graphical representation of at least some of the first data set associated with the particular region to be displayed on the display device simultaneously with a graphical representation of at least some of the measurements of the parameter set for the particular region.

[0169] Example 93 includes the subject matter of any of Examples 75-92, and further specifies that the second logic is for causing the display device to display, simultaneously with the graphical representation of at least some of the measured values ​​of the set of parameters for the particular region, a graphical representation of at least some of the predicted values ​​of the set of parameters for the particular region.

[0170] Example 94 includes the subject matter of any of Examples 75-93, and further specifies that the second logic is for displaying, on the display device, one or more indicators of one or more performance metrics of the second machine learning computational model.

[0171] Example 95 includes the subject matter of any of Examples 75-94, further specifying that the fourth logic is for determining whether a second imaging round of the specific region is performed by the charged particle microscope based on the predicted values ​​of the set of parameters of the specific region by comparing at least some of the predicted values ​​of the set of parameters of the specific region to one or more selection criteria.

[0172] Example 96 includes the subject matter described in Example 95, and further specifies that the fourth logic is for determining whether a second imaging round of the specific region is to be performed by the charged particle microscope based on the predicted value of the set of parameters of the specific region, by determining that a second imaging round of the specific region is to be performed when the predicted value of the set of parameters of the specific region indicates that the specific region is likely to have good data quality.

[0173] Example 97 includes the subject matter of example 95 or 96, and further specifies that the fourth logic is for determining whether a second round of imaging of the specific region is to be performed by the charged particle microscope based on the predicted value of the set of parameters of the specific region, by determining that a second round of imaging of the specific region is to be performed when the predicted value of the set of parameters of the specific region indicates that the specific region is likely to have insufficient data quality.

[0174] Example 98 includes the subject matter of any of Examples 70-97, further including a charged particle microscope.

[0175] Example 99 includes the subject matter of any of Examples 70-98, further specifying that the second machine learning computational model includes a multilayer neural network model.

[0176] Example 100 includes the subject matter of any of Examples 70-99, further specifying that the set of parameters includes a data quality indicator.

[0177] Example 101 includes the subject matter of any of Examples 70-100, further specifying that the set of parameters includes ice thickness.

[0178] Example 102 includes the subject matter of any of examples 70-101, further specifying that the set of parameters includes a number of elements of interest, a level of contamination, an amount of specimen movement, an amount of specimen degradation, or a directional distribution of the elements of interest.

[0179] Example A includes any of the CPM assistance modules disclosed herein.

[0180] Example B includes any of the methods disclosed herein.

[0181] Example C includes any of the GUIs disclosed herein.

[0182] Example D includes any of the CPM-assisted computing devices and systems disclosed herein.

Claims

Claim 1 A charged particle microscope support device, comprising: first logic for generating a first data set associated with the first region of the specimen by processing data from a first imaging round of the first region by a charged particle microscope; second logic for providing the first data set associated with the first region to a machine learning calculation model to generate a predicted value of a set of parameters of the first region; third logic for determining whether a second imaging round of the first region is to be performed by the charged particle microscope based on the predicted value of the set of parameters of the first region, wherein, in response to the determination by the third logic that the second imaging round of the first region is to be performed, the first logic is for generating a second data set associated with the first region by processing data from the second imaging round of the first region by the charged particle microscope, the second data set including measured values of the set of parameters of the first region; fourth logic for retraining the machine learning calculation model using the first data set associated with the first region and the second data set associated with the first region; wherein the first logic is for generating a first data set associated with a second region of the specimen different from the first region by processing data from a first imaging round of the second region by the charged particle microscope; the second logic is for providing the first data set associated with the second region to the machine learning calculation model to generate a predicted value of a set of parameters of the second region; the first logic is for not generating a second data set including measured values of the set of parameters of the second region, a charged particle microscope support device. Claim 2 The charged particle microscope support device according to claim 1, wherein an acquisition time of the first imaging round is less than an acquisition time of the second imaging round. Claim 3 The charged particle microscope support device according to claim 1, wherein a radiation dose of the first imaging round is less than a radiation dose of the second imaging round. Claim 4 Before the second logic provides the first dataset associated with the first region to the machine learning computational model to generate a predicted value of the set of parameters of the first region, by providing other data associated with other regions to an image processing computational model different from the machine learning computational model, a predicted value of the set of parameters of the other regions of the specimen is generated, and it is for generating a predicted value of the set of parameters of the other regions of the specimen, The charged particle microscope support device according to any one of claims 1 to 3, wherein the first logic is for generating the other dataset associated with the other region of the specimen by processing data from an imaging round of the other region by the charged particle microscope before generating the first dataset associated with the first region.

5. The charged particle microscope support device according to claim 4, wherein the image processing computational model includes linear regression or histogram binarization.

6. The charged particle microscope support device according to any one of claims 1 to 3, wherein the set of parameters includes a data quality indicator.

7. The charged particle microscope support device according to any one of claims 1 to 3, wherein the set of parameters includes the thickness of ice.

8. The charged particle microscope support device according to any one of claims 1 to 3, wherein the set of parameters includes the number of elements of interest, the level of contamination, the amount of movement of the specimen, the amount of degradation of the specimen, or the orientation distribution of the elements of interest.

9. A charged particle microscope support device, a first logic for generating a first dataset associated with a region of a specimen by processing data from a first imaging round of the region by a charged particle microscope, a second logic for providing the first dataset associated with the region to a machine learning computational model to generate a predicted value of a set of parameters of the region, wherein the first logic is for generating a second dataset associated with the region by processing data from a second imaging round of the region by the charged particle microscope, and the second dataset includes measured values of the set of parameters of the region, and the second logic, A third logic for causing a plurality of graphical labels superimposed on an image to be displayed on a display device, each graphical label corresponding to a range of the region and graphically depicting a difference between a corresponding one of the predicted values of the range and a corresponding one of the measured values of the range, and a charged particle microscope support device including the third logic.

10. The region of the specimen is a first region of the specimen, The first logic is for generating a first data set associated with a second region of the specimen different from the first region of the specimen by processing data from a first imaging round of the second region by the charged particle microscope, The second logic is for providing the first data set associated with the second region to the machine learning calculation model to generate a predicted value of a set of parameters of the second region, The charged particle microscope support device according to claim 9, wherein the first logic is for not generating a second data set including measured values of the set of parameters of the second region.

11. The charged particle microscope support device according to claim 9, further including a fourth logic for retraining the machine learning calculation model using the first data set and the second data set.

12. The charged particle microscope support device according to claim 9, wherein the first data set includes a first image having a lower resolution than a second image of the second data set.

13. The charged particle microscope support device according to any one of claims 9 to 12, wherein the third logic is for causing at least some graphical representations of the first data set associated with the region to be displayed on a display device.

14. The charged particle microscope support device according to any one of claims 9 to 12, wherein the third logic is for causing at least some graphical representations of the predicted values of the set of parameters of the region to be displayed on the display device.

15. The charged particle microscope support device according to any one of claims 9 to 12, wherein the third logic is for causing at least some graphic representations of the measured values of the set of parameters of the region to be displayed on the display device.

16. The charged particle microscope support device according to any one of claims 9 to 12, wherein the third logic is for causing one or more indicators of one or more performance metrics of the machine learning calculation model to be displayed on a display device.

17. A charged particle microscope support device, a first logic for storing a first machine learning calculation model related to a first element of interest, the first machine learning calculation model receiving, as an input, a low-resolution charged particle microscope dataset representing a region of a specimen including the first element of interest, and outputting a predicted value of a set of parameters of the region of the specimen including the first element of interest, a second logic for receiving a user index of a second element of interest different from the first element of interest, wherein the first logic is for generating an initial second machine learning calculation model related to the second element of interest using the first machine learning calculation model, the initial second machine learning calculation model receiving, as an input, a low-resolution charged particle microscope dataset representing a region of a specimen including the second element of interest, and outputting a predicted value of a set of parameters of the region of the specimen including the second element of interest.

18. The charged particle microscope support device according to claim 17, wherein the first logic is for generating an initial second machine learning calculation model by adopting at least some parameters from the first machine learning calculation model.

19. The charged particle microscope support device according to claim 18, wherein the first logic is for generating the initial second machine learning calculation model using the similarity between the first element of interest and the second element of interest.

20. The charged particle microscope support device according to claim 19, wherein the first element of interest is a first protein, the second element of interest is a second protein, and the similarity between the first protein and the second protein is the similarity of the protein sequences of the first protein and the second protein.

21. The charged particle microscope support device according to claim 9, wherein each graphical label includes a first indicator representing a corresponding one of the predicted values of the range and a second indicator representing a corresponding one of the measured values of the range.

Images