Digital sample model for instrumental optimization

Abstract

System and methods are disclosed for a scientific instrument support system. In at least one embodiment, the system includes a sample, a physical asset configured to interact with the sample, and one or more computing devices having executable instructions stored thereon. When executed, the executable instructions cause the one or more computing devices to generate one or more predictions based on simulations of one or more of the sample, the physical asset, or an interaction of the physical asset with the sample, and update a physics-based digital model of the sample based on accuracy of the one or more predictions. The physics-based digital model is used to reconstruct chemical and physical properties of the sample to generate the simulations of the sample.

Description

DIGITAL SAMPLE MODEL FOR INSTRUMENTAL OPTIMIZATIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Application No. US63 / 635,491, filed on April 17, 2024, titled “DIGITAL TWINS FOR SAMPLE-BEAM INTERACTIONS IN CHARGED PARTICLE MICROSCOPY,” the entire disclosure of which is incorporated herein by reference.BACKGROUND

[0002] Modern scientific instruments, such as charged particle microscopes, are often extremely complex systems with various interacting components. Many such instruments can only be successfully operated and controlled by highly skilled technicians, limiting their usefulness in scientific and commercial settings. Systems and methods to allow a more diverse range of user expertise to operate complex scientific instruments are therefore desirable.SUMMARY

[0003] As described herein, in one embodiment, a scientific instrument support system includes a sample, a physical asset configured to interact with the sample, and one or more computing devices having executable instructions stored thereon. When executed, the executable instructions cause the one or more computing devices to generate one or more predictions based on simulations of one or more of the sample, the physical asset, or an interaction of the physical asset with the sample. The executable instructions further cause the one or more computing devices to update a physics-based digital model of the sample based on accuracy of the one or more predictions, wherein the physics-based digital model is used to reconstruct chemical and physical properties of the sample to generate the simulations of the sample.

[0004] In at least one embodiment, a method includes receiving a request to perform an operation, using a physical asset, on a sample, and performing, in response to the request, one or more simulations of an interaction between the physical asset and the sample based, at least in part, on a digital model of the sample. The method further includes returning, based on theone or more simulations, optimized settings for the physical asset. The digital model is updated based on a comparison of an outcome of using the optimized settings to a predicted outcome.

[0005] In yet another embodiment, a non-transitory computer-readable medium stores computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform one or more simulations, using a physics-based model of a sample and a physics-based model of a physical asset, based on a requested outcome, and predict settings, based on the one or more simulations, to be used to operate the physical asset. The computer-executable instructions further cause the one or more processors to operate the physical asset using the predicted settings, compare an actual outcome of the operation of the physical asset to the requested outcome, and use the comparison to adjust the physics-based model of the sample.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0006] To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced.

[0007] FIG. 1 illustrates a computational system that includes a communicative coupling between a physical asset controller and a digital twins system, in accordance with one embodiment.

[0008] FIG. 2 illustrates a set of component models that may be included in a digital twins system, in accordance with one embodiment.

[0009] FIG. 3 illustrates a charged particle dual beam microscope system that may be simulated by a digital twins system, in accordance with one embodiment.

[0010] FIG. 4A illustrates an image generated by a physical asset, in accordance with one embodiment.

[0011] FIG. 4B illustrates analysis of the image of FIG. 4A by a digital twins system, in accordance with one embodiment.

[0012] FIGS. 5A-5D illustrate simulation of a milling plan by a digital twins system, in accordance with one embodiment.

[0013] FIG. 6 illustrates a system for simulating milling of a sample, in accordance with one embodiment.

[0014] FIG. 7 illustrates implementation of artificial intelligence with respect to a digital twins system, in accordance with one embodiment.

[0015] FIG. 8 illustrates a physical asset support module for performing support operations, in accordance with one embodiment.

[0016] FIG. 9 illustrates a method of performing support operations for a physical asset, in accordance with one embodiment.

[0017] FIG. 10 illustrates a method of using a digital twins system in conjunction with a physical asset to optimize operation of the physical asset, in accordance with one embodiment.

[0018] FIG. 11 illustrates a method of updating a digital twins system, in accordance with one embodiment.

[0019] FIG. 12 illustrates a method of training an Al-based model, in accordance with one embodiment.

[0020] FIG. 13 illustrates a graphical user interface that may be used to facilitate performance of at least some of the support operations for a physical asset described herein, in accordance with one embodiment.

[0021] FIG. 14 illustrates a computing device that may perform at least some of the support methods described herein, in accordance with one embodiment.

[0022] FIG. 15 illustrates a physical asset support system in which at least some of the support operations for a physical asset described herein may be performed, in accordance with one embodiment.DETAILED DESCRIPTION

[0023] Disclosed herein are physical asset (e.g., scientific instrument) support systems, as well as related methods, computing devices, and computer-readable media. For example, in some embodiments, the systems and methods disclosed herein may provide physics-based simulations of sample-beam interactions in charged particle microscopy, enabling improved simulation and control of complex microscopes. For example, the sample-beam interactions may include impingement of a charged particle beam on a sample and effects thereof. As described herein, physics-based simulations may be used for a physics-based digital model, where “physics-based” refers to use of principles and law of physics. The physics-based digital model may be used to reconstruct chemical and physical properties of, for example, a sample that is to be analyzed, measured, or otherwise interacted with by a physical asset. A set ofphysics-based simulations representing operation and control of a particular physical asset may also be referred to hereinafter as a digital twin of said scientific instrument and a set of physicsbased simulations representing a sample may also be referred to hereinafter as digital twin of said sample. The simulations may be used to support users of various scientific instrument systems where the users may have varying experience and expertise with respect to operation of a scientific instrument. Information generated by the simulations may guide the users through operation and control of the scientific instrument without demanding a high degree of previous knowledge, thereby decreasing an amount of time and effort demanded of users to be able to access capabilities of the scientific instrument.

[0024] The scientific instrument support embodiments disclosed herein may achieve improved performance relative to conventional approaches. For example, various ones of the embodiments disclosed herein may enable full simulation and control over a charged particle microscope and its interactions with a sample (e.g., during imaging, milling, or other interactions). The embodiments disclosed herein thus provide improvements to scientific instrument technology (e.g., improvements in the computer technology supporting such scientific instruments, among other improvements).

[0025] Various ones of the embodiments disclosed herein may improve upon conventional approaches to achieve the technical advantages of faster and more accurate simulation and control by more complete modeling. Such technical advantages are not achievable by routine and conventional approaches, and all users of systems including such embodiments may benefit from these advantages. The technical features of the embodiments disclosed herein are thus decidedly unconventional in the field of scientific instrumentation, as are the combinations of the features of the embodiments disclosed herein. As discussed further herein, various aspects of the embodiments disclosed herein may improve the functionality of a computer itself; for example, a computer supporting the operation of a charged particle microscope. The computational and user interface features disclosed herein do not only involve the collection and comparison of information, but apply new analytical and technical techniques to change the operation of charged particle microscopy systems. The present disclosure thus introduces functionality that neither a conventional computing device, nor a human, could perform.

[0026] Accordingly, the embodiments of the present disclosure may serve any of a number of technical purposes, such as controlling a specific technical system or process (e.g., charged particle microscopes); determining from measurements how to control a machine; andsimulating the behavior of a technical item or process (e.g., charged particle microscope systems and processes). In particular, the present disclosure provides technical solutions to technical problems, including but not limited to the challenges of simulation and control of complex scientific instruments when interacting with a sample.

[0027] The embodiments disclosed herein thus provide improvements to scientific instrument technology (e.g., improvements in the computer technology supporting scientific instrumentation, among other improvements).

[0028] In the following detailed description, reference is made to the accompanying drawings that form a part hereof wherein like numerals designate like parts throughout, and in which is shown, by way of illustration, embodiments that may be practiced. It is to be understood that other embodiments may be utilized, and structural or logical changes may be made, without departing from the scope of the present disclosure. Therefore, the following detailed description is not to be taken in a limiting sense.

[0029] Various operations may be described as multiple discrete actions or operations in turn, in a manner that is most helpful in understanding the subject matter disclosed herein. However, the order of description should not be construed as to imply that these operations are necessarily order dependent. In particular, these operations may not be performed in the order of presentation. Operations described may be performed in a different order from the described embodiment. Various additional operations may be performed, and / or described operations may be omitted in additional embodiments.

[0030] For the purposes of the present disclosure, the phrases "A and / or B" and "A or B" mean (A), (B), or (A and B). For the purposes of the present disclosure, the phrases "A, B, and / or C" and "A, B, or C" mean (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C). Although some elements may be referred to in the singular (e.g., “a processing device”), any appropriate elements may be represented by multiple instances of that element, and vice versa. For example, a set of operations described as performed by a processing device may be implemented with different ones of the operations performed by different processing devices. As used herein, the phrase “based on” should be understood to mean “based at least in part on,” unless otherwise specified.

[0031] The description uses the phrases "an embodiment," “various embodiments,” and "some embodiments," each of which may refer to one or more of the same or different embodiments. Furthermore, the terms "comprising," "including," "having," and the like, as used with respectto embodiments of the present disclosure, are synonymous. When used to describe a range of dimensions, the phrase "between X and Y" represents a range that includes X and Y. As used herein, an “apparatus” may refer to any individual device, collection of devices, part of a device, or collections of parts of devices. The drawings are not necessarily to scale.

[0032] FIG. l is a diagram of a computational system 100 in which a physical asset 102 (such as a charged particle microscope 106 interacting with a sample 108) and a digital twins system 104 that simulates the physical asset 102 are communicatively coupled (e.g., as indicated by dashed arrows). The physical asset 102 may be one of various types of charged particle microscopes, including, but not limited to, a scanning electron microscope (SEM) or a transmission electron microscope (TEM). The digital twins system 104 includes physics-based digital replications of the physical asset 102, and the physical sample 108, as illustrated by a synthetic physical asset 110 and a synthetic physical sample 112 in FIG. 1, as well as interactions between the physical asset 102 and the physical sample 108. In one example, the digital twins system 104 is implemented at a computing device 114 that may be coupled via a wireless or hardwire connection to a control system of the physical asset 102. In another example, the computing device 114 may host both the digital twins system 104 and the control system of the physical asset 102. The digital twins system 104 may include one or more digital twins (e.g., physics-based simulators) to predict each of a behavior of the physical asset 102, properties of the physical sample 108, and outcomes of interactions between the physical asset 102 and the physical sample 108.

[0033] This coupling allows the digital twins system 104 to generate a synthetic image 116 that simulates an instrument image, generated by the physical asset 102, of the physical sample 108. A digital twin system (e.g., including one or more digital twins) may therefore include a virtual representation of a full image construction system of a physical asset, using physicsbased models. It will be appreciated that although the digital twins system 104 is described in the context of generating images, a digital twins system may be used to simulate generation of a variety of types of data, besides images, as well as processes and operations performed by a physical asset, and behavior of a physical sample in response to the processes and operations.

[0034] In addition, in at least one embodiment, a digital twins system may be implemented using artificial intelligence techniques, such as machine learning, deep learning, natural language processing, computer vision, etc. In other examples, a digital twins system may be implemented using models of real-world systems, such as a physical asset and / or materials, andsuitable programming languages, algorithms, and software. Regardless of implementation mode, a digital twins system may be similarly verified, validated, and updated in a continuous and ongoing manner.

[0035] FIG. 2 illustrates a set of example component models 200 that may be included in one or more digital twins of a digital twins system. The component models 200 include software code (e.g., algorithms) that are implemented at a computing device, such as the computing device 114 of FIG. 1 or a computing device 1400 of FIG. 14. The software code is configured to simulate components of a physical asset, such as the physical asset 102 of FIG. 1, as well as data processing performed at a control system of the physical asset. For example, the physical asset may be a charged particle microscope. In at least one embodiment, the component models 200 allow simulation of the charged particle microscope and results generated by the charged particle microscope.

[0036] As shown in FIG. 2, the set of example component models 200 includes a source model 202 that simulates a source of charged particles (e.g., an electron beam source or an ion beam source), a beam optics model 204 that simulates manipulation of a charged particle beam produced by the source of charged particles, a probe formation model 206 that simulates generation of a focused beam of charged particles, and a scanning model 208 that simulates scanning the focused beam across a sample. The set of example component models 200 also includes a detectors model 210 that simulates capture of signals, at detectors of the physical asset, that are generated via interaction of the focused beam with the sample (e.g., secondary electron detection, backscattered electron detection, x-ray detection, transmitted electron detection, etc.), a signal acceptance model 212 that simulates capture and processing of the signals by the detectors to be used to as image data, a beam-sample interaction model 214 that simulates reactions and activities occurring at a sample and effects on the charged particle beam resulting from interaction between the beam and the sample, and a specimen definition model 216 that simulates visualization and distinguishing of features from the image data. The set of example component models 200 further includes an acquisition model 218 that simulates collection and recording of the image data, a raw image model 220 that simulates generation of an initial, unprocessed image that is captured by the detectors as signals are received, a digital image post processing model 222 that simulates application of various techniques to the initial image to enhance image quality, correct artifacts, and / or extract information therefrom, and asynthetic image model 224 that simulates generation of a final, processed image that would be output by a control system of the charged particle microscope.

[0037] A computational system, e.g., the computational system 100 of FIG. 1 that includes a digital twins system utilizing the set of component models 200, may therefore represent a complete feedback loop between the digital twins system and a physical asset (including, e.g., a tool). As noted above, such a computational system may generate an artificial image for specific sample and operating conditions, in one example. In some embodiments, the digital twins system may run in parallel with the physical asset, and the system may use this parallel operation to simulate, check, tune, or otherwise adjust the operation of the physical asset to achieve higher precision, fewer issues that require the intervention of a human technician, and increased image quality. In other words, the digital twin may be used to continuously update and optimize operational settings of the physical asset to yield a desired result of applying the physical asset to a sample. This may be achieved by continuously using feedback from the physical asset to update a model of the sample. In at least one embodiment, the feedback may be acquired in real-time, and in an automated manner, to update and increase an accuracy of a model that simulates the sample. Optimized settings may then be inferred and recommended when operations are subsequently to be performed on the same type of sample.

[0038] In some embodiments, a digital twins system may include a model (e.g., a digital twin) of a physical asset (e.g., a scientific instrument) and a model of a sample with which the physical asset interacts. The digital twin of the physical asset may be generated at least in part on a representation (e.g., a computer-aided design (CAD) drawing) of the physical asset. For example, as shown in FIG. 3, the physical asset may be a charged particle dual beam microscope system 300 (such as a focused ion beam-scanning electron microscope system, or FIB-SEM) that includes an electron source 302. The electron source 302 generates an electron beam that is propagated through an electron beam column 304.

[0039] The electron source 302 can include one or more emitters configured to generate electrons and to direct the electrons into the electron beam column 304. The emitters can include thermionic emitters, Schottky emitters, field-emission source emitters, or combinations thereof, operably coupled to power systems configured to apply a high-voltage (e.g., on the order of kilovolts to hundreds of kilovolts) to an emission region of the emitter material. The electron beam column 304 includes electromagnetic optics (e.g., electrostatic lenses, electromagnetic lenses, monochromators, etc.) and apertures configured to shape, focus,defocus, and direct the beam of electrons such that the beam is focused onto a sample 308, along beam axis A, in accordance with a set of operating parameters. The operating parameters can include a beam current, a beam energy (e.g., in volts, in electron volts, or the like), a magnification parameter, a scan pattern, a dwell time, and / or one or more pulse parameters. In this way, the example system 300 can function as an SEM to image portions of the sample 308 and / or can be used for e-beam assisted deposition of material onto the sample 308 (e.g., in coordination with a gas injector system, or GIS, 318)

[0040] The sample 308 is located within a vacuum chamber 312 of the charged particle dual beam microscope system 300, which allows the sample 308 to be maintained under a reduced pressure environment (e.g., lower than atmospheric pressure). A position of the sample relative to the electron beam column 304 may be adjusted using stage controls 306.

[0041] The charged particle dual beam microscope system 300 may further include an ion source 314 that generates an ion beam that is propagated through an FIB column 316. The ion source 314 can include one or more components configured to generate a beam of ions and to direct the ions, along a beam axis B, into the FIB column 316. As with the electron beam column 304, the FIB column can include electromagnetic optics (e.g., electrostatic lenses, electromagnetic lenses, monochromators, etc.) and apertures configured to shape, focus, defocus, and direct the beam of ion such that the beam is focused onto the sample 308, in accordance with a set of operating parameters. The operating parameters can include a beam current, a beam energy (e.g., in volts, in electron volts, or the like), a magnification parameter, a scan pattern, a dwell time, and / or one or more pulse parameters. In this way, the example system 300 can function as a FIB to remove portions of the sample 308 and / or can be used for ion-beam assisted deposition of material onto the sample 308 (e g., in coordination with the GIS 318).

[0042] The charged particle dual beam microscope system 300 may also include one or more retractable devices, such as the GIS 318. The GIS 318 may be oriented toward the sample 308 and configured to direct a gas stream into the vacuum chamber 312. Gas introduced in the vacuum chamber 312 by the gas injection system 318 may be used to etch the sample 308, deposit materials onto the sample 308, enhance imaging contrast or resolution, and / or modify surface properties of the sample 308.

[0043] The operation of one or more components of the example system 300 can be coordinated by control circuitry (e.g., a controller 301), in accordance with machine-executableinstructions (e.g., software, firmware, etc.) that can be stored in machine-readable storage media and / or received from external systems via wired and / or wireless communication techniques (e g., over a WiFi or Bluetooth link). To that end, components of the example system can be automated (e.g., operating without human intervention), pseudo-automated (e.g., operating with limited human intervention to initiate operations, analyze output and confirm, or the like), or manually operated (e.g., where individual operations of the example system 300 are performed by a human user). In an illustrative example, the sample stage 310 can be mechanically coupled with automated stage controls 306 that permit the sample 308 to be reversibly tilted relative to the beam axes A and B, such that the surface of the sample 308 is substantially normal to a given beam axis during operation of the corresponding charged particle beam source.

[0044] The controller 301 may be configured similarly to a computing device 1400 depicted in FIG. 14 and described further below. In at least one embodiment, the controller 301 receives signals from various sensors and detectors of the charged particle dual beam microscope system 300 and transmits instructions to actuators (e.g., actuatable components) of the charged particle dual beam microscope system 300 based on the signals. Moreover, the controller 301 may be communicatively coupled to a digital twins system (e.g., as shown in FIG. 1) such that information, such as settings and / or parameters for operation of the charged particle dual beam microscope system 300, may be transmitted from the digital twins system to the controller 301 to cause the controller 301 to apply the information to operation of the charged particle dual beam microscope system 300. Furthermore, the digital twins system may obtain experimental results from the controller 301 to update at least a sample model of the digital twins system based on the results. In some instances, the digital twins system may control operation of the charged particle dual beam microscope system 300 through the communicative coupling between the controller 301 and the digital twins system.

[0045] Embodiments of the present disclosure can omit one or more components of example system 100. For example, one or more of the sources 302 and 314 and / or columns 304 and 316 can be omitted. In an illustrative example, an SEM system can be configured to perform operations of the beam-induced deposition processes of the present disclosure.

[0046] In at least one embodiment, as described above, a CAD drawing of at least a portion of a physical asset, such as the charged particle dual beam microscope system 300, may be provided to a digital twins system (e.g., input to a computing device at which the digital twinssystem is implemented) to be modeled or simulated by the digital twins system. For example, a physical asset digital twin of the digital twins system may be used to model a behavior and / or operation of the charged particle dual beam microscope system 300, to predict a state of the charged particle dual beam microscope system 300, and / or to control the charged particle dual beam microscope system 300. The CAD drawing may, for example, include the entire charged particle dual beam microscope system 300, or a portion of the charged particle dual beam microscope system 300, and any other columns or devices coupled to the vacuum chamber 312. In other examples, multiple CAD drawings of different portions of the electron microscope system 300 may be used to model operation of the charged particle dual beam microscope system 300 via the digital twins system. Furthermore, in some examples, additional information regarding the physical asset (e.g., instrumental specifications and parameters) may be input to the computing device along with one or more CAD drawings to allow the digital twin to model the physical asset accurately. The digital twin may further obtain information regarding the charged particle dual beam microscope system 300, such as apparatus properties and characteristics, via communicative coupling of the digital twins system to the controller 301 of the charged particle dual beam microscope 300. The control system may acquire data from various databases and other data repositories or archives. The acquired data may be used, for example, to populate one or more lookup tables to be used to generate recommended operation parameters of the physical asset.

[0047] In a digital twins system, a digital twin of a sample may model physical and chemical properties of the sample, and may be used, for example, to predict an effect of a physical asset on the sample (e.g., to predict an effect of charged particle beam of a charged particle microscope on the sample). For example, the sample digital twin may be used in conjunction with a physical asset digital twin, and / or a digital twin that models interaction of a physical asset with a sample, to obtain optimized instrumental settings for operating the physical asset with respect to a requested operation. As an example, the physical asset may be used to perform the operation based on sample information (e.g., sample type, identification, material, etc.) and characteristics of the physical asset, as incorporated into the physical asset digital twin. The sample information may be input by, for example, an operator, or may be retrieved from a database or lookup table based on selection by the operator, although other ways to provide sample information to the digital twins system are possible. The sample digital twin may be initially calibrated, as described further below, and may be used to estimate how the physicalasset interacts with the sample. Predicted results may be used to adjust operating parameters of the physical asset, and actual (e.g., experimental) results of the operations may be used to adjust and refine the sample digital twin. This allows the sample digital twin to reconstruct a sample with greater accuracy.

[0048] A digital twin of a sample may incorporate a diverse range of information and modeling approaches. For example, material properties such as stiffness, stress-strain relationships, conductivity, and resistivity may be included in a digital twin of a sample. In some embodiments, those material properties may be provided, by an operator or user, to a computer running a digital twin, while in other embodiments, those material properties may be measured by a physical asset or generated by a computational model (e.g., a machine learning model, such as a neural network model) based on measured information. For example, in some embodiments, a sample may be prescanned using a technique like energy dispersive spectroscopy (EDS) to identify the chemical elements included in the sample, and that chemical element profile may be provided to a machine learning model that has been trained to infer material properties (such as those listed above) from the chemical element profile for certain classes of material.

[0049] In at least one example, such as for a dual beam microscope system, a digital twins system can be used to provide increasingly accurate predictions of milling results. For example, a physical asset digital twin of the digital twins system may initially obtain relevant operating parameters and values to simulate use of a physical asset to perform an operation. The operation may be simulated to yield an outcome requested by an operator. A sample digital twin of the digital twins system may undergo initial calibration based on sample information provided by the operator and / or obtained using one or more analytical techniques. At least the physical asset digital twin and sample digital twin may then be used in conjunction to predict operating parameters to be used. Upon selection of the predicted operating parameters by the operator, an actual outcome may be compared to the requested outcome and the comparison may be used to at least update the sample digital twin. In some instances, the comparison may be used to update and adjust parameters of one or more of the sample digital twin and the physical asset digital twin.

[0050] In some instances, a closed feedback loop between one or more digital twins and a physical asset may be used to at least update a sample digital twin (e.g., updating of the sample digital twin is prioritized) due to a broader range of diversity and variability among samples incomparison to the physical asset. For example, the physical asset may be associated with relatively stable and readily predictable characteristics because a variability in operation and behavior of the physical asset may be constrained to be within a known range. In contrast, a variety of sample types that can be analyzed by the physical asset may have wide range of physical attributes, physical states, and / or chemical properties, thereby introducing an amount of variability, and unpredictability, that can hinder accurate simulation. Continual and prioritized updating of the sample digital twin may thereby expand and improve a knowledge base of the digital twins system to allow more accurate simulation of a sample, and therefore of an interaction between the physical asset and the sample.

[0051] In at least one embodiment, a sample digital twin may be at least initially calibrated based on sample information specific to a sample, including but not limited to sample composition, crystal structure, and / or geometry / shape. A high-level overview of calibration of a digital twins system (e.g., calibration of a sample digital twin of the digital twins system) is depicted in FIG. 10 and described further below. In at least one embodiment, where a sample is to be analyzed by a charged particle microscope system (e.g., as a physical asset), initial sample information may be obtained prior to calibration and refinement via analytical techniques such as energy dispersive x-ray spectroscopy (EDS) and electron energy loss spectroscopy (EELS). For example, data may be obtained via EDS to generate an elemental map of the sample and data may be obtained via EELS to identify an elemental composition and a chemical state of the sample, an electronic structure of sample materials, a thickness of the sample and / or dielectric properties of the sample. The data may indicate shapes of interfaces of the sample, whether the sample is crystalline or amorphous, and distances between atoms. Initial sample parameter values may be determined based on the initial sample information.

[0052] In at least one embodiment, the sample digital twin may utilize one or more neural networks, such as a large language model (LLM). The LLM may be trained to facilitate communication between the digital twins system and a controller of the charged particle microscope system, thereby at least partially controlling operation of the charged particle microscope system based on inferences output by the LLM.

[0053] To calibrate the sample digital twin (which may be used in conjunction with one or more digital twins that simulate the charged particle microscope system and / or interaction of a charged particle beam with the sample), a set of predicted results are obtained using the digitaltwins system with the initial sample parameter values applied to the sample digital twin. The digital twins system simulates operation of the charged particle microscope system to analyze the sample, which yields the predicted results. A set of corresponding experimental results (e.g., actual results obtained through operation of the charged particle microscope system) are generated by applying the same initial sample parameter values to the system. A comparison of the set of predicted results to the set of experimental results can then be used to calibrate the sample digital twin. For example, the sample parameter values of the sample digital twin may be adjusted based on one or more differences between the set of predicted results and the set of experimental results, thereby allowing corrective modifications to be made to the sample digital twin.

[0054] The digital twins system, including the calibrated sample digital twin, is subsequently used to plan experiments at the charged particle beam microscope. For example, based on the sample parameter values of the calibrated sample digital twin, optimal settings for the charged particle beam microscope can be determined. As examples, the optimal settings may include values or parameters to be used to adjust contrast to yield high quality images, to mitigate distributed beam damage, enhance defect detection capabilities, and / or increase metrology accuracy. The optimal settings may be provided to an operator, e.g., at a user interface or a graphical user interface (GUI) of the charged particle microscope system controller.

[0055] In at least one embodiment, the sample digital twin may be calibrated for a variety of sample types to generate a collection of calibrated sample digital twins that are each representative of a type of sample. In some instances, the calibrated sample digital twins do not replicate physical samples, but rather provide suitable physics-based predictions of parameters to be used according to characteristics of a type of sample (e.g., the calibrated sample digital twin does not need to be an exact replica of a specific sample).

[0056] Moreover, in instances where the sample digital twin utilizes one or more neural networks, such as an LLM, the LLM may be used, in addition to facilitating use of a suitable sample digital twin and optimized instrument settings, to detect operator-initiated deviations from optimized instrument settings provided by the digital twins system. For example, during operation of the charged particle microscope system, the operator may adjust the instrument settings based on one or more of real-time observations and / or technical expertise. The adjustment may be detected by the LLM and used as feedback to update the sample digital twin. In addition, experimental results obtained through subsequent application of theoptimized parameters inferred by the digital twins system, and based on the calibrated sample digital twin, may be used as feedback to update the sample digital twin and further increase its accuracy, as shown in FIGS. 10-11.Examples

[0057] In one embodiment, a digital twins system including one or more digital twins configured to simulate a physical asset, a sample, and / or an interaction between the physical asset and the sample, may be used to provide a prediction based on information input to the digital twins system. For example, when the physical asset acquires one or more images of the sample, the one or more images, as well as sample information, may be input to a computing device at which one or more digital twins are implemented. Generation of simulated results based on an input image is illustrated in FIGS. 4A-4B. Specifically, FIG. 4A shows an image of a sample 400, the image generated by an electron microscope, and FIG. 4B shows a digital twin output 450 corresponding to the sample 400 depicted in the image. The image depicts the sample 400 with square recesses 402 of different depths (e.g., as defined along axis 404) that have been milled using a focused ion beam (FIB).

[0058] In particular, the digital twin output 450 is illustrated as a graph in FIG. 4B that shows the depth of the sample 400 taken along dashed line 406 of FIG. 4A. For example, the depth of the sample 400 is simulated by the digital twin across a first recess 402a, a second recess 402b, and a third recess 402c, along a direction indicated by arrow 408. In the digital output 450 of FIG. 4B, the graph compares experimentally measured heights of the first recess 402a, the second recess 402b, and the third recess 402c that are determined from experimental data 452 (shown as a series of Xs). An x-axis of the graph corresponds to a distance, in pm, across the sample 400 along the direction indicated by arrow 408 of FIG. 4A and a y-axis of the graph corresponds to depth (e.g., height) in pm. The depth of the sample 400 determined from the experimental data 452 is overlaid with synthetic data 454 predicted by the digital twin of the sample (e.g., in solid lines). For example, the graph provides a comparison of experimental versus synthetic depth data of the first recess 402a, the second recess 402b, and the third recess 402c. As depicted in the graph, the synthetic data 454 yields predicted depths of the recesses 402 that are similar to actual depths measured using the experimental data 452.

[0059] A comparison of experimental measurements to predicted measurements may be used in a computational system, such as the computational system 100 of FIG. 1, to continually update digital twins of the system. For instance, such comparisons for a representative suite ofreference samples with known milled depths (or some other known measurement depending on a physical asset type) may be used to initialize an accuracy of the digital twins. As an example, an error associated with the predicted measurements generated by the digital twins relative to the known milled depths may be used to correct and optimize the digital twins. The digital twins may be periodically updated in a similar manner using different reference samples to increase simulation capabilities of the digital twins to evaluate any drift or deviation in accuracy of the digital twins.

[0060] By utilizing and updating digital twins of a computational system as described above, operating parameters of a physical asset may be optimized without relying on sacrificial samples. This allows a target result to be obtained without demanding trial-by-error testing or highly skilled technician expertise to identify suitable parameters. In other words, a closed feedback loop configuration of the computation system, e.g., between a digital twins system and a physical asset, allows automated simulation of physical asset operation to obtain a target outcome. This may be achieved with reduced user input relative to systems without such a closed feedback loop.

[0061] Moreover, the digital twins system may utilize one or more of a physical model and a chemical model that predicts effects of operating parameters based on an atomic structure and / or chemical composition of a sample. The digital twins system may thereby generate simulations with high accuracy because the digital twins system can account for variations in interactions between, for example, a charged particle beam and a sample that occur due to material variations of the sample. As a result, the digital twins system may provide accurate forecasting of what instrument settings (e.g., of a physical asset) are to be used in order to obtain a desired effect of facilitating interaction between the physical asset and a sample. As such, digital twins of a computational system may be used to both predict optimal instrument settings to obtain a target outcome (e.g., a milling depth and / or geometry sample) and to predict an outcome based on specific instrument settings.

[0062] The systems and methods disclosed herein may be used in any of a number of different applications, including autonomous FIB milling. In autonomous FIB milling, a human user may input a specific desired result (e.g., a shape into which a sample is to be machined), and a computational system (e.g., the computational system 100 of FIG. 1) may use a digital twins system that simulates a sample, FIB column, and interactions between the sample and a beam of the FIB column, to output a set of FIB parameters (e.g., beam energy, beam shape, etc.) anda patterning strategy (e.g., end milling, vertical milling, etc.) to achieve the desired result. In some embodiments, as described above, modeling of the interactions between a beam and a sample digital twin may include a physical model (e.g., that can predict how the sample will physically respond to being hit with an electron beam of a given energy) and a chemical model (e.g., that predicts chemical interactions between a sample and a beam based on atom-level mechanics). An output of the computational system may depend on the parameters of the underlying digital twins (e g., chemical composition and materials properties of the sample).

[0063] In at least one embodiment, various stages in an exemplary digital twins output of a simulated milling plan for an autonomous FIB milling system is depicted in FIGS. 5A-5D. The milling plan is used to guide removal of material from an edge of a block sample using a FIB beam. The milling plan may be input to a digital twin, of a digital twins system, that simulates the FIB milling system, as well as a digital twin that simulates a sample 500, where the milling plan specifies instrumental settings to be used to mill the sample 500. One or more of a physical model and a chemical model may be included in the digital twins to allow the digital twins to output a predicted result of operating the FIB milling system according to the milling plan. The digital twins system may further include a digital twin to simulate interaction of the FIB beam on the sample, using information regarding the FIB milling system and the sample that may be obtained from the other digital twins.

[0064] The sample 500, as shown in FIG. 5A is initially a pristine block of material (e.g., untouched by the FIB milling system). An early stage of simulated milling is shown in FIG. 5B where a milled region 502 is formed in the sample 500 due to interaction of a FIB of the FIB milling system with the sample 500, which causes a portion of the sample 500 to be removed along an edge of the sample 500. As shown in FIGS. 5B and 5C, as simulated milling proceeds, the FIB continues to remove a material of the sample 500, increasing a size of the milled region 502. A final, target removal of the material of the sample 500 is achieved in FIG. 5D.

[0065] A final geometry (e g., depth, width, shape, etc.) of the milled region 502, as shown in FIG. 5D, may be predicted based on a physical model and a chemical model implemented by the digital twins. For example, a different sample may be formed of a material that is more or less resistant to removal by milling or may be of a chemical composition that interacts differently with the FIB. As such, different sample materials may, for a given duration of milling exemplified in FIGS. 5A-5D, remove more or less material and form a milled region with a different shape than shown therein. The digital twins system may thereby simulate FIBmilling of a sample based on a physical model and / or chemical model in an automated and precise manner according to a feedback loop between a physical asset and the digital twins system, e.g., as depicted in FIG. 1.

[0066] As described above, one or more digital twins may be configured to predict optimized parameters to be applied to a physical asset to perform an operation. As one example, the physical asset may be a dual beam FIB-SEM system and a digital twins system may be used as a milling simulator to obtain predicted FIB settings for a milling pattern or recipe to achieve a desired result. By implementing the milling simulator, a more accurate estimate for a milling depth resulting from application of a milling pattern (e.g., milling recipe) obviates trial-by- error determination of optimal milling parameters. For example, without a milling simulator implemented by a digital twins system as described herein, an operator may sacrifice numerous samples that are used as testing samples in order to determine a suitable milling pattern. This may arise from using a patterning engine (e.g., software or code to set a milling pattern) that is provided at a controller of the FIB-SEM system, where the patterning engine utilizes a simplistic model to estimate milling parameters (e.g., mill depth, beam spillover, cutface angle, etc.). However, the simplistic model may not be able to account for parameter variations and may rely on idealized modeling of the FIB beam. In contrast, the milling simulator uses information received from the FIB-SEM system to compute a rapid simulation of surface evolution, thereby allowing the milling simulator to return more accurate patterning predictions.

[0067] For example, as shown in FIG. 6, a digital twins system 600 includes a milling simulator that is communicatively coupled to a controller 602, where the milling simulator 604 is implemented by one or more digital twins. The one or more digital twins at least includes a digital twin that simulates interaction of a FIB beam of a FIB-SEM system (e.g., a physical asset) with a sample. The milling simulator 604 may further include a digital twin that simulates operation of the FIB-SEM system and / or a digital twin that simulates behavior of the sample. The controller 602 includes one or more processors and other circuitry to control operation of the FIB-SEM system. Furthermore, the controller 602 is communicatively coupled to a variety of data sources 606, such as databases, data repositories, and storage locations associated with various data collection software.

[0068] The digital twins system 600 utilizes a two-way exchange of information between the controller 602 and the milling simulator 604 to provide recommendations to an operator basedon a request input to the digital twins system 600 by the operator. For example, the operator may request an estimated depth of milling obtained based on a selected milling pattern and a selected amount of time, or may request an estimate of parameters for a milling pattern according to a desired milling depth and / or duration of milling.

[0069] To generate a result based on the request, the milling simulator 604 acquires relevant information from the controller 602, as indicated by arrow 608, where the controller 602 may collect at least a portion of the relevant information from the data sources 606. For example, the milling simulator 604 may obtain, from the controller 602, beam properties such as ion species, ion energy, beam current, and / or beam diameter, pattern properties according to x, y, and dwell values, such as scan points, scan strategy, passcount, defocus, etc., and target properties such as stage tilt, material and / or applications files.

[0070] The milling simulator 604 uses the information obtained from the controller 602 to estimate one or more values, depending on the input request. For example, the milling simulator 604 may be used to obtain an estimation of mill depth, passcount, dose, and / or time. In at least one embodiment, the one or more values are estimated based on a last or most recently entered value from an operator. For instances where cross-sectional sample (CCS) or rotary cross-section (RCS) milling is to be performed, the milling simulator 604 may further return a pattern spatial offset relative to a desired edge to be milled and / or a predicted cutface angle.

[0071] The one or more values may be predicted based on simulations carried out by the milling simulator 604. In one example, the milling simulator defines a simulation volume (e.g., a defined three-dimensional space to conduct a simulation) and generates simulation objects (e.g., elements that interact within the simulation volume) at a first simulation step 612 illustrated in FIG. 6. At a second simulation step 614, a fast surface evolution simulation (e.g., a computational modeling of change in shapes and surfaces over time) is performed on a digital reconstruction of a sample, as facilitated by a digital twin, until a stop condition is reached.

[0072] The predicted one or more values output by the milling simulator 604 are provided to the controller 602, as indicated by arrow 610. The controller 602 may perform one or more operations with the FIB-SEM system, based on the operator request, using the predicted one or more values to acquire experimental results. The experimental results may be provided to the milling simulator 604 from the controller 602 to allow the milling simulator 604 to compare the experimental results to the simulations. For example, a difference between the predicted one ormore values and corresponding values obtained from the experimental results may be computed and used to update the one or more digital twins implementing the milling simulator 604. In particular, the experimental results may be used to update a digital twin configured to simulate a sample, thereby increasing an accuracy of the digital twin and / or expanding a knowledge base from which the digital twin obtains relevant information.

[0073] In another example, implementation of a digital twins system that utilizes one or more digital twins to simulate one or more of a physical asset, a sample, and / or interaction between the physical asset and the sample, may be achieved using artificial intelligence (Al). For instance, machine learning algorithms defining operations of one or more neural networks may be incorporated into an Al-based workflow that relies on the digital twins system to achieve fast and accurate predictions. The use of a digital twins system to obtain Al-based predictions may be applied to any of the use-case scenarios described herein.

[0074] In at least one embodiment, machine learning algorithms can be leveraged to provide a fully automated workflow that reduces a threshold minimum amount of expertise required to operate the physical asset efficiently and effectively, while being able adapt to variations that affect actual operating conditions. Sub-optimal conditions and parameters, as well as deviations from an expected course, can be detected in a more reliable and robust manner, compared to relying merely on observation by an operator, thereby alleviating a burden on the operator to determine when experimental conditions are not optimal. The automated workflow includes obtaining predictions of optimized settings and conditions for the physical asset and determining adjustments to be made to operation of the physical asset, in real-time, during execution of the workflow.

[0075] An efficacy and robustness of an Al-based workflow, as well as automation facilitated by the workflow, however, may be dependent on how well an Al model (e.g., neural network) can be trained. For example, an ability of the Al model to accurately predict optimal instrumental settings and conditions for desired outcome may be constrained by the dataset used to train the Al model. In some instances, such as when a wide variety of types of samples, and experimental conditions, may be analyzed and processed by a physical asset, the Al model may be not able to generate accurate predictions for a broad range of samples and scenarios because obtaining sufficiently large training datasets may be challenging. In at least one embodiment, by leveraging a digital twins system, a large and diverse amount of training datamay be readily obtained, thereby allowing an Al model to generate accurate predictions in realtime despite highly variable experimental contexts.

[0076] As an example, an Al-based workflow 700 is depicted in FIG. 7 with respect to operation of a FIB system (which may, for example, be included in a FIB-SEM system), although the Al-based workflow 700 may be applied in a similar manner to other apparatuses and instrumental systems. As illustrated in FIG. 7, the Al-based workflow 700 includes a training and validation component 702 that is communicatively linked to an execution component 704. The training and validation component 702 is used to produce one or more neural networks (e.g., neural network models) that have been trained and validated according to simulated experiments. In at least one embodiment, the one or more neural networks are trained by a digital twins system. In another embodiment, the one or more neural network may be trained, by a digital twins system or using other algorithms and software, to operate as a digital twins system that simulates the FIB system to guide operation of the FIB system in real-time.

[0077] For example, a library or collection of simulated mill processes may be used to train a first neural network (e.g., NN1) to predict outcomes of FIB milling, where the simulated mill processes are generated by one or more digital twins of a digital twins system that are configured to reconstruct one or more of the FIB system, a sample, and interaction between the FIB beam and the sample. In at least one embodiment, a sample digital twin may be calibrated as described above and shown in FIG. 10. The one or more digital twins may be deployed also using Al-based algorithms (e.g., machine learning) or may be implemented using other types of non-AI models. Accuracies of the one or more digital twins may be verified based on comparison to experimental results (e.g., as described above with references to FIGS. 4B and 6 and the verified digital twins may be deployed to simulate FIB milling processes based on a set of experimental and sample information. As such, as large and varied training dataset can be generated to efficiently train the first neural network.

[0078] The first neural network may be trained to simulate milling processes and milling outcomes based on inputs 706 provided thereto, such as different beam profiles, patterning, scanning strategies, and / or sample types. Upon receiving the inputs 706, the first neural network may construct a simulation environment that mimics a sample environment corresponding to the FIB-SEM system predict outcomes 708 using the input 706. Furthermore, the first neural network may learn associations, e.g., what sample information, instrumentsettings and conditions, etc., between the inputs 706 and milling outcomes 708 based on the training dataset generated by the digital twins system.

[0079] For example, the outcomes 708 may include a time-evolution of a mill surface and generation of ancillary signals such as, but not limited to, stage current measurements and / or secondary charged particles. During training, the first neural network may determine correlations between stage current and / or detection of secondary charged particles to mill depth and / or surface topography using the simulations provided by the digital twins system. In at least one embodiment, the training may be performed offline, e.g., not during operation of the FIB system. Furthermore, the training may be conducted at a computing device that is different from one or more computing devices used to control the FIB system and / or host the Al-based workflow 700.

[0080] In at least one embodiment, by performing the training offline, a workflow for operating the FIB-SEM system may be streamlined by obviating determination of optimal settings and milling parameters by trial-and-error. For example, in instances where the AI- based workflow 700 is not used, the workflow may include performing the simulations, and / or other time-consuming processes, as an initial step to identify a suitable milling pattern to be used, thereby prolonging the workflow and decreasing throughput.

[0081] The outcomes 708 inferred by the first neural network based on the simulated mill processes may be validated using selected simulations (e.g., selected from simulations generated by the digital twins system). For example, simulations that represent different experimental scenarios may be selected for a validation dataset, where the validation dataset is different from the training dataset (e.g., simulations that were not used for training are used for validation). Once validated, the trained first neural network may be used to generate mill predictions 710 that are provided, as indicated by arrow 712, to the execution component 702 of the Al-based workflow 700.

[0082] The execution component 702 is deployed at the computing device hosting the AI- based workflow 700. At the computing device, the execution component 702 receives operator input 714, such as known sample information and a desired outcome. For example, the known sample information may include a composition and distribution of structural and / or compositional constituents (e.g., homogeneous or heterogeneous), and, if heterogeneous, a spatial arrangement of the structural and / or compositional constituents. In at least one example, the computing device may include a GUI that allows the operator to create a virtual three-dimensional (3D) model of the sample, or to create or input one or more voice commands corresponding to the known sample information, a pre-existing virtual 3D model (such as a CAD model), or a textual description of the sample. The desired outcome may include a target value to be obtained via the Al-based workflow 700, such as a target mill depth. The desired outcome may further include specifications such as a milling site at the sample, a target milling shape, spatial dimensions of a feature milled at the milling site, temporal constraints, and / or unallowable tolerances for the outcome specifications.

[0083] The trained first neural network receives the operator input 714 and uses the operator input 714 to output one or more inferences 716. The trained first neural network also uses system state information 718 to generate the one or more inferences 716. For example, the AI- based workflow 700 may include software algorithms to cause the system state information to be provided to the trained first neural network. The system state information includes one or more of a spatial arrangement and location of the FIB source, voltages applied to focusing lenses and other beam-shaping devices of the FIB system, and physical locations of any beammodulating apertures of the FIB system.

[0084] The one or more inferences 716 generated by the trained first neural network, based on the operator input 714 and the system state information 718, is provided to the FIB system controller to cause the FIB system controller to apply the one or more inferences 716 to a milling operation to be performed by the FIB system. For example, the one or more inferences 716 include predicted initial settings and parameters for the milling operation in order to obtain the desired outcome. The milling operation is carried out using the one or more inferences and the trained first neural network performs real-time observation 720 over the duration of the milling operation. The real-time observation 720 includes monitoring of a progress of the milling operation and comparing the progress to corresponding predicted results. For example, the trained first neural network may periodically determine milling depth over set intervals of time over the duration of the milling operation, where the milling depth may be measured based on signals generated during milling and / or periodic acquisition of images of the milling site.

[0085] In instances where the milling progress is monitored based on images acquired during milling, e.g., electron images and / or FIB images, a trained second neural network (e.g., NN2 in FIG. 1) may be used to perform image recognition 722. For example, the trained second neural network may be trained and optimized to identify image features corresponding to milling of a sample, such as surface topology during milling and formation of a milling feature. The trainedsecond neural network may be used to evaluate the evolving milled surface of the sample and confirm real-time signal feedback based on in-process signals.

[0086] The real-time observation 720, as performed by the trained first neural network, further includes inferring modification of the milling (e.g., by adjusting milling parameters, FIB system settings, etc.) based on the real-time signal feedback. For example, the trained first neural network may identify deviations from expected progression of the milling, e.g., deviations that exceed a predetermined amount from an expected value, and infer one or more adjustments to be made to the milling process to correct for the deviations. As such, the AI- based workflow 700 may include iterating between generating one or more inferences 716 and performing real-time observation 720 of the milling process until the milling process is complete. Upon completion, a final milling feature 724 is created.

[0087] In at least one embodiment, the trained first neural network may be implemented as a digital twins system for the FIB system (or FIB-SEM system) and may simulate one or more of a sample, the FIB system, and / or interaction of the FIB beam with the sample. For instance, while referred to in the singular, the trained first neural network may include more than one neural network model, e.g., a network of models, such that individual models may be trained to perform different tasks (such as sample simulation, FIB system simulation, and simulation of beam-sample interactions). Alternatively, different layers of the first neural network may be configured to simulate a sample, a physical asset, and interaction between the sample and the physical asset. The first neural network may be trained using simulations generated by a digital twins system, as described above, or via simulations generated using some other technique or software. The trained first neural network is then deployed itself as a digital twins system during operation of the FIB system.

[0088] For example, the trained first neural network may simulate a milling process based on the operator input 714 and the system state information 718 and determine optimal instrument settings to be used to yield the requested outcome. The trained first neural network, as described above, may monitor the progress of the milling experiment and compare the experimental progress to the simulated milling process to predict and provide real-time adjustments to the instrument settings. When a stop condition is reached, e.g., a requested duration of time, or a requested mill depth is obtained, deviations from the expected (e.g., predicted) instrument settings are determined. For example, if the operator requests a milling process to proceed until a target depth is reached, a deviation from a predicted amount of time,or from a predicted milling site shape when the target depth is obtained may be used to update the digital twins system. As an example, weights of the first neural network corresponding to simulation of the sample may be updated based on the deviation. In this way, a digital twins system may be trained and optimized according to a physical asset (e.g., based on training data generate by another digital twins system), and may be continuously updated upon deployment. Exemplary systems and operating methods for implementation

[0089] FIG. 8 is a block diagram of a scientific instrument support module 800 for performing support operations, in accordance with various embodiments. The scientific instrument support module 800 may be implemented by circuitry (e.g., including electrical and / or optical components), such as a programmed computing device (e.g., any of the computing device 114 of FIG. 1 or the computing device 1400 shown in FIG. 14). Logic of the scientific instrument support module 800 may be included in a single computing device, or may be distributed across multiple computing devices that are in communication with each other as appropriate.Examples of computing devices that may, singly or in combination, implement the scientific instrument support module 800 are discussed herein with reference to the computing device 1400 of FIG. 14, and examples of systems of interconnected computing devices, in which the scientific instrument support module 800 may be implemented across one or more of the computing devices, is discussed herein with reference to a scientific instrument support system 1500 of FIG. 15. Moreover, the scientific instrument support module 800 may include the component models 200 of FIG. 2, where the component models may be distributed across individual logic elements of the scientific instrument support module 800.

[0090] The scientific instrument support module 800 may include first logic 802, second logic 804, and third logic 806. As used herein, the term “logic” may include an apparatus that is to perform a set of operations associated with the logic. For example, any of the logic elements included in the support module 800 may be implemented by one or more computing devices programmed with instructions to cause one or more processing devices of the computing devices to perform the associated set of operations. In a particular embodiment, a logic element may include one or more non-transitory computer-readable media having instructions thereon that, when executed by one or more processing devices of one or more computing devices, cause the one or more computing devices to perform the associated set of operations. As used herein, the term “module” may refer to a collection of one or more logic elements that, together, perform a function associated with the module. Different ones of the logic elementsin a module may take the same form or may take different forms. For example, some logic in a module may be implemented by a programmed general-purpose processing device, while other logic in a module may be implemented by an application-specific integrated circuit (ASIC). In another example, different ones of the logic elements in a module may be associated with different sets of instructions executed by one or more processing devices. A module may not include all of the logic elements depicted in the associated drawing; for example, a module may include a subset of the logic elements depicted in the associated drawing when that module is to perform a subset of the operations discussed herein with reference to that module. Furthermore, in at least one embodiment, algorithms (e.g., code) applied at a module may include executable instructions to perform Al-based (e.g., one or more neural networks) processes and operations. For example, a digital twin of a digital twin system implemented at the scientific instrument support module 800 may include one or more neural networks.

[0091] The first logic 802 may provide a computational simulation of a scientific instrument (e.g., a physical asset), in accordance with any of the embodiments disclosed herein. For example, the first logic 802 may be utilized in a digital twin that models (e.g., generates predictions) instrument behavior, based on instrument specifications. As an example, a computing device hosting both the scientific instrument support module 800 and a control system of the scientific instrument, or a computing device implementing a control system of the scientific instrument and communicatively coupled to a computing device hosting the scientific instrument support module 800, may provide the instrument specifications to the digital twin. The first logic 802 may use the instrument specifications to facilitate simulation of instrument operation according to, for example, a given set of operating parameters, such as a mill plan.

[0092] The second logic 804 may provide a computational simulation of a sample, in accordance with any of the embodiments disclosed herein. For example, the second logic 804 may be utilized in a digital twin that models physical and chemical properties of the sample. In at least one embodiment, the second logic 804 may utilize a physical model and a chemical model corresponding to the sample, and provided thereto by a user, to allow the digital model to simulate behavior of the sample under various conditions, such as an analytical environment.

[0093] The third logic 806 may provide a computational simulation of the interactions between the scientific instrument and the sample, in accordance with any of the embodiments disclosed herein. For example, the third logic 806 may be utilized in a digital twin that models an outcome of applying the scientific instrument to the sample. The third logic 806 maygenerate predictions in conjunction with the first logic 802 and the second logic 804, e.g., by leveraging simulations of scientific instrument behavior and simulations of sample properties. This allows the digital twin to accurately estimate an effect of operating the scientific instrument behavior to modify the sample.

[0094] FIG. 9 is a high-level flow diagram of a method 900 of performing support operations for a scientific instrument (e.g., a physical asset), in accordance with various embodiments. The method 900 is performed by a computing device, e.g., one or more processors, hosting a scientific instrument support module, such as the scientific instrument support module 800 of FIG. 8. The computing device is included in a computational system (e.g., the computational system 100 of FIG. 1) that includes a physical asset communicatively coupled, via a closed feedback loop, to a digital twins system such that information stored at and used by a control system of the physical asset is accessible to the digital twins system. Furthermore, the computing device includes memory at which executable instructions to perform the method 900 are stored. The computing device may also implement a control system of the physical asset (e.g., a scientific instrument) or may be communicatively coupled to another computing device that implements the control system. Although the operations of the method 900, as well as methods depicted in FIGS. 10-12 and described below, may be illustrated with reference to particular embodiments disclosed herein (e.g., the scientific instrument support module 800 discussed herein with reference to FIG. 8, a GUI 1300 discussed herein with reference to FIG. 13, computing devices 1400 discussed herein with reference to FIG. 14, and / or the scientific instrument support system 1500 discussed herein with reference to FIG. 15), the method 900 (and the methods of FIGS. 10-12) may be used in any suitable setting to perform any suitable support operations. Operations are illustrated once each and in a particular order in FIG. 9, but the operations may be reordered and / or repeated as desired and appropriate (e g., different operations performed may be performed in parallel, as suitable). Furthermore, in at least one embodiment, at least a portion of the method 900 may leverage artificial intelligence (e.g., one or more neural networks) to infer evolutions of processes and operations and / or predict outcomes of the processes and operations.

[0095] At 902, the method 900 includes receiving instrument and sample information. For example, a user may provide, e.g., at a user interface of the computing device, operating parameters and / or instrumental specifications of the scientific instrument to be used in a simulation of instrument behavior and performance facilitated by one or more digital twins.The user may further provide information regarding a material type, state, and any previous processing of a sample that is to be simulated by one or more digital twins.

[0096] At 904, the method 900 includes receiving a request for a result. For example, a user may input, to the digital twins system, a desired simulation outcome. As one example, the desired simulation outcome may be a set of operating parameters of the scientific instrument that are predicted to yield a targeted effect of interaction between the scientific instrument and the sample. For instance, the desired simulation outcome may be operating parameters for a FIB system to mill a recess of a specific shape and dimensions in a sample. As another example, the desired simulation outcome may be an effect of interaction between the scientific instrument and the sample, on the sample. For instance, the desired simulation outcome may be an image and corresponding metrology determining how much material is removed from a sample by a FIB using a given set of operating parameters for a particular type of sample.

[0097] At 906, the method 900 includes outputting a simulated outcome based on simulations performed by the digital twins system and according to the received request for the result. For example, as described above, the result may include operating parameters for the scientific instrument, data regarding an outcome of interaction between the scientific instrument and the sample, and / or other simulated results associated with the scientific instrument and / or the sample. In at least one embodiment, the result may be displayed to a user at a user interface of the computing device, relayed to another computing device, stored in memory or a database, etc.

[0098] FIG. 10 is a flow diagram of a method 1000 of performing support operations to calibrate a digital twins system that includes one or more digital twins, and to use the digital twins system to optimize operation of a physical asset, in accordance with various embodiments. The method 1000 may be performed by a computing device having one or more processors, where the computing device may be included in a computational system (e.g., the computational system 100 of FIG. 1) having a physical asset communicatively coupled to the digital twins system. The digital twins system may be configured to simulate one or more of the physical asset, a sample, and an interaction between the physical asset and the sample. The computing device includes memory at which executable instructions to perform the method 1000 are stored and also implements a control system of the physical asset (e.g., a scientific instrument). Operations are illustrated once each and in a particular order in FIG. 10, but the operations may be reordered and / or repeated as desired and appropriate (e.g., differentoperations performed may be performed in parallel, as suitable). Furthermore, in at least one embodiment, at least a portion of the method 1000 may leverage artificial intelligence (e.g., one or more neural networks) to infer evolution of processes and operations and / or predict outcomes of the processes and operations.

[0099] At 1002, the method 1000 includes acquiring one or more initial datasets. In at least one embodiment, the one or more initial datasets may include a parameterized digital twin modeling a specific type of sample (e.g., based on composition, crystal structure, geometric shape) and / or any sample information input by an operator, as described above with reference to the operator input 714 of FIG. 7. The one or more initial datasets may further include information regarding settings, characteristics, and / or a current state of the physical asset. For example, system state information, such as the system state information 718 of FIG. 7, may be obtained from a controller of the physical asset. In addition, the one or more initial datasets may include data obtained from various databases, repositories, and other storage locations storing experimental and simulated processes and results. Moreover, the one or more initial datasets may also include a set of experiments and measurements used to determine initial parameters of the digital twins system (e.g., initial values for a sample digital twin configured to simulate a sample). For example, as described previously, data collected using EDS and / or EELS may be used to obtain sample information from which suitable initial sample parameter values may be identified.

[0100] At 1004, the method 1000 includes performing a set of experiments to obtain a corresponding set of experiment-based measurements and results that are used to calibrate the digital twins system. For example, the set of experiments may be performed using the physical asset and a sample to be modeled to obtain the set of experiment-based measurements and results, or, if data according to the set of experiments is already available, the set of experiment-based measurements and results may be retrieved from storage. Corresponding predicted measurements and / or predicted results are obtained by the digital twins system and compared to the set of experiment-based measurements and results. The comparison may be used to correct a deviation of the digital twins system from an expected result determined based on the set of experiment-based measurements and results. For instance, the initial values used for the sample digital twin may be adjusted according to the comparison. As an example, when one or more neural networks are used to implement the sample digital twin, a loss may be computed based on the comparison (e.g., using a loss function) and the loss used to updateweights of the one or more neural networks to achieve a minimum threshold level of accuracy. In at least one embodiment, further calibration of the digital twins system may be performed until the minimum threshold level of accuracy is achieved.

[0101] At 1006, the method 1000 includes optimizing settings of the physical asset using the calibrated digital twins system. For example, when the physical asset is an FIB-SEM system, an operator may indicate a desired outcome and / or a desired operation to be performed. As an example, the operator may enter information regarding the desired outcome and / or operation at a GUI of the FIB-SEM controller. The calibrated digital twins system may receive the information and use the information to output optimized parameters, such as settings to control contrast, distributed beam damage, defect detection, and / or metrology accuracy.

[0102] At 1008, the method 1000 includes applying the optimized settings to the physical asset upon indication that the physical asset is to perform the desired operation. For example, applying the optimized settings may include presenting to the operator, e.g., at the GUI, the optimized settings determined by the digital twins system. If confirmation is received from the operator to proceed with the operation, the optimized settings are used at the physical asset to perform the operation. The application of optimized instrumental settings may also be carried out based on information determined at 1106 of FIG. 11, described further below.

[0103] At 1010, the method 1000 includes acquiring experimental data from the physical asset, based on the application of the optimized instrumental settings. For example, the operation may be executed by the physical asset. Measurements, including metrology data, may be obtained as results of the operation and stored at a storage location (e.g., in memory, at a database, etc.). In addition, detection of any operator-initiated deviations from the optimized instrumental settings may be recorded. For instance, when the physical asset is an electron microscope, the operator may adjust the contrast setting while the operation is in progress, based on visual observation. This adjustment may be recorded and used to update the digital twins system, as described in FIG. 11.

[0104] FIG. 11 is a flow diagram of a method 1100 of performing update and / or training operations for a digital twins system, in accordance with various embodiments. The method 1100 is performed by a computing device, e.g., having one or more processors, hosting a scientific instrument support module, such as the scientific instrument support module 800 of FIG. 8. The computing device is included in a computational system (e.g., the computational system 100 of FIG. 1) that includes a physical asset communicatively coupled, via a closedfeedback loop, to one or more digital twins of the digital twins system such that information stored at and used by a control system of the physical asset is accessible to the one or more digital twins. Furthermore, the computing device includes memory at which executable instructions to perform the method 1100 are stored. The computing device may also be a controller of the physical asset (e.g., a scientific instrument) or may be communicatively coupled to another computing device controls the physical asset. Operations are illustrated once each and in a particular order in FIG. 1100, but the operations may be reordered and / or repeated as desired and appropriate (e.g., different operations performed may be performed in parallel, as suitable). Furthermore, in at least one embodiment, at least a portion of the method 1100 may leverage artificial intelligence (e.g., one or more neural networks) to infer evolutions of processes and operations and / or predict outcomes of the processes and operations.

[0105] At 1102, the method 1100 includes comparing experimental data obtained by a physical asset to simulated data generated by the digital twins system. For example, results acquired by the physical asset using optimized instrumental settings, as described above in conjunction with FIG. 10, may be compared to simulated results output by the digital twins system based on the optimized instrument settings.

[0106] At 1104, the method 1100 includes determining a difference between the experimental data and the simulated results. In one example, such as when the digital twins system is implemented using one or more neural networks, the difference is used to compute a loss via a loss function.

[0107] At 1106, the method 1100 includes updating the digital twins system based on the difference. Updating the digital twins system may include modifying initial or most recently used model parameters of the digital twins system. For example, as shown in FIG. 4B, milling of a sample by a FIB may be simulated and depths of one or more simulated recesses milled in the sample may be measured. The simulated depth measurements may be compared to known recess depths of a reference sample prepared using the same instrumental setting and type of sample as in the simulation. A difference or amount of error may be determined using the comparison. In one embodiment, at least a sample digital twin (e.g., a digital twin modeling behavior and characteristics of a type of sample) of the digital twin system may be updated to increase an accuracy of the sample digital twin with respect to how the sample digital twin is able to simulate an effect of interaction with physical asset. The sample digital twin may be updated by altering values used by the sample digital twin that correspond to sampleparameters, such as composition, crystal structure, and geometry. Furthermore, when the sample digital twin includes one or more neural networks, updating the sample digital twin includes computing gradients based on the loss and adjusting weights of the neural networks based on the gradients. The method 1100 may then return to 1008 of FIG. 10 to apply updated settings of the digital twins system to the physical asset.

[0108] FIG. 12 is a flow diagram of a method 1200 for training an Al-based model, e.g., utilizing one or more neural networks, that may be implemented in a digital twins system to simulate one or more of a physical asset, a sample, and interaction of the physical asset with the sample. The method 1200 is performed by a computing device, e.g., having one or more processors, which may be different or the same as a computing device hosting a scientific instrument support module (e.g., the scientific instrument support module 800 of FIG. 8). The computing device includes memory at which executable instructions to perform the method 1200 are stored. Operations are illustrated once each and in a particular order in FIG. 1200, but the operations may be reordered and / or repeated as desired and appropriate (e.g., different operations performed may be performed in parallel, as suitable).

[0109] At 1202, the method 1200 includes obtaining one or more datasets relevant to training the one or more neural networks. For example, the one or more datasets may be retrieved from one or more data sources. In some instances, the one or more datasets may be preprocessed to be in a suitable format for training the one or more neural networks. The one or more datasets may further be split into subsets to be used for specific operations. As an example, the one or more datasets may be split at least into a training subset for fitting the Al-based model and a validation subset for tuning or refining the model.

[0110] At 1204, the method 1200 includes initializing the Al-based model. In at least one embodiment, initializing the model includes defining architectures of the one or more neural networks. For example, a number of layers, types of layers, and number of neurons per layer may be specified. Initializing the model may further include determining initial weights and biases of the one or more neural networks.[0U1] At 1206, the method 1200 includes performing forward propagation. For example, input data from the training subset may be passed through the one or more neural networks to generate predictions, which may include sequentially computing an output of each layer of a neural network, an input layer to an output layer of the neural network.

[0112] At 1208, the method 1200 includes computing a loss. The loss may be a difference between the predicted outputs of the one or more neural networks and expected outcomes, e.g., ground truth. For example, the expected outcomes may correspond to labeled data included in the training subset. The loss may be computed using any one or a variety of loss functions, such as mean squared error (MSE), cross-entropy, or some other type of loss function.

[0113] At 1210, the method 1200 includes performing backward propagation to determine a propagation of error backward through the one or more neural networks. Gradients are computed with respect to a neural network’s weights and biases.

[0114] At 1212, the method 1200 includes updating the weights and biases of the one or more neural networks. The weights and biases are updated based on the computed gradients using an optimization algorithm, such as stochastic gradient descent (SGD) or adaptive moment estimation (Adam), among others.

[0115] At 1214, the method 1200 includes validating the Al-based model using the validation subset. Validation of the Al-based model allows a performance of the model to be monitored during training. For example, the validation results may be used to evaluate how well the model generalizes data that was not used to train the model, thereby mitigating overfitting.

[0116] At 1216, the method 1200 includes confirming if convergence is reached, based on the performance of the Al -based model. In at least one embodiment, convergence is determined based on the loss falling below a predetermined threshold value, or the performance of the model on the validated subset reaching a minimum threshold level. If convergence is not reached, the method 1200 includes returning to 1206 to perform forward propagation for further training. If convergence is reached, the method 1200 includes proceeding to 1218 to use the Al-based model. For example, the trained model may be applied to testing data to evaluate a final performance of the model on unseen data (e.g., new data). Additionally or alternatively, the trained model may be deployed at a computing device hosting a digital twins system.

[0117] The scientific instrument support methods disclosed herein may include interactions with a human user (e.g., via a user local computing device 1520 discussed herein with reference to FIG. 15). These interactions may include providing information to the user (e.g., information regarding the operation of a scientific instrument such as a scientific instrument 1510 of FIG. 15, information regarding a sample being analyzed or other test or measurement performed by a scientific instrument, information retrieved from a local or remote database, or other information) or providing an option for a user to input commands (e.g., to control theoperation of a scientific instrument such as the scientific instrument 1510 of FIG. 15, or to control the analysis of data generated by a scientific instrument), queries (e.g., to a local or remote database), or other information. In some embodiments, these interactions may be performed through a graphical user interface (GUI) that includes a visual display on a display device (e.g., a display device 1410 discussed herein with reference to FIG. 14) that provides outputs to the user and / or prompts the user to provide inputs (e.g., via one or more input devices, such as a keyboard, mouse, trackpad, or touchscreen, included in other I / O devices 1412 discussed herein with reference to FIG. 14). The scientific instrument support systems disclosed herein may include any suitable GUIs for interaction with a user.

[0118] FIG. 13 depicts an example GUI 1300 that may be used in the performance of some or all of the support methods disclosed herein, in accordance with various embodiments. As noted above, the GUI 1300 may be provided on a display device (e.g., the display device 1410 discussed herein with reference to FIG. 14) of a computing device (e g., the computing device 1400 discussed herein with reference to FIG. 14) of a scientific instrument support system (e.g., the scientific instrument support system 1500 discussed herein with reference to FIG. 15), and a user may interact with the GUI 1300 using any suitable input device (e.g., any of the input devices included in the other I / O devices 1412 discussed herein with reference to FIG. 14) and input technique (e.g., movement of a cursor, motion capture, facial recognition, gesture detection, voice recognition, actuation of buttons, etc.).

[0119] The GUI 1300 may include a data display region 1302, a data analysis region 1304, a scientific instrument control region 1306, and a settings region 1308. The particular number and arrangement of regions depicted in FIG. 13 is simply illustrative, and any number and arrangement of regions, including any desired features, may be included in a GUI.

[0120] The data display region 1302 may display data generated by a physical asset such as a scientific instrument (e.g., the scientific instrument 1510 discussed herein with reference to FIG. 15). For example, the data display region 1302 may display measurements of the physical asset in a digital twins system as compared to simulations of the same values (e.g., as shown in FIG. 4B).

[0121] The data analysis region 1304 may display the results of data analysis (e.g., the results of analyzing the data illustrated in the data display region 1302 and / or other data). For example, the data analysis region 1304 may display a recommended set of FIB parameters and patterning strategy for autonomous FIB milling, as discussed above. In some embodiments, thedata display region 1302 and the data analysis region 1304 may be combined in the GUI 1300 (e.g., to include data output from a scientific instrument, and some analysis of the data, in a common graph or region).

[0122] The scientific instrument control region 1306 may include options that allow the user to control a scientific instrument (e g., the scientific instrument 1510 discussed herein with reference to FIG. 15). For example, the scientific instrument control region 1306 may include current operational parameters of a scientific instrument (which may be automatically set by feedback between the scientific instrument and the digital twins system, or manually set by a human technician).

[0123] The settings region 1308 may include options that allow the user to control the features and functions of the GUI 1300 (and / or other GUIs) and / or perform common computing operations with respect to the data display region 1302 and data analysis region 1304 (e.g., saving data on a storage device, such as a storage device 1404 discussed herein with reference to FIG. 14, sending data to another user, labeling data, etc.).

[0124] As noted above, the scientific instrument support module 800 of FIG. 8 may be implemented by one or more computing devices. FIG. 14 is a block diagram of the computing device 1400 that may perform some or all of the scientific instrument support methods disclosed herein, in accordance with various embodiments. In some embodiments, the scientific instrument support module 800 may be implemented by a single computing device 1400 or by multiple computing devices 1400. Further, as discussed below, a computing device 1400 (or multiple computing devices 1400) that implements the scientific instrument support module 800 may be part of one or more of the scientific instrument 1510, the user local computing device 1520, a service local computing device 1530, or a remote computing device 1540 of FIG. 15.

[0125] The computing device 1400 of FIG. 14 is illustrated as having a number of components, but any one or more of these components may be omitted or duplicated, as suitable for the application and setting. In some embodiments, some or all of the components included in the computing device 1400 may be attached to one or more motherboards and enclosed in a housing (e.g., including plastic, metal, and / or other materials). In some embodiments, some of these components may be fabricated onto a single system-on-a-chip (SoC) (e.g., an SoC may include one or more processing devices 1402 and one or more storage devices 1404). Additionally, in various embodiments, the computing device 1400 may notinclude one or more of the components illustrated in FIG. 14, but may include interface circuitry (not shown) for coupling to the one or more components using any suitable interface (e.g., a Universal Serial Bus (USB) interface, a High-Definition Multimedia Interface (HDMI) interface, a Controller Area Network (CAN) interface, a Serial Peripheral Interface (SPI) interface, an Ethernet interface, a wireless interface, or any other appropriate interface) . For example, the computing device 1400 may not include the display device 1410, but may include display device interface circuitry (e.g., a connector and driver circuitry) to which the display device 1410 may be coupled.

[0126] The computing device 1400 may include the processing device 1402 (e.g., one or more processing devices). As used herein, the term "processing device" may refer to any device or portion of a device that processes electronic data from registers and / or memory to transform that electronic data into other electronic data that may be stored in registers and / or memory. The processing device 1402 may include one or more digital signal processors (DSPs), application-specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptoprocessors (specialized processors that execute cryptographic algorithms within hardware), server processors, or any other suitable processing devices.

[0127] The computing device 1400 may include a storage device 1404 (e.g., one or more storage devices). The storage device 1404 may include one or more memory devices such as random access memory (RAM) (e.g., static RAM (SRAM) devices, magnetic RAM (MRAM) devices, dynamic RAM (DRAM) devices, resistive RAM (RRAM) devices, or conductive- bridging RAM (CBRAM) devices), hard drive-based memory devices, solid-state memory devices, networked drives, cloud drives, or any combination of memory devices. In some embodiments, the storage device 1404 may include memory that shares a die with the processing device 1402. In such an embodiment, the memory may be used as cache memory and may include embedded dynamic random access memory (eDRAM) or spin transfer torque magnetic random access memory (STT-MRAM), for example. In some embodiments, the storage device 1404 may include non-transitory computer readable media having instructions thereon that, when executed by one or more processing devices (e.g., the processing device 1402), cause the computing device 1400 to perform any appropriate ones of or portions of the methods disclosed herein.

[0128] The computing device 1400 may include an interface device 1406 (e.g., one or more interface devices 1406). The interface device 1406 may include one or more communicationchips, connectors, and / or other hardware and software to govern communications between the computing device 1400 and other computing devices. For example, the interface device 1406 may include circuitry for managing wireless communications for the transfer of data to and from the computing device 1400. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communications channels, etc., that may communicate data through the use of modulated electromagnetic radiation through a nonsolid medium. The term does not imply that the associated devices do not contain any wires, although in some embodiments they might not. Circuitry included in the interface device 1406 for managing wireless communications may implement any of a number of wireless standards or protocols, including but not limited to Institute for Electrical and Electronic Engineers (IEEE) standards including Wi-Fi (IEEE 802.11 family), IEEE 802.16 standards (e.g., IEEE 802.16-2005 Amendment), Long-Term Evolution (LTE) project along with any amendments, updates, and / or revisions (e g., advanced LTE project, ultra mobile broadband (UMB) project (also referred to as "3GPP2"), etc ). In some embodiments, circuitry included in the interface device 1406 for managing wireless communications may operate in accordance with a Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE network. In some embodiments, circuitry included in the interface device 1406 for managing wireless communications may operate in accordance with Enhanced Data for GSM Evolution (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). In some embodiments, circuitry included in the interface device 1206 for managing wireless communications may operate in accordance with Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Telecommunications (DECT), Evolution-Data Optimized (EV-DO), and derivatives thereof, as well as any other wireless protocols that are designated as 3G, 4G, 5G, and beyond. In some embodiments, the interface device 1406 may include one or more antennas (e.g., one or more antenna arrays) to receipt and / or transmission of wireless communications.

[0129] In some embodiments, the interface device 1406 may include circuitry for managing wired communications, such as electrical, optical, or any other suitable communication protocols. For example, the interface device 1406 may include circuitry to support communications in accordance with Ethernet technologies. In some embodiments, the interfacedevice 1406 may support both wireless and wired communication, and / or may support multiple wired communication protocols and / or multiple wireless communication protocols. For example, a first set of circuitry of the interface device 1406 may be dedicated to shorter-range wireless communications such as Wi-Fi or Bluetooth, and a second set of circuitry of the interface device 1406 may be dedicated to longer-range wireless communications such as global positioning system (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, or others. In some embodiments, a first set of circuitry of the interface device 1406 may be dedicated to wireless communications, and a second set of circuitry of the interface device 1406 may be dedicated to wired communications.

[0130] The computing device 1400 may include battery / power circuitry 1408. The battery / power circuitry 1408 may include one or more energy storage devices (e.g., batteries or capacitors) and / or circuitry for coupling components of the computing device 1400 to an energy source separate from the computing device 1400 (e g., AC line power).

[0131] The computing device 1400 may include the display device 1410 (e.g., multiple display devices). The display device 1410 may include any visual indicators, such as a headsup display, a computer monitor, a projector, a touchscreen display, a liquid crystal display (LCD), a light-emitting diode display, or a flat panel display.

[0132] The computing device 1400 may include the other input / output (I / O) devices 1412. The other I / O devices 1412 may include one or more audio output devices (e.g., speakers, headsets, earbuds, alarms, etc.), one or more audio input devices (e.g., microphones or microphone arrays), location devices (e.g., GPS devices in communication with a satellitebased system to receive a location of the computing device 1400, as known in the art), audio codecs, video codecs, printers, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors, vibration sensors, accelerometers, gyroscopes, etc.), image capture devices such as cameras, keyboards, cursor control devices such as a mouse, a stylus, a trackball, or a touchpad, bar code readers, Quick Response (QR) code readers, or radio frequency identification (RFID) readers, for example.

[0133] The computing device 1400 may have any suitable form factor for its application and setting, such as a handheld or mobile computing device (e.g., a cell phone, a smart phone, a mobile internet device, a tablet computer, a laptop computer, a netbook computer, an ultrabook computer, a personal digital assistant (PDA), an ultra mobile personal computer, etc.), adesktop computing device, or a server computing device or other networked computing component.

[0134] One or more computing devices implementing any of the scientific instrument support modules or methods disclosed herein may be part of a scientific instrument support system. FIG. 15 is a block diagram of an example scientific instrument support system 1500 in which some or all of the scientific instrument support methods disclosed herein may be performed, in accordance with various embodiments. The scientific instrument support modules and methods disclosed herein (e.g., the scientific instrument support module 800 of FIG. 8 and the methods 800-1200 of FIGS. 9-12) may be implemented by one or more of the scientific instrument 1510, the user local computing device 1520, the service local computing device 1530, or the remote computing device 1540 of the scientific instrument support system 1500.

[0135] Any of the scientific instrument 1510, the user local computing device 1520, the service local computing device 1530, or the remote computing device 1540 may include any of the embodiments of the computing device 1400 discussed herein with reference to FIG. 14, and any of the scientific instrument 1510, the user local computing device 1520, the service local computing device 1530, or the remote computing device 1540 may take the form of any appropriate ones of the embodiments of the computing device 1400 discussed herein with reference to FIG. 14.

[0136] The scientific instrument 1510, the user local computing device 1520, the service local computing device 1530, or the remote computing device 1540 may each include a processing device 1502, a storage device 1504, and an interface device 1506. The processing device 1502 may take any suitable form, including the form of any of the processing devices 1402 discussed herein with reference to FIG. 14, and the processing devices 1502 included in different ones of the scientific instrument 1510, the user local computing device 1520, the service local computing device 1530, or the remote computing device 1540 may take the same form or different forms. The storage device 1504 may take any suitable form, including the form of any of the storage devices 1404 discussed herein with reference to FIG. 14, and the storage devices 1504 included in different ones of the scientific instrument 1510, the user local computing device 1520, the service local computing device 1530, or the remote computing device 1540 may take the same form or different forms. The interface device 1506 may take any suitable form, including the form of any of the interface devices 1406 discussed herein with reference to FIG. 14, and the interface devices 1506 included in different ones of thescientific instrument 1510, the user local computing device 1520, the service local computing device 1530, or the remote computing device 1540 may take the same form or different forms.

[0137] The scientific instrument 1510, the user local computing device 1520, the service local computing device 1530, and the remote computing device 1540 may be in communication with other elements of the scientific instrument support system 1500 via communication pathways 1508. The communication pathways 1508 may communicatively couple the interface devices 1506 of different ones of the elements of the scientific instrument support system 1500, as shown, and may be wired or wireless communication pathways (e.g., in accordance with any of the communication techniques discussed herein with reference to the interface devices 1406 of the computing device 1400 of FIG. 14). The particular scientific instrument support system 1500 depicted in FIG. 15 includes communication pathways between each pair of the scientific instrument 1510, the user local computing device 1520, the service local computing device 1530, and the remote computing device 1540, but this “fully connected” implementation is simply illustrative, and in various embodiments, various ones of the communication pathways 1508 may be absent. For example, in some embodiments, a service local computing device 1530 may not have a direct communication pathway 1508 between its interface device 1506 and the interface device 1506 of the scientific instrument 1510, but may instead communicate with the scientific instrument 1510 via the communication pathway 1508 between the service local computing device 1530 and the user local computing device 1520 and the communication pathway 1508 between the user local computing device 1520 and the scientific instrument 1510.

[0138] The scientific instrument 1510 may include any appropriate scientific instrument, such as a charged particle microscope (e.g., an electron microscope, such as a TEM or SEM), a FIB system, or a charged particle dual beam microscope / system.

[0139] The user local computing device 1520 may be a computing device (e.g., in accordance with any of the embodiments of the computing device 1400 discussed herein) that is local to a user of the scientific instrument 1510. In some embodiments, the user local computing device 1520 may also be local to the scientific instrument 1510, but this need not be the case; for example, a user local computing device 1520 that is in a user’s home or office may be remote from, but in communication with, the scientific instrument 1510 so that the user may use the user local computing device 1520 to control and / or access data from the scientific instrument 1510. In some embodiments, the user local computing device 1520 may be a laptop,smartphone, or tablet device. In some embodiments the user local computing device 1520 may be a portable computing device.

[0140] The service local computing device 1530 may be a computing device (e.g., in accordance with any of the embodiments of the computing device 1400 discussed herein) that is local to an entity that services the scientific instrument 1510. For example, the service local computing device 1530 may be local to a manufacturer of the scientific instrument 1510 or to a third-party service company. In some embodiments, the service local computing device 1530 may communicate with the scientific instrument 1510, the user local computing device 1520, and / or the remote computing device 1540 (e.g., via a direct communication pathway 1508 or via multiple “indirect” communication pathways 1508, as discussed above) to receive data regarding the operation of the scientific instrument 1510, the user local computing device 1520, and / or the remote computing device 1540 (e.g., the results of self-tests of the scientific instrument 1510, calibration coefficients used by the scientific instrument 1510, the measurements of sensors associated with the scientific instrument 1510, etc.). In some embodiments, the service local computing device 1530 may communicate with the scientific instrument 1510, the user local computing device 1520, and / or the remote computing device 1540 (e.g., via a direct communication pathway 1508 or via multiple “indirect” communication pathways 1508, as discussed above) to transmit data to the scientific instrument 1510, the user local computing device 1520, and / or the remote computing device 1540 (e.g., to update programmed instructions, such as firmware, in the scientific instrument 1510, to initiate the performance of test or calibration sequences in the scientific instrument 1510, to update programmed instructions, such as software, in the user local computing device 1520 or the remote computing device 1540, etc.). A user of the scientific instrument 1510 may utilize the scientific instrument 1510 or the user local computing device 1520 to communicate with the service local computing device 1530 to report a problem with the scientific instrument 1510 or the user local computing device 1520, to request a visit from a technician to improve the operation of the scientific instrument 1510, to order consumables or replacement parts associated with the scientific instrument 1510, or for other purposes.

[0141] The remote computing device 1540 may be a computing device (e.g., in accordance with any of the embodiments of the computing device 1400 discussed herein) that is remote from the scientific instrument 1510 and / or from the user local computing device 1520. In some embodiments, the remote computing device 1540 may be included in a datacenter or otherlarge-scale server environment. In some embodiments, the remote computing device 1540 may include network-attached storage (e.g., as part of the storage device 1504). The remote computing device 1540 may store data generated by the scientific instrument 1510, perform analyses of the data generated by the scientific instrument 1510 (e.g., in accordance with programmed instructions), facilitate communication between the user local computing device 1520 and the scientific instrument 1510, and / or facilitate communication between the service local computing device 1530 and the scientific instrument 1510.

[0142] In some embodiments, one or more of the elements of the scientific instrument support system 1500 illustrated in FIG. 15 may not be present. Further, in some embodiments, multiple ones of various ones of the elements of the scientific instrument support system 1500 of FIG.15 may be present. For example, a scientific instrument support system 1500 may include multiple user local computing devices 1520 (e.g., different user local computing devices 1520 associated with different users or in different locations). In another example, a scientific instrument support system 1500 may include multiple scientific instruments 1510, all in communication with service local computing device 1530 and / or a remote computing device 1540; in such an embodiment, the service local computing device 1530 may monitor these multiple scientific instruments 1510, and the service local computing device 1530 may cause updates or other information may be “broadcast” to multiple scientific instruments 1510 at the same time. Different ones of the scientific instruments 1510 in a scientific instrument support system 1500 may be located close to one another (e.g., in the same room) or farther from one another (e.g., on different floors of a building, in different buildings, in different cities, etc.). In some embodiments, a scientific instrument 1510 may be connected to an Internet-of-Things (loT) stack that allows for command and control of the scientific instrument 1510 through a web-based application, a virtual or augmented reality application, a mobile application, and / or a desktop application. Any of these applications may be accessed by a user operating the user local computing device 1520 in communication with the scientific instrument 1510 by the intervening remote computing device 1540. In some embodiments, a scientific instrument 1510 may be sold by the manufacturer along with one or more associated user local computing devices 1520 as part of a local scientific instrument computing unit 1512.

[0143] In some such embodiments, the remote computing device 1540 and / or the user local computing device 1520 may combine data from different types of scientific instruments 1510 included in a scientific instrument support system 1500.

[0144] Moreover, in at least one embodiment, a digital twins system may be implemented at one or more computing devices of the scientific instrument support system 1500. For example, the digital twins system may be deployed at the user local computing device 1520, the remote computing device 1540, and / or the service local computing device 1530. Regardless of location, the digital twins system may receive information and data from the scientific instrument 1510 and use the information and data to simulate one or more of a physical asset (e.g., the scientific instrument 1510), a sample, and interaction between the physical asset and the sample using logic implemented at a scientific instrument support module, as described above with reference to the scientific instrument support module of FIG. 8.

[0145] The following paragraphs provide various examples of the embodiments disclosed herein:In at least one embodiment, a scientific instrument support system comprises a sample, a physical asset configured to interact with the sample, and one or more computing devices having executable instructions stored thereon that, when executed, cause the one or more computing devices to generate one or more predictions based on simulations of one or more of the sample, the physical asset, or an interaction of the physical asset with the sample, and update a physics-based digital model of the sample based on the accuracy of the one or more predictions, wherein the physics-based digital model is used to reconstruct chemical and physical properties of the sample to generate the simulations of the sample. In an embodiment that includes the preceding embodiment, the executable instructions further cause the one or more computing devices to use the physics-based digital model to generate training data to train one or more neural networks to generate the one or more predictions, and wherein the one or more neural networks are implemented at the one or more computing devices. In an embodiment that includes any of the preceding embodiments, the physics-based digital model of the sample is calibrated based on data obtained using one or more analytical techniques to set initial parameter values of the physics-based digital model, and on comparison of a simulated outcome of the interaction of the physical asset with the sample to an experimental outcome of the interaction of the physical asset with the sample. In an embodiment that includes any of the preceding embodiments, feedback from a controller of the physical asset during operation of the physical asset is used to update the physics-based digital model during the operation of the physical asset. In an embodiment that includes any of the preceding embodiments, the accuracy of the one or more predictions is evaluated based on a comparisonof data obtained using a simulated outcome of operation of the physical asset to data obtained from an experimental outcome of operation of the physical asset. In an embodiment that includes any of the preceding embodiments, the one or more predictions are generated based on a request received from an operator, and wherein the one or more predictions are further generated based on information regarding one or more properties and a state of the physical asset, the information retrieved from a controller of the physical asset. In an embodiment that includes any of the preceding embodiments, the one or more predictions are output by one or more neural networks trained to infer optimized settings of the physical asset to be used during operation of the physical asset, and wherein the one or more neural networks are trained based on the simulations. In an embodiment that includes any of the preceding embodiments, the physical asset comprises an apparatus configured to emit a charged particle beam, and wherein the interaction of the physical asset with the sample comprises impingement of the charged particle beam on the sample.

[0146] In at least one embodiment, a method comprises receiving, at one or more computing devices, a request to perform an operation, using a physical asset, on a sample, performing, at the one or more computing devices and in response to the request, one or more simulations of an interaction between the physical asset and the sample based, at least in part, on a digital model of the sample, returning, at a user interface of the one or more computing devices, optimized settings for the physical asset determined based on the one or more simulations, and updating, using the one or more computing devices, the digital model based on a comparison of an outcome of using the optimized settings to a predicted outcome. In an embodiment that includes the preceding embodiment, receiving the request comprises receiving sample information and / or a target outcome, and wherein each of a simulation of the operation and execution of the operation using the physical asset is performed based on the target outcome. In an embodiment that includes any of the preceding embodiments, updating the digital model comprises using the comparison to determine a difference between the outcome of using the optimized settings and the predicted outcome, and adjusting parameters of the digital model based on the difference. In an embodiment that includes any of the preceding embodiments, the digital model is updated while the operation is performed using the physical asset, and wherein the digital model is updated based on periodically obtained comparisons of experimental progress of the operation to predicted progress of the operation. In an embodiment that includes any of the preceding embodiments, the digital model comprises one or more neural networks,and wherein the one or more neural networks are trained to perform the one or more simulations, at least in part, using simulations generated by a physics-based model configured to reconstruct chemical and physical properties of a sample. In an embodiment that includes any of the preceding embodiments, in response to receiving the request, the method further comprises acquiring information regarding properties of the physical asset relevant to performing the operation, and predicting values to be used for the optimized settings, and wherein the predicted values are adjusted, while the physical asset performs the operation, in response to detected deviations between measured progress of the operation and predicted progress of the operation. In an embodiment that includes any of the preceding embodiments, the physical asset comprises a focused ion beam microscope, and wherein the one or more simulations comprise removal of material from the sample using a focused ion beam emitted by the focused ion beam microscope.

[0147] In at least one embodiment, a non -transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform one or more simulations, using a physics-based model of a sample and a physics-based model of a physical asset, based on a requested outcome, predict settings, based on the one or more simulations, to be used to operate the physical asset, operate the physical asset using the predicted settings, compare an actual outcome of the operation of the physical asset to the requested outcome, and use the comparison to adjust the physics-based model of the sample. In an embodiment that the preceding embodiment, the physics-based model of the physical asset is generated based on a computer-aided design (CAD) drawing of at least a portion of the physical asset. In an embodiment that includes any of the preceding embodiments, the physics-based model of the sample is generated based on sample information comprising one or more of known material properties of the sample, a chemical element profile of the sample, or a compositional distribution of the sample, and wherein the sample information is obtained from one or more of an operator, a three-dimensional (3D) model of the sample, a database, a lookup table, or a textual description. In an embodiment that includes any of the preceding embodiments, the one or more simulations are performed based on the definition of a 3D space to conduct the one or more simulations, generation of objects interacting within the 3D space, and wherein the one or more simulations comprise modeling of surface evolution during a milling process to be performed on the sample. In an embodiment that includes any of the preceding embodiments, the physics-based model of the sample isimplemented by one or more neural networks that are trained offline to perform the one or more simulations, and wherein the comparison is used to update weights of the one or more neural networks.

[0148] While the present disclosure has been described in detail with reference to specific embodiments, it will be apparent to those skilled in the art that various modifications and changes can be made without departing from the spirit and scope of the disclosure.Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

Claims

CLAIMSWhat is claimed is:

1. A scientific instrument support system, comprising: a sample; a physical asset configured to interact with the sample; and one or more computing devices having executable instructions stored thereon that, when executed, cause the one or more computing devices to: generate one or more predictions based on simulations of one or more of the sample, the physical asset, or an interaction of the physical asset with the sample; and update a physics-based digital model of the sample based on accuracy of the one or more predictions, wherein the physics-based digital model is used to reconstruct chemical and physical properties of the sample to generate the simulations of the sample.2 The scientific instrument support system of claim 1, wherein the executable instructions further cause the one or more computing devices to: use the physics-based digital model to generate training data to train one or more neural networks to generate the one or more predictions, and wherein the one or more neutral networks are implemented at the one or more computing devices.

3. The scientific instrument support system of claim 1, wherein the physics-based digital model of the sample is calibrated based on data obtained using one or more analytical techniques to set initial parameter values of the physics-based digital model, and on comparison of a simulated outcome of the interaction of the physical asset with the sample to an experimental outcome of the interaction of the physical asset with the sample.

4. The scientific instrument support system of claim 1, wherein feedback from a controller of the physical asset during operation of the physical asset is used to update the physics-based digital model during the operation of the physical asset.

5. The scientific instrument support system of claim 1, wherein the accuracy of the one or more predictions is evaluated based on a comparison of data obtained using a simulated outcome of operation of the physical asset to data obtained from an experimental outcome of operation of the physical asset.

6. The scientific instrument support system of claim 1, wherein the one or more predictions are generated based on a request received from an operator, and wherein the one or more predictions are further generated based on information regarding one or more properties and a state of the physical asset, the information retrieved from a controller of the physical asset.

7. The scientific instrument support system of claim 1, wherein the one or more predictions are output by one or more neural networks trained to infer optimized settings of the physical asset to be used during operation of the physical asset, and wherein the one or more neural networks are trained based on the simulations.

8. The scientific instrument support system of claim 1, wherein the physical asset comprises an apparatus configured to emit a charged particle beam, and wherein the interaction of the physical asset with the sample comprises impingement of the charged particle beam on the sample.

9. A method, comprising: receiving, at one or more computing devices, a request to perform an operation, using a physical asset, on a sample; performing, at the one or more computing devices and in response to the request, one or more simulations of an interaction between the physical asset and the sample based, at least in part, on a digital model of the sample; returning, at a user interface of the one or more computing devices, optimized settings for the physical asset determined based on the one or more simulations,; and updating, using the one or more computing devices, the digital model based on a comparison of an outcome of using the optimized settings to a predicted outcome.

10. The method of claim 9, wherein receiving the request comprises receiving sample information and / or a target outcome, and wherein each of a simulation of the operation and execution of the operation using the physical asset is performed based on the target outcome.

11. The method of claim 9, wherein updating the digital model comprises using the comparison to determine a difference between the outcome of using the optimized settings and the predicted outcome, and adjusting parameters of the digital model based on the difference.

12. The method of claim 9, wherein the digital model is updated while the operation is performed using the physical asset, and wherein the digital model is updated based on periodically obtained comparisons of experimental progress of the operation to predicted progress of the operation.

13. The method of claim 9, wherein the digital model comprises one or more neural networks, and wherein the one or more neural networks are trained to perform the one or more simulations, at least in part, using simulations generated by a physics-based model configured to reconstruct chemical and physical properties of a sample.

14. The method of claim 9, wherein, in response to receiving the request, the method further comprises acquiring information regarding properties of the physical asset relevant to performing the operation, and predicting values to be used for the optimized settings, and wherein the predicted values are adjusted, while the physical asset performs the operation, in response to detected deviations between measured progress of the operation and predicted progress of the operation.

15. The method of claim 9, wherein the physical asset comprises a focused ion beam microscope, and wherein the one or more simulations comprise removal of material from the sample using a focused ion beam emitted by the focused ion beam microscope.

16. A n on-transitory computer- readable medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to: perform one or more simulations, using a physics-based model of a sample and a physics-based model of a physical asset, based on a requested outcome; predict settings, based on the one or more simulations, to be used to operate the physical asset; operate the physical asset using the predicted settings; compare an actual outcome of the operation of the physical asset to the requested outcome; and use the comparison to adjust the physics-based model of the sample.

17. The non-transitory computer-readable medium of claim 16, wherein the physics-based model of the physical asset is generated based on a computer-aided design (CAD) drawing of at least a portion of the physical asset.

18. The non-transitory computer-readable medium of claim 16, wherein the physics-based model of the sample is generated based on sample information comprising one or more of known material properties of the sample, a chemical element profile of the sample, or a compositional distribution of the sample, and wherein the sample information is obtained from one or more of an operator, a three-dimensional (3D) model of the sample, a database, a lookup table, or a textual description.

19. The non-transitory computer-readable medium of claim 16, wherein the one or more simulations are performed based on definition of a 3D space to conduct the one or more simulations, generation of objects interacting within the 3D space, and wherein the one or more simulations comprise modeling of surface evolution during a milling process to be performed on the sample.

20. The non-transitory computer-readable medium of claim 16, wherein the physics-based model of the sample is implemented by one or more neural networks that is trained offline to perform the one or more simulations, and wherein the comparison is used to update weights of the one or more neural networks.

Images