Microscope aberration correction

Abstract

To provide a support apparatus for a scientific instrument which includes an electron microscope column and associated Ronchigram acquisition circuitry.SOLUTION: An interface device receives acquired Ronchigrams and transmits control signals for electron-beam optics thereof. The processing device converts a measured Ronchigram into an input token for a transformer, produces an output token based on a tokenized sentence ending with the input token, and determines adjustments of the control signals based on the input and output tokens. The input and output tokens belong to a plurality of tokens representing reference Ronchigrams sampling an alignment parameter space of the electron-beam optics. The transformer implements an autoregressive masked language model trained on a corpus of reference sentences representing paths through the alignment parameter space to a target alignment state of the electron-beam optics.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Non-provisional Patent Application No. 18 / 345,675, filed June 30, 2023, the disclosure of which is incorporated by reference herein in its entirety.

[0002] Various illustrative embodiments relate generally, but not exclusively, to electron microscope components, instruments, systems, and methods. [Background technology]

[0003] The resolution of an electron microscope may be limited by imperfections in the electron beam lenses. The results of such imperfections are referred to as aberrations due to their similarity to the aberrations observed in optical microscopes. For example, before an experimental measurement with a scanning transmission electron microscope (STEM), an operator typically needs to adjust the electron beam lenses of the microscope so that at least some aberrations are corrected. In some cases, such adjustments need to be repeated, for example, several times a day, to offset detuning caused by various drift processes in the circuitry associated with the electron beam lenses. When performed manually by an operator, this type of adjustment is typically tedious, time-consuming, and / or suboptimal. Summary of the Invention

[0004] Disclosed herein are various examples, aspects, features, and embodiments of a scientific instrument including, among other things, an electron microscope column and associated Ronchigram acquisition circuitry. An electronic controller of the scientific instrument employs a transformer to automatically correct at least some geometric aberrations of the electron beam optics in the electron microscope column. According to an exemplary embodiment, the transformer implements an autoregressive masked language model trained with a corpus of reference sentences representing various paths through the alignment parameter space of the electron beam optics to its target alignment state. A return loop of the transformer is connected to the Ronchigram acquisition circuitry, enabling the transformer to automatically drive the alignment process of the scientific instrument towards the target alignment state based on the simultaneously acquired Ronchigrams, thereby beneficially eliminating the need for manual (re)alignment of the electron beam optics by an operator.

[0005] One embodiment provides a support apparatus for a scientific instrument, comprising: an interface device configured to receive data representing a Ronchigram acquired using an electron microscope column of the scientific instrument and further configured to send one or more control signals for the electron beam optics of the electron microscope column; and a processing device configured to convert the measured Ronchigram into input tokens for a transformer, generate output tokens using the transformer based on tokenized sentences ending with the input tokens, and determine one or more adjustments to the one or more control signals based on the input tokens and the output tokens, wherein the input tokens and the output tokens belong to a plurality of tokens representing a plurality of reference Ronchigrams sampling an alignment parameter space of the electron beam optics, and the transformer implements an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a respective path through the alignment parameter space to a target alignment state of the electron beam optics.

[0006] Another embodiment provides an automated method implemented via a computing device for providing assistance to a scientific instrument, the automated method including: converting a measured Ronchigram into input tokens for a transformer, where the measured Ronchigram is received via data acquired using an electron microscope column of the scientific instrument; generating output tokens using the transformer based on tokenized sentences ending with the input tokens; and determining one or more adjustments to one or more control signals for the electron beam optics of the electron microscope column based on the input tokens and the output tokens, wherein the input tokens and the output tokens belong to a plurality of tokens representing a plurality of reference Ronchigrams sampling an alignment parameter space of the electron beam optics, and the transformer implements an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a respective path through the alignment parameter space to a target alignment state of the electron beam optics.

[0007] Yet another embodiment provides a non-transitory computer-readable medium storing instructions that, when executed by a computing device, cause the computing device to perform operations including the automated methods described above. [Brief description of the drawings]

[0008] The foregoing aspects and many of the attendant advantages of the present disclosure will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, in which: [Figure 1] 1 is a block diagram illustrating an exemplary scientific instrument, according to some embodiments. [Diagram 2] 2 is a block diagram illustrating the acquisition of a Ronchigram on the scientific instrument of FIG. 1 in accordance with some embodiments. [Figure 3A] 1 is a Ronchigram chart corresponding to a two-dimensional (2D) alignment parameter space of a scientific instrument 100, according to some embodiments. [Figure 3B] 1 is a Ronchigram chart corresponding to a two-dimensional (2D) alignment parameter space of a scientific instrument 100, according to some embodiments. [Figure 3C] 1 is a Ronchigram chart corresponding to a two-dimensional (2D) alignment parameter space of a scientific instrument 100, according to some embodiments. [Figure 4A] 2 diagrammatically illustrates the generation performance of a digital twin of the scientific instrument of FIG. 1 , according to some embodiments. [Figure 4B] 2 diagrammatically illustrates the generation performance of a digital twin of the scientific instrument of FIG. 1 , according to some embodiments. [Figure 4C] 2 diagrammatically illustrates the generation performance of a digital twin of the scientific instrument of FIG. 1 , according to some embodiments. [Figure 4D] 2 diagrammatically illustrates the generation performance of a digital twin of the scientific instrument of FIG. 1 , according to some embodiments. [Figure 4E] 2 diagrammatically illustrates the generation performance of a digital twin of the scientific instrument of FIG. 1 , according to some embodiments. [Figure 4F] 2 diagrammatically illustrates the generation performance of a digital twin of the scientific instrument of FIG. 1 , according to some embodiments. [Figure 4G] 2 diagrammatically illustrates the generation performance of a digital twin of the scientific instrument of FIG. 1 , according to some embodiments. [Figure 4H] 2 diagrammatically illustrates the generation performance of a digital twin of the scientific instrument of FIG. 1 , according to some embodiments. [Diagram 5] 2 is a flow chart illustrating a method of training an artificial intelligence (AI) agent to perform Ronchigram alignment in the scientific instrument of FIG. 1 according to some embodiments. [Figure 6] 6 is a Ronchigram chart illustrating certain operations of the method of FIG. 5, according to some embodiments. [Figure 7] 2 is a block diagram illustrating a transformer used to perform Ronchigram alignment in the scientific instrument of FIG. 1 in accordance with some embodiments. [Figure 8] 8 is a block diagram illustrating an optical system alignment circuit incorporating the transformer of FIG. 7 according to some embodiments. [Figure 9] 2 is a flowchart of an automated method for aligning electron beam optics in the scientific instrument of FIG. 1 in accordance with some embodiments. [Figure 10] FIG. 1 is a block diagram of an example computing device configured to perform at least some scientific instrument-assisted operations, in accordance with various embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An exemplary STEM alignment method includes two or more sets of alignment operations. One set of such alignment operations, sometimes referred to as Ronchigram alignment, involves adjusting the electron beam optics based on an observed Ronchigram pattern. Ronchigram alignment can be used to correct both low-order and high-order aberrations. In some implementations, after Ronchigram alignment, a condenser lens aperture is inserted, and then at least some residual aberrations are finely corrected to obtain high-quality STEM images. Disadvantageously, at least some users find Ronchigram alignment difficult to perform.

[0010] As used herein, the term "Ronchigram" refers to the diffraction pattern of a convergent electron beam focused on an amorphous specimen. In mathematical terms, the lens aberration function χ(k) can be expressed by Equation (1):

[0011]

number

[0012]

number

[0013] Important features encoded in the texture of the Ronchigram are magnification and symmetry. For an in-focus beam, the center of the Ronchigram has high local magnification, representing a coherent and substantially aberration-free portion of the electron beam. Further away from the optical axis, aberrations reduce the local magnification. The presence of aberrations affects the structure and symmetry of the Ronchigram. For example, asymmetric aberrations (m>0) break the rotational symmetry. Two-fold astigmatism (C 12 ) typically stretches high magnification areas in one direction, producing pronounced streaking. 21 ) shifts the center of the Ronchigram. Three-fold astigmatism (C 23 ) produces three-fold symmetric lobes, and so on. When the microscope is aligned, the various currents through the electron beam lenses can be adjusted to minimize the various manifestations of aberrations in the Ronchigram. In a well-aligned microscope, low-order aberrations are almost perfectly corrected, resulting in large featureless areas of high local magnification, often with the characteristic six-fold contours due to residual higher-order aberrations.

[0014] FIG. 1 is a block diagram illustrating a scientific instrument 100, according to some embodiments. The scientific instrument 100 includes a scanning transmission electron microscope (STEM) column 102 coupled to a vacuum chamber 108. The vacuum chamber 108 houses a movable sample holder 110 therein and can be evacuated using one or more vacuum pumps (not explicitly shown in FIG. 1). In an exemplary embodiment, the sample holder 110 is independently movable parallel to an XY coordinate plane and parallel to a Z coordinate axis, with the corresponding coordinate system depicted by the XYZ coordinate triad shown in FIG. 1. A sample S to be investigated using the scientific instrument 100 is mounted on the sample holder 110 as shown in FIG. 1.

[0015] In the example shown, the STEM column 102 comprises an electron source 112 and two or more electron beam lenses, of which only two are shown diagrammatically in Figure 1 for illustrative purposes: objective lens 106 and condenser lens 116. In some examples, a different number of such lenses (other than two) may be used in the STEM column 102. In some embodiments, the objective lens 106 may be an ultra-high-resolution (UHR) lens.

[0016] In operation, the electron source 112 generates an electron beam 114 that propagates generally along a longitudinal axis 115 of the STEM column 102. The electron beam lenses 106 and 116 operate to generate electric and magnetic fields that affect the electron trajectories within the electron beam 114. Control signals 152, 156 generated by an electronic controller 150 are used to modify the strength and / or spatial configuration of these fields to impart desired characteristics to the electron beam 114. In general, the electron beam lenses 106 and 116, the control signals 152 and 156, and other associated components of the scientific instrument 100 can be used to perform various operations and assist in various functions such as beam focusing, aberration mitigation, aperture cropping, filtering, etc. The STEM column 102 further comprises a deflection unit 118 that can steer the electron beam 114 in response to control signals 154 applied by the electronic controller 150. Such beam steering can be used to move a focused portion of the electron beam 114 along a desired path across the sample S, for example to perform a raster or vector scan of the sample S.

[0017] The scientific instrument 100 also includes detectors 160, 170, 180 located within the vacuum chamber 108 in relative proximity to the sample S. During operation, the detectors 160, 170, and 180 generate streams of measurements 162, 172, and 182 that are received by the electronic controller 150. The specific types of detectors 160, 170, 180 depend on the embodiment of the scientific instrument 100 and can typically be selected from a variety of detector types suitable for detecting different types of emissions and / or radiation from the sample S generated in response to the electron beam 114. Exemplary types of emissions / radiation that may be generated in this manner include, but are not limited to, x-rays, infrared, visible light, ultraviolet light, backscattered electrons, secondary electrons, Auger electrons, elastically scattered electrons, non-scattered (e.g., zero energy loss) electrons, and non-elastically scattered electrons. In various embodiments, a different number (other than three) of such detectors can be used within the scientific instrument 100. In some embodiments, detectors 160, 170, 180 are selected from the group consisting of high angle annular dark field detectors, medium angle annular dark field detectors, annular bright field detectors, segmented annular detectors, differential phase contrast detectors, and two-dimensional (e.g., pixelated) diffraction pattern detectors. Other detectors capable of detecting various of the above-mentioned types of emission / radiation may also be used in various additional embodiments.

[0018] FIG. 2 is a block diagram 200 illustrating the acquisition of a Ronchigram image 210 with a scientific instrument 100, according to some embodiments. For simplicity, only one probe-forming lens, namely, an electron beam lens 202, is shown in FIG. 2. The electron beam lens 202 represents the objective lens pre-field and may in fact be implemented with two or more electron beam lenses (see, for example, electron beam lenses 106, 116 in FIG. 1). The compression of the electron beam 114 caused by the objective lens post-field and the effect of any post-specimen lenses are not explicitly shown in FIG. 2. The Ronchigram image 210 is captured by a two-dimensional pixelated detector (e.g., a CCD camera) 180.

[0019] In some implementations, prior to Ronchigram acquisition, the sample S and / or the electron beam 114 are positioned such that the electron beam 114 strikes an amorphous region of the sample S. In some other implementations, a crystalline region of the sample S may also be used. During acquisition of the Ronchigram image 210, the electron beam 114 remains stationary and the post-specimen intensity of the beam is recorded by the CCD camera 180, or by a functionally equivalent device. To perform the Ronchigram alignment referred to above, a series of Ronchigram images 210 are acquired, each of which is captured after a small change in the electrical excitation of selected components of the probe-forming lens 202.

[0020] Since the intensity of the electron beam 114 formed on the Fraunhofer diffractive surface of the STEM varies strongly with angle, and this variation is a sensitive function of lens aberrations and defocus d, the Ronchigram series described above can be used to guide STEM alignment. Small misalignments are revealed by translations of the Ronchigram pattern away from circular symmetry as the excitation of the optical components of the probe-forming lens 202 is slightly changed. Furthermore, the presence or absence of interference fringes in the Ronchigram pattern correlates with the amount of incoherent broadening.

[0021] At large defocus d, the crossover point F of the electron beam is at a relatively large distance from the sample S along the longitudinal axis 115, and the projected image can be observed using the detector 180. As Gaussian focus is approached, an angular dependence on the magnification typically appears, for example, due to lens aberrations and the manner in which lens aberrations affect the phase of the electron beam 114. At slight underfocus (i.e., when the crossover point F is slightly below the sample S), azimuthal and radial circles of substantially infinite magnification may be observed. These are the angles at which the defocus d and spherical aberration effectively cancel each other out. Axial astigmatism can be accurately corrected, for example, by exciting the astigmatism coils in the STEM column 102 to make their Ronchigram features approximately circularly symmetric. As the electron beam 114 becomes better focused on the sample S, the central low angles display progressively higher magnifications. In this position of the Ronchigram the coma-free axis is clearly defined and all alignments and positioning of the detector and aperture can be beneficially performed relative to that axial spot. The direct observability of the coma-free axis in the series of Ronchigram images 210 is an important advantage of Ronchigram alignment compared to some other alignment methods where, for example, the current or voltage center of the objective lens is used as a surrogate reference, although this type of reference is typically not as accurate as a directly visible one.

[0022] After the electron beam 114 is aligned with the coma-free axis, control of the intensity distribution of the beam depends on the exact setting of the electron beam lens 202 and the size of the STEM objective aperture inserted to filter out the peripheral aberrated beam, which typically corresponds to the high angles. For example, low order aberrations can be corrected by, for example, "rotating" the A1 and C1 virtual knobs of the scientific instrument 100 based on their manifestation in the Ronchigram image 210 captured by the detector 180. As used herein, the term "A1 virtual knob" refers to the two-fold astigmatism (C1) of the lens aberration function χ(k). 12) component. The term "C1 virtual knob" refers to one or more settings of the electron beam lens 202 that primarily affect the defocus d. The A1 and C1 virtual knobs are typically wiggled to navigate from a detuned lens configuration to a configuration in which the associated low-order aberrations are substantially corrected. A series of Ronchigram images 210 are typically required to guide this navigation process due to inherent ambiguities in the inference of actual A1 and C1 values ​​from a single Ronchigram image 210, as described below with reference to, for example, FIG. 3A. These inherent ambiguities are manifestations of corresponding symmetries in the lens aberration function χ(k) (see also Eq. (1)). Various embodiments disclosed herein are directed to automating the navigation process through the alignment parameter space of electron beam optics, as described in more detail below, for example.

[0023] 3A is a Ronchigram chart 300 corresponding to the A1-C1 parameter space of a scientific instrument 100 according to one embodiment. The Ronchigram chart 300 includes a respective reference Ronchigram 310 for each (A1, C1) value pair that includes the i-th value of the A1 parameter and the j-th value of the C1 parameter. ij where i and j are indices. ij is the reciprocal space (k-space) Ronchigram, which is related to the corresponding x-space Ronchigram image 210 captured by the detector 180 via a Fourier transform.

[0024] Chart 300, Ronchigram 310 00 corresponds to a target alignment state of the scientific instrument 100, and the associated low order aberrations are substantially corrected. ij The inherent ambiguity in inferring actual A1 and C1 parameter values ​​from, for example, the Ronchigram 310 in the chart 300. 8,-10 , 310 10,-8 and 310 9,-9It is clear from a comparison of these individual Ronchigrams that they are very similar to one another, even though each of them corresponds to a different respective pair of (A1, C1) values. Thus, the Ronchigram 310 8,-10 , 310 10,-8 , and 310 9,-9 Even if one obtains a single Ronchigram that looks like either (A1, C1), it is difficult to unambiguously infer the corresponding actual (A1, C1) parameter value pair. However, without knowing the current location of the instrument in the A1-C1 parameter space, even the sign of the required A1 and / or C1 parameter adjustment may not be unambiguously determinable in some cases.

[0025] Various embodiments disclosed herein address at least some of the above-indicated problems in the state of the art using machine learning techniques. In a representative example, a deep learning architecture is provided to guide the instrument alignment process through an alignment parameter space (illustrated by chart 300 in FIG. 3A ) by interpreting Ronchigram alignment paths through the alignment parameter space of a scientific instrument 100 as sentences in an “alignment” language. Words of this alignment language are represented by Ronchigrams, and grammatically correct sentences of this alignment language are represented by Ronchigrams 310 in chart 300. 00 The Ronchigram sequence that leads to a target alignment state of the scientific instrument 100, such as a state corresponding to (i) a reference Ronchigram for a number of known alignment parameter space settings, (ii) training an AI agent to align the digital twin towards the target alignment state, and (iii) deployment of the trained AI agent with the electronic controller 150 of the scientific instrument 100 for automated control of the Ronchigram alignment process.

[0026] As used herein, the term "digital twin" refers to a computer model of a scientific instrument 100 that is configured to output a calculated Ronchigram in response to the input of any valid set of alignment parameter values. For the embodiment illustrated in FIG. 3A, the digital twin outputs a respective Ronchigram 310 in response to receiving as input a valid (A1, C1) parameter value pair consisting of the i-th value of the A1 parameter and the j-th value of the C1 parameter. ij For different embodiments of the STEM column tuning circuitry, the digital twin of the scientific instrument 100 can be configured to calculate Ronchigrams for other virtual knob combinations, e.g., combinations that control alignment parameters other than the A1 and / or C1 parameters, and / or combinations that have a different number of virtual knobs (from two) in the knob combinations used to tune the STEM column.

[0027] 3B-3C show Ronchigram charts 301 and 302, respectively, corresponding to the A1-C1 parameter space of a scientific instrument 100 according to another embodiment. Both Ronchigram charts 301 and 302 are generated using a digital twin of the scientific instrument 100. The Ronchigram chart 301 (FIG. 3B) is a Ronchigram image 320. ij , each of which is similar to the Ronchigram image 210. The Ronchigram chart 302 (FIG. 3C) includes a reciprocal space (k-space) Ronchigram 330. ij , each of which is a Ronchigram image 320 shown in FIG. ij In each of the Ronchigram charts 301 and 302, the horizontal axis represents the C1 parameter and the vertical axis represents the A1 parameter. 00 and reciprocal space Ronchigram 330 00 represents the target alignment state of the scientific instrument 100.

[0028] Some embodiments disclosed herein use a Ronchigram image to perform Ronchigram alignment. The Ronchigram image used in these embodiments may be the measured Ronchigram image 210 (FIG. 2) or the digital twin Ronchigram image 320. ij (FIG. 3B). Some other embodiments disclosed herein use a reciprocal space Ronchigram to perform the Ronchigram alignment. The reciprocal space Ronchigram used in these embodiments is reciprocal space Ronchigram 310. ij (FIG. 3A) or reciprocal space Ronchigram 330 ij (Figure 3C).

[0029] 4A-4H graphically illustrate the performance of a digital twin of a scientific instrument 100, according to some embodiments. More specifically, each of FIGS. 4A-4H shows a respective Ronchigram 310 for the indicated index values. ij Ronchigram 310 ij are presented in pairs (grouped as Figs. 4A and 4B, 4C and 4D, 4E and 4F, and 4G and 4H, respectively), with each pair being accompanied by a corresponding experimentally measured digital twin Ronchigram 310 for ease of comparison between the two. ij The experimentally measured Ronchigrams are shown in Figures 4A, 4C, 4E, and 4G. The corresponding digital twin Ronchigrams are shown in Figures 4B, 4D, 4F, and 4H, respectively. Comparison of the Ronchigrams within each pair shows the experimentally measured Ronchigrams and the digital twin Ronchigrams 310. ij This reveals some discrepancies between the Ronchigrams and the major and minor axes, which are visually indicated by differences in the fitting of the major and minor axes for some Ronchigram pairs.

[0030] In a different embodiment, an experimentally measured Ronchigram 310 ij Or Digital Twin Ronchigram 310 ij An AI agent can be trained against the experimentally measured Ronchigram 310 ijGenerally speaking, the Digital Twin Ronchigram 310 ij This provides more accurate training of AI agents than the Ronchigram 310. However, the improvement in accuracy is not as good as the Ronchigram 310. ij On the other hand, the cost of measuring the number of experiments is the time required. ij Some loss of accuracy due to the use of a relatively large number of digital twin Ronchigrams 310 may be justified when a relatively accurate digital twin of the scientific instrument 100 is available. ij The presence of the inconsistencies illustrated in FIGS. 4A-4H may be offset by the speed and ease with which the digital twin Ronchigram 310 can be generated. ij Although such a digital twin Ronchigram may reduce the convergence rate of an AI agent trained on a target state, such a digital twin Ronchigram is not a suitable experimental Ronchigram for the convergence to occur. ij Therefore, depending on the cost-benefit analysis applicable to different use cases, the experimentally measured Ronchigram 310 ij Or Digital Twin Ronchigram 310 ij Either of these could be used to train the AI ​​agent. In some cases, the Digital Twin Ranchigram 310 ij The use of may be preferred for training AI agents.

[0031] 5 is a flow chart illustrating a method 500 of training an AI agent to perform Ronchigram alignment on a scientific instrument 100, according to some embodiments. For illustrative purposes, and without any implied limitation, the method 500 is described with reference to the exemplary A1-C1 alignment parameter space described above. Based on the description provided, one skilled in the art will be able to create and use other embodiments of the method 500 that correspond to other alignment parameter spaces.

[0032] The operation of block 502 of method 500 involves generating a plurality of reference Ronchigrams 310 that adequately sample an alignment parameter space, such as the A1-C1 alignment parameter space (FIGS. 3A-3C). ij In some embodiments, the plurality of reference Ronchigrams 310 provided in block 502 may include ij In some other embodiments, the plurality of Ronchigrams 310 provided in block 502 may include experimentally measured Ronchigrams. ij includes a digital twin Ronchigram. The granularity of the parameter space sampling by the reference Ronchigram is a selectable configuration parameter of the method 500. In different embodiments, the sampling granularity of the A1 and C1 dimensions of the alignment parameter space can be the same or different. In some embodiments, the sampling granularity for a given dimension of the alignment parameter space can be non-uniform. In some such embodiments, a relatively fine sampling granularity is used to sample the target state 310 of the scientific instrument 100. 00 , and a relatively coarse sampling granularity is used near the target state 310 of the scientific instrument 100. 00 Used far away from.

[0033] The operations of block 504 of method 500 may include reordering the Ronchigrams 310 provided to reflect the fact that substantially similar Ronchigrams may represent different locations in the alignment parameter space. ij In one embodiment, the act of block 504 includes giving each set of such substantially similar Ronchigrams a different respective label.

[0034] 6 is a Ronchigram chart 600 illustrating the labeling operation of block 504 according to some embodiments of method 500. Chart 600 is generally similar to chart 300 (FIG. 3A), but for simplicity, nine reciprocal space (k-space) Ronchigrams 610 are shown. k(where k=0, 1, ..., 8). The Ronchigram 6100 represents the target alignment state of the scientific instrument 100 (310 in FIG. 3A). 00 6104 and 6106 are substantially similar to each other and are given the label "1". Ronchigrams 6101 and 6105 are substantially similar to each other and are given the label "2". Ronchigrams 6103 and 6107 are substantially similar to each other and are given the label "3". Ronchigrams 6104 and 6108 are substantially similar to each other and are given the label "4".

[0035] The operation of block 506 of the method 500 generates the labeled Ronchigram 310 produced by the labeling operation of block 504. ij The resulting embedding is a labeled Ronchigram 310. ij In some embodiments, the corresponding vectors are real-valued and represent the labeled Ronchigram 310 in the alignment parameter space. ij , and such meaning is constructed to encode the “meaning” of the target state 310 of the scientific instrument 100. 00 In other words, labeled Ronchigrams 310 that have embeddings relatively close to each other in vector space ij is the target state 310 in the chart 300. 00, are expected to have the same degree of alignment with respect to A. In various embodiments, the operations of block 506 may be implemented using one or more of neural networks, dimensionality reduction, probabilistic models, explainable knowledge-based methods, and / or other methods typically used for these purposes in natural language processing (NLP) models. In some other embodiments, the embedding does not perfectly reflect the A / C parameters or preserve their distance in the original parameter space. In such embodiments, the Ronchigrams are encoded in a way that reflects their FFT shape, since there are multiple combinations of A-C that lead to (substantially) the same Ronchigram. The encoder operates to respect this symmetry and relabel the corresponding Ronchigrams accordingly.

[0036] The operation of block 508 of the method 500 includes (i) generating a number of different alignment sequences, each of which corresponds to a target state 310 of the scientific instrument 100. 00or 6100; and (ii) constructing a training text corpus by generating a set of alignment sequences, which leads to Ronchigram 6101 or 6102; and (ii) representing each alignment sequence in a corresponding tokenized sentence. In FIG. 6, an exemplary alignment sequence of such a training text corpus is illustrated by arrows 612 and 614. The initial alignment state of the sequence in this example corresponds to Ronchigram 6101. Arrow 612 represents a "rotation" of the C1 virtual knob of scientific instrument 100 that transitions the instrument from the initial alignment state to an intermediate alignment state corresponding to Ronchigram 6102. Arrow 614 represents a "rotation" of the A1 virtual knob of scientific instrument 100 that transitions the instrument from the intermediate alignment state to a target alignment state corresponding to Ronchigram 6100. The tokenized sentence representing this exemplary alignment sequence is "X1X2X0", where X1, X2, and X0 are tokens corresponding to labeled Ronchigrams 6101, 6102, and 6100, respectively. In the illustrated embodiment, the alignment paths (612, 614) are linear in parameter space, with segments oriented parallel to one of the parameter space axes, such that each segment of the alignment path represents the rotation of a single knob. In other embodiments, various of the alignment path segments may represent the rotation of a single knob or the simultaneous rotation of two knobs. In such embodiments, the alignment path is still linear, but its segments are not necessarily parallel to one of the parameter space axes.

[0037] In some embodiments, the training text corpus constructed in block 508 includes tokenized sentences of various lengths, (i) each tokenized sentence ends with the token X0, and (ii) any two adjacent tokens of a tokenized sentence are connected by a one dimensional (1D) alignment action. As used herein, the term "1D alignment action" refers to an alignment action in which a single virtual knob of the scientific instrument 100 is rotated. In the embodiment illustrated in FIG. 6, each of the arrows 612 and 614 represents a respective 1D alignment action. For the arrow 612, the corresponding 1D alignment action includes rotating the C1 virtual knob but does not include rotating the A1 virtual knob. For the arrow 614, the corresponding 1D alignment action includes rotating the A1 virtual knob but does not include rotating the C1 virtual knob. The path through the A1-C1 alignment parameter space represented by the arrows 612, 614 is linear. In general, any two adjacent tokens of a tokenized sentence correspond to opposite ends of respective linear segments of a straight line path in the parameter space. Some of the linear segments are parallel to the first (e.g., A1) dimension of the parameter space. Some other linear segments are parallel to the second (e.g., C1) dimension of the alignment parameter space. In some other embodiments, the AI ​​agent is configured to implement some of the tuning steps by adjusting both the A1 and C1 knobs in a 2D tuning action.

[0038] The operation of block 510 of method 500 includes training an autoregressive masked language model on the training text corpus constructed in block 508. In an exemplary embodiment, the training and subsequent use of the autoregressive masked language model is performed using an AI agent, which is described in more detail below with reference to Figures 7 and 8. After training of the model is complete, the AI ​​agent can operate to predict and select a next alignment state of the scientific instrument 100 based on a preceding sequence of measured Ronchigrams. Progressive adjustment of the scientific instrument 100 through such a sequence of next states will eventually lead to a target alignment state corresponding to, for example, Ronchigram 6100 (Figure 6).

[0039] FIG. 7 is a block diagram illustrating a transformer 702 used to perform Ronchigram alignment in a scientific instrument 100, according to some embodiments. In the illustrated embodiment, the transformer 702 is part of a training circuit 700 used to train the transformer 702 according to method 500. In the training circuit 700, the transformer 702 is configured to receive a reference sentence 704 selected from a text corpus constructed in block 508 of method 500, for example as described above with reference to FIGS. 5 and 6. In operation, the selected reference sentence 704 is applied to an input pre-processing module 710 of the transformer 702. In one embodiment, the input pre-processing module 710 performs embedding and positional encoding operations on the received reference sentence 704. An output 712 of the input pre-processing module 710 is a representation (x1,...,x n ), the encoder is a sequence of representations (x1,...,x n ) represents the sequence 712 of z = (z1, ..., z noperates to map to the sequence 722 of (). In various embodiments, the encoder 720 is implemented as a single encoder or a stack of encoders, and each such encoder includes two respective sub-layers. The first sub-layer of the two sub-layers includes a respective multi-head self-attention mechanism for the input vector. The second sub-layer of the two sub-layers includes a respective positionally fully connected feed-forward neural network. In other embodiments, other suitable transformer configurations can also be used.

[0040] Given z, the decoder 740 operates to generate an output vector 742 having a sequence of symbols (y1,..., y m ) one element at a time. In some embodiments, the decoder 740 is implemented using a stack of decoders, and each decoder in the stack includes three respective sub-layers. The first sub-layer of the three sub-layers is a respective masked multi-head self-attention mechanism for the output vector of the previous iteration. The second sub-layer of the three sub-layers is a respective multi-head attention mechanism for the output 722 of the encoder 720 and the output of the first sub-layer. The third sub-layer of the three sub-layers is a respective positionally fully connected feed-forward neural network.

[0041] The output vector 742 of the decoder 740 is applied to an output post-processing module 750. In an exemplary embodiment, the output post-processing module 750 includes a linear layer followed by a softmax layer. The linear layer is typically implemented using a fully connected neural network that projects the output vector 742 into a larger vector, often referred to as a logit vector. The softmax layer then operates to convert the logit vector into a set of probabilities (all positively valued and summing to 1). The cell with the highest probability is selected and the token associated with that cell is generated as the output 752 for that time step. The tokens in the output 752 are returned to block 706 via a return loop 754, where a sequence of such tokens is<end of sentence> A symbol is generated by placing each next token 752 to the right of the corresponding previous token 752 until a symbol is generated by the decoder 740. Each incomplete sentence of block 706 is applied to an output pre-processing module 730 to begin the next decoding iteration. The output pre-processing module 730 is functionally similar to the input pre-processing module 710, and performs embedding and positional encoding operations on the sentences 706.

[0042] <end of sentence> After symbols are generated, the completed sentences of block 706 are compared to the corresponding original sentences 704 in a loss function module 708. In some embodiments, the loss function module 708 guides the training of the Transformer 702 based on cross-entropy and Kullback-Leibler divergence. When the completed output sentences 706 do not successfully reconstruct the input sentences 704, the loss function module 708 imposes a corresponding penalty. Various parameters of the neural network used in the Transformer 702 are iteratively adjusted until the penalty imposed by the loss function module 708 on any of the reference sentences 704 from the text corpus constructed in block 508 of method 500 is less than a selected threshold.

[0043] 8 is a block diagram illustrating an optical alignment circuit 800 used in the electronic controller 150 of the scientific instrument 100, according to some embodiments. The optical alignment circuit 800 is configured to connect the (trained) transformer 702 to the electron beam optics of the STEM column 102 and the two-dimensional pixelated detector 180, for example as shown in FIG. 8. It should be noted that the optical alignment circuit 800 allows various components thereof to be inserted into the return loop 754 (see also FIG. 7) of the transformer 702, allowing the latter to drive the Ronchigram alignment process of the scientific instrument 100, for example as described in more detail below.

[0044] In operation, the two-dimensional pixelated detector 180 captures a Ronchigram 210 as described above with reference to Figure 2. The detector 180 then transmits the captured Ronchigram 210 to a Fourier Transform (FT) module 810 of the circuit 800, for example via a stream of measurements 182. The FT module 810 performs an inverse Fourier transform on the received Ronchigram 210 to generate a corresponding reciprocal space (k-space) Ronchigram 310, which is then applied to the mapper 820. In some embodiments, the Fourier transform operation is omitted.

[0045] It should be noted that the Ronchigram 310 applied to the mapper 820 belongs to a continuous space of Ronchigrams, and the experimentally measured Ronchigram 210 pattern may generally vary smoothly. However, the transformer 702 uses a Ronchigram 310 that discretely samples the continuous space of Ronchigrams, as explained above. ij Thus, the mapper 820 converts a Ronchigram from the continuous space of Ronchigrams into the Ronchigram 310 that the transformer 702 trained via method 500. ij It works by mapping onto a finite set of

[0046] In one embodiment, the mapper 820 maps any continuous spatial Ronchigram 310 to its closest corresponding finite set Ronchigram 310. ij In some embodiments, the conversion process includes two sets of operations: During a first set of operations, an encoder portion of the variational autoencoder maps the continuous-space Ronchigram 310 onto a corresponding latent representation. During a second set of operations, a decoder portion of the variational autoencoder maps the latent representation to a reference Ronchigram 310. ij The mapper 820 then projects the reference Ronchigram 310 thus selected by the variational autoencoder onto the corresponding one of ij 7. The output token 752' is input to block 706, as described above with reference to Figure 7. A copy of the token 752' is also provided to a delta (Δ) circuit 830.

[0047] In some embodiments, the functionality of both the FT module 810 and the mapper 820 can be implemented using (differently trained) variational autoencoders. In such embodiments, the encoder portion of the variational autoencoder is trained to map the Ronchigram 210 onto a corresponding latent representation, with or without performing a Fourier transform as an explicit processing step. The decoder portion of the variational autoencoder maps the latent representation to the reference Ronchigram 310. ij , the vectors are trained to project onto a corresponding one of

[0048] The transformer 706 then operates to process the current incomplete sentence 706 to output a next token 752 for the sentence, e.g., as described above with reference to FIG. 7. The delta circuit 830 then operates to process the tokens 752 and 752′ to determine a parameter difference 832 between the tokens in an alignment parameter space, e.g., the (A1, C1) alignment parameter space illustrated in FIGS. 3A-3C. In some embodiments corresponding to the parameter space of FIG. 3A, the parameter difference 832 has a non-zero A1 component. In some other embodiments corresponding to the parameter space of FIG. 3A, the parameter difference 832 has a non-zero C1 component. The determined parameter difference 832 is used to convert the received parameter difference 832 into one or more corresponding adjustments 8421-842 for a set of control signals applied by the electronic controller 150 to the electron beam optics of the STEM column 102. N In the embodiment illustrated in FIG. 1, one or more adjustments 8421-842 are applied to a knob rotation circuit 840 configured to convert the adjustments into a corresponding set of N can be used to modify one or both of the control signals 152 and 156. Adjusting the control signals appropriately changes the alignment state of the electron beam optics of the STEM column 102. The latter change then appears in the subsequent Ronchigram 210 captured by the detector 180.

[0049] 9 is a flow chart of an automated method 900 for aligning electron beam optics in a scientific instrument 100, according to some embodiments. In some examples, the method 900 is implemented using the optical alignment circuitry 800. The method 900 is described below with continued reference to FIGS. 2 and 7-9.

[0050] The operations of block 902 of method 900 include operating scientific instrument 100 to measure a Ronchigram 210 corresponding to a current alignment state of electron beam optics of STEM column 102. An example of such a measurement is described above with reference to, for example, FIG. 2. The operations of block 904 of method 900 include transforming the measured Ronchigram 210 into a corresponding token 752′ from a plurality of reference tokens on which transformer 702 was trained. In some embodiments, the transformation operation of block 904 is performed using FT module 810 and mapper 820 (see also FIG. 8). The operations of block 906 of method 900 include operating transformer 702 to process a current incomplete sentence 706 and output a next token 752 of the sentence, for example as described above with reference to FIG. 7.

[0051] The operations of decision block 908 include determining whether a target alignment state for the scientific instrument 100 has been reached. In one embodiment, the target alignment state is:<end of sentence> Transformer 702 generates the symbol 904. When it is determined that the target alignment state has been reached ("YES" at decision block 908), the operations of method 900 end. When it is determined that the target alignment state has not been reached ("NO" at decision block 908), method 900 proceeds to perform the operations of block 910.

[0052] The operation of block 910 includes adjusting one or more control signals applied to electron beam optics in STEM column 102 based on tokens 752 and 752′. In some embodiments, the adjustment of the one or more control signals is performed using delta circuit 830 and knob rotation circuit 840 (see also FIG. 8 ). Once the operation of block 910 is completed, method 900 loops back to block 902.

[0053] 10 is a block diagram of an example computing device 1000 configured to perform at least some scientific instrument support operations, according to various embodiments. For example, in some embodiments, computing device 1000 performs at least some operations of electronic controller 150 (see also FIG. 1). In various embodiments, the support modules of scientific instrument 100 may be implemented by a single computing device 1000 or by multiple computing devices 1000.

[0054] Although the computing device 1000 of FIG. 10 is illustrated as having several components, any one or more of these components may be omitted or duplicated as appropriate for the application and configuration. In some embodiments, some or all of the components included in the computing device 1000 may be mounted on one or more motherboards and enclosed in a housing. In some embodiments, some of these components may be fabricated on a single system-on-a-chip (SoC) (e.g., an SoC may include one or more processing devices 1002 and one or more storage devices 1004). Additionally, in various embodiments, computing device 1000 may not include one or more of the components illustrated in FIG. 10, but may include interface circuitry for coupling to one or more components using any suitable interface (e.g., a Universal Serial Bus (USB) interface, a High-Definition Multimedia Interface (HDMI) interface, a Controller Area Network (CAN) interface, a Serial Peripheral Interface (SPI) interface, an Ethernet interface, a wireless interface, or any other suitable interface). For example, computing device 1000 may not include a display device 1010, but may include display device interface circuitry (e.g., connectors and driver circuits) to which an external display device 1010 may be coupled.

[0055] Computing device 1000 may include a processing device 1002 (e.g., one or more processing devices). As used herein, the term "processing device" refers to any device or portion of a device that processes electronic data from registers and / or memory and converts the electronic data into other electronic data that may be stored in registers and / or memory. In various embodiments, processing device 1002 may be one or more digital signal processors (DSPs), application-specific integrated circuits (ASICs), or other integrated circuits (ICs). integrated circuit, ASIC, central processing unit The processing device may include a CPU, a graphics processing unit (GPU), a server processor, or any other suitable processing device.

[0056] The computing device 1000 also includes a storage device 1004 (e.g., one or more storage devices). In various embodiments, the storage device 1004 may be a random-access memory (RAM) device (e.g., static RAM, The storage device 1004 may include one or more memory devices, such as static random access memory (SRAM) devices, magnetic RAM (MRAM) devices, dynamic RAM (DRAM) devices, resistive RAM (RRAM) devices, or conductive-bridging RAM (CBRAM) devices, hard-drive-based memory devices, solid-state memory devices, network drives, cloud drives, or any combination of memory devices. In some embodiments, the storage device 1004 may include memory that shares a die with the processing device 1002. In such embodiments, the memory may be used as a cache memory and may include, for example, embedded dynamic random-access memory (eDRAM) or spin transfer torque magnetic random-access memory (STT-MRAM). In some embodiments, the storage device 1004 may include a non-transitory computer-readable medium having instructions that, when executed by one or more processing devices (e.g., the processing device 1002), cause the computing device 1000 to perform any suitable of the methods, or portions of such methods, disclosed herein below.

[0057] The computing device 1000 further includes an interface device 1006 (e.g., one or more interface devices 1006). In various embodiments, the interface device 1006 may include one or more communication chips, connectors, and / or other hardware and software to manage communications between the computing device 1000 and other computing devices. For example, the interface device 1006 may include circuitry to manage wireless communications to transfer data to and from the computing device 1000. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communication channels, etc. that may communicate data via modulated electromagnetic radiation over a non-solid medium. This term does not imply that the associated devices do not include any wiring, although in some embodiments they may not. The circuitry included in interface device 1006 for managing wireless communications may implement any of a number of wireless standards or protocols, including, but not limited to, Wi-Fi (IEEE 802.11 family), Institute for Electrical and Electronic Engineers (IEEE) standards including the IEEE 802.16 standard, the Long-Term Evolution (LTE) project (e.g., the Advanced LTE project, the Ultramobile Broadband (UMB) project (also referred to as “3GPP®2”), etc.) with any amendments, updates, and / or revisions.In some embodiments, the circuitry included in the interface device 1006 for managing wireless communications includes circuitry for Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Evolved HSPA (Evolved HSPA), and other standard radio services. In some embodiments, the circuitry included in the interface device 1006 for managing wireless communications may operate in accordance with a GSM HSPA, E-HSPA, or LTE network. In some embodiments, the circuitry included in the interface device 1006 for managing wireless communications may operate in accordance with an Enhanced Data for GSM Evolution (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). In some embodiments, the circuitry included in the interface device 1006 for managing wireless communications may operate in accordance with a Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), or LTE network. Division Multiple Access (TDMA), Digital Enhanced Cordless Communications (Digital Enhanced Cordless Telecommunications (DECT), Evolution-Data Optimization EV-DO (Echo Optimized, EV-DO), and derivatives thereof, as well as any other wireless protocols designated as 3G, 4G, 5G, and beyond. In some embodiments, the interfacing device 1006 may include one or more antennas (e.g., one or more antenna arrays) configured to receive and / or transmit wireless signals.

[0058] In some embodiments, the interface device 1006 may include circuitry for managing wired communications, such as electrical, optical, or any other suitable communication protocol. For example, the interface device 1006 may include circuitry supporting communications according to Ethernet technology. In some embodiments, the interface device 1006 may support both wireless and wired communications, and / or multiple wired and / or wireless communications protocols. For example, a first set of circuits of the interface device 1006 may be dedicated to short-range wireless communications, such as Wi-Fi or Bluetooth, and a second set of circuits of the interface device 1006 may be dedicated to long-range wireless communications, such as global positioning system (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, or others. In some other embodiments, the first set of circuits of the interface device 1006 may be dedicated to wireless communications, and the second set of circuits of the interface device 1006 may be dedicated to wired communications.

[0059] Computing device 1000 also includes battery / power circuitry 1008. In various embodiments, battery / power circuitry 1008 may include one or more energy storage devices (e.g., batteries or capacitors) and / or circuitry for coupling components of computing device 1000 to an energy source separate from computing device 1000 (e.g., to an AC line power source).

[0060] Computing device 1000 also includes a display device 1010 (e.g., one or more separate display devices). In various embodiments, display device 1010 may include any visual indicator, such as a head-up display, a computer monitor, a projector, a touch screen display, a liquid crystal display (LCD), a light emitting diode display, or a flat panel display.

[0061] Computing device 1000 also includes additional input / output (I / O) devices 1012. In various embodiments, I / O devices 1012 may include one or more data / signal transfer interfaces, audio I / O devices (e.g., a microphone or microphone array, a speaker, a headset, an earphone, an alarm, etc.), an audio codec, a video codec, a printer, sensors (e.g., a thermocouple or other temperature sensor, a humidity sensor, a pressure sensor, a vibration sensor, etc.), image capture devices (e.g., one or more cameras), human interface devices (e.g., a cursor control device such as a keyboard, a mouse, a stylus, a trackball, or a touchpad), etc.

[0062] Depending on the particular embodiment of the scientific instrument 100 and / or instrument portion 200, various components of the interface device 1006 and / or I / O device 1012 may be configured to output suitable control signals (e.g., 152, 154, 156) for the various components of the scientific instrument 100, receive suitable control signals / telemetry signals from the various components of the scientific instrument 100, and receive streams of measurements (e.g., 162, 172, 182) from the various detectors of the scientific instrument 100. In some examples, the interface device 1006 and / or the I / O device 1012 include one or more analog-to-digital converters (ADCs) for converting received analog signals into a digital form suitable for operations performed by the processing device 1002 and / or the storage device 1004. In some additional embodiments, the interface device 1006 and / or the I / O device 1012 include one or more digital-to-analog converters (DACs) for converting digital signals provided by the processing device 1002 and / or the storage device 1004 into an analog form suitable for communication to corresponding components of the scientific instrument 100.

[0063] According to one embodiment disclosed above, for example in the Overview section and / or with reference to any one or any combination of some or all of Figures 1-10, there is provided a support apparatus for a scientific instrument, comprising: an interface device configured to receive data representing a Ronchigram acquired using an electron microscope column of the scientific instrument and further configured to transmit one or more control signals for the electron beam optics of the electron microscope column; and a processing device configured to convert the measured Ronchigram into input tokens for a transformer, generate output tokens using the transformer based on tokenized sentences ending with the input tokens, and determine one or more adjustments to the one or more control signals based on the input tokens and the output tokens, wherein the input tokens and the output tokens belong to a plurality of tokens representing a plurality of reference Ronchigrams sampling an alignment parameter space of the electron beam optics, and the transformer implements an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a respective path through the alignment parameter space to a target alignment state of the electron beam optics. As used herein, the term "Ronchigram" should be interpreted to encompass Ronchigram images (ie, x-space Ronchigrams) and reciprocal space (k-space) Ronchigrams.

[0064] In some embodiments of the above support apparatus, the plurality of reference Ronchigrams includes one or more Ronchigrams generated via a computer model of the scientific instrument.

[0065] In some embodiments of any of the above support apparatus, the plurality of reference Ronchigrams includes one or more Ronchigrams obtained using an electron microscope column of a scientific instrument.

[0066] In some embodiments of any of the above assistance apparatus, the processing device is configured to iteratively update the tokenized sentence by appending to the tokenized sentence input tokens representing measured Ronchigrams obtained after a last adjustment of the one or more control signals.

[0067] In some embodiments of any of the above aids, different dimensions of the alignment parameter space represent different respective components of the geometric aberrations.

[0068] In some embodiments of any of the above aids, each respective path is linear in the alignment parameter space.

[0069] In some implementations of any of the above aids, any two adjacent tokens of a tokenized sentence correspond to opposite ends of respective linear segments of respective paths through the alignment parameter space.

[0070] In some embodiments of any of the above support devices, each linear segment is parallel to a first dimension of the alignment parameter space representing two-fold astigmatism or parallel to a second dimension of the alignment parameter space representing defocus.

[0071] In some embodiments of any of the above assistive devices, the one or more adjustments are determined based on the length of each linear segment in the alignment parameter space.

[0072] In some embodiments of any of the above support apparatus, the processing device is configured to convert the measured Ronchigram into an input token by applying an inverse Fourier transform to the measured Ronchigram to calculate a corresponding reciprocal space Ronchigram, mapping the corresponding reciprocal space Ronchigram to a corresponding one of a plurality of reference Ronchigrams, and providing a token representing the corresponding one of the plurality of reference Ronchigrams in the plurality of tokens as an input token.

[0073] In some embodiments of any of the above assistance devices, the transformer comprises an encoder including a first multi-head self-attention mechanism and a first positionally fully connected feedforward neural network, and a decoder including a second multi-head self-attention mechanism, an output of the multi-head attention mechanism and the second multi-head self-attention mechanism on an output of the encoder, and a second fully connected feedforward neural network.

[0074] According to another embodiment disclosed above, for example in the Summary section and / or with reference to any one or any combination of some or all of Figures 1-10, an automated method implemented via a computing device for providing assistance to a scientific instrument is provided, the automated method including: converting a measured Ronchigram into input tokens for a transformer, where the measured Ronchigram is received via data acquired using an electron microscope column of the scientific instrument; generating output tokens using the transformer based on tokenized sentences ending with the input tokens; and determining one or more adjustments to one or more control signals for the electron beam optics of the electron microscope column based on the input tokens and the output tokens, the input tokens and the output tokens belonging to a plurality of tokens representing a plurality of reference Ronchigrams sampling an alignment parameter space of the electron beam optics, the transformer implementing an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a respective path through the alignment parameter space to a target alignment state of the electron beam optics.

[0075] In some embodiments of the above automated methods, the automated methods further include iteratively updating the tokenized sentences by appending to the tokenized sentences input tokens representing measured Ronchigrams obtained after a last adjustment of the one or more control signals.

[0076] In some embodiments of any of the above automated methods, different dimensions of the alignment parameter space represent different respective components of the geometric aberrations.

[0077] In some embodiments of any of the above automated methods, each respective path is linear in the alignment parameter space.

[0078] In some implementations of any of the automated methods described above, any two adjacent tokens of a tokenized sentence correspond to opposite ends of respective linear segments of respective paths through the alignment parameter space.

[0079] In some embodiments of any of the above automated methods, each linear segment is parallel to a first dimension of an alignment parameter space representing two-fold astigmatism or parallel to a second dimension of an alignment parameter space representing defocus.

[0080] In some embodiments of any of the above assistive devices, the one or more adjustments are determined based on the length of each linear segment in the alignment parameter space.

[0081] In some embodiments of any of the above automated methods, the transforming includes applying an inverse Fourier transform to the measured Ronchigram to calculate a corresponding reciprocal space Ronchigram, mapping the corresponding reciprocal space Ronchigram to a corresponding one of the plurality of reference Ronchigrams, and providing a token representing the corresponding one of the plurality of reference Ronchigrams in the plurality of tokens as an input token.

[0082] In accordance with yet another embodiment disclosed above, for example in the Summary section and / or with reference to any one or any combination of some or all of Figures 1-10, a non-transitory computer-readable medium is provided that stores instructions that, when executed by a computing device, cause the computing device to perform operations including any of the automated methods described above.

[0083] It should be understood that the above description is intended to be illustrative and not restrictive. Many implementations and applications other than the examples provided will become apparent upon reading the above description. The scope should not be determined with reference to the above description, but instead with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that future developments will occur in the art discussed herein, and that the disclosed systems and methods will be incorporated into such future examples. In short, it should be understood that this application is capable of modification and variation.

[0084] All terms used in the claims are intended to be given their broadest reasonable interpretation and their ordinary meaning as understood by one skilled in the art described herein, unless expressly indicated to the contrary herein. In particular, the use of singular articles such as "a," "the," "said," etc., should be read to recite one or more of the indicated elements, unless the claims set forth an express limitation to the contrary.

[0085] The Abstract is provided to enable the reader to quickly ascertain the nature of the technical disclosure. The Abstract is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the above Detailed Description, it may be seen that various features have been grouped together in various examples for the purpose of streamlining the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the claimed subject matter incorporates more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed example. Accordingly, the following claims are incorporated into the Detailed Description with each claim standing on its own as independently claimed subject matter.

[0086] Unless expressly stated otherwise, each numerical value and range should be construed as being approximate, as if the word "about" or "approximately" were preceding that value or range.

[0087] In the method claims that follow, elements, if any, are recited in a particular order with corresponding labeling, but unless the claim description otherwise suggests a particular order for implementing some or all of the elements, the elements are not necessarily intended to be limited to being implemented in that particular order.

[0088] Unless otherwise specified herein, the use of ordinal adjectives such as "first," "second," "third," etc. to refer to one object of a plurality of similar objects is intended only to indicate that different instances of such similar objects are being referred to, and is not intended to imply that the similar objects so referred to must be in a corresponding order or sequence, either in time, space, ranking, or in any other manner.

[0089] Unless otherwise specified herein, the conjunction "if" shall mean, in addition to its plain meaning, "when" or "upon" or "in response to determining" or "in response to detecting." "in response to detecting," which interpretation may depend on the specific context. For example, the phrase "when determined" or "when [a stated condition] is detected" may be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]."

[0090] Also, for purposes of this description, the terms "couple," "coupling," "coupled," "connected," "connecting," or "connected" refer to any manner known in the art or later developed that allows energy to be transferred between two or more elements, where the intervening presence of one or more additional elements is contemplated, but not required. Conversely, the terms "directly coupled," "directly connected," and the like, imply the absence of such additional elements.

[0091] The functionality of the various elements shown in the figures, including any functional blocks labeled "processor" and / or "controller," may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functionality may be provided by a single dedicated processor, by a single shared processor, or by multiple individual processors, some of which may be shared. Furthermore, explicit use of the terms "processor" or "controller" should not be construed as referring exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), read only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional and / or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Those functions may be performed through operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, with the particular technique being selectable by the implementer, as more particularly understood from the context.

[0092] As used in this application, the term "circuit" or "circuitry" may refer to one or more or all of the following: (a) hardware-only circuit implementations (such as implementations with only analog and / or digital circuitry); (b) (where applicable), (i) combinations of analog and / or digital hardware circuitry with software / firmware; and (ii) combinations of hardware circuitry and software, such as any portion of a hardware processor with software (including digital signal processors), software, and memory that work together to cause a device such as a mobile phone or server to perform various functions; and (c) hardware circuits and / or processors, such as a microprocessor or portions of a microprocessor, that require software (e.g., firmware) to operate, but the software may be absent when not required for operation. This definition of circuitry applies to all uses of the term in this application, including any claims. As a further example, as used in this application, the term circuitry also covers merely a hardware circuit or processor (or processors), or portions of a hardware circuit or processor, and its (or their) associated software and / or firmware implementations. The term circuitry also covers, for example, baseband or processor integrated circuits for mobile devices, or similar integrated circuits in servers, cellular network devices, or other computing or network devices, where applicable to particular claim elements.

[0093] Those skilled in the art will appreciate that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the present disclosure. Similarly, any flow charts, flow diagrams, state transition diagrams, pseudocode, or the like, may be substantially represented in a computer-readable medium and may represent various processes that may be performed by such a computer or processor, whether or not a computer or processor is explicitly depicted.

Claims

1. 1. A support device for a scientific instrument, said support device comprising: an interface device configured to receive data representative of a Ronchigram acquired at an electron microscope column of the scientific instrument, the interface device being further configured to transmit one or more control signals for electron beam optics of the electron microscope column; a processing device, the processing device comprising: converting the measured Ronchigram into input tokens for a transformer; generating output tokens based on tokenized sentences ending with the input tokens using the transformer; and determining one or more adjustments to the one or more control signals based on the input tokens and the output tokens; the input tokens and the output tokens belong to a plurality of tokens representing a plurality of reference Ronchigrams sampling an alignment parameter space of the electron beam optics; The support apparatus, wherein the transformer implements an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a respective path through the alignment parameter space to a target alignment state of the electron beam optics.

2. The support apparatus of claim 1 , wherein the plurality of reference Ronchigrams includes one or more Ronchigrams generated via a computer model of the scientific instrument.

3. 2. The support apparatus of claim 1, wherein the plurality of reference Ronchigrams comprises one or more Ronchigrams acquired at the electron microscope column of the scientific instrument.

4. 2. The assistance apparatus of claim 1, wherein the processing device is configured to iteratively update the tokenized sentences by appending to the tokenized sentences the input tokens representing the measured Ronchigrams obtained after a last adjustment of the one or more control signals.

5. The apparatus of claim 1 , wherein different dimensions of the alignment parameter space represent different respective components of geometric aberrations.

6. The apparatus of claim 1 , wherein each respective path is linear in the alignment parameter space.

7. 2. The apparatus of claim 1, wherein any two adjacent tokens of the tokenized sentence correspond to opposite ends of a respective linear segment of the respective path through the alignment parameter space.

8. 8. The apparatus of claim 7, wherein each linear segment is parallel to a first dimension of the alignment parameter space representing two-fold astigmatism or parallel to a second dimension of the alignment parameter space representing defocus.

9. The apparatus of claim 8 , wherein the one or more adjustments are determined based on a length of the respective linear segment in the alignment parameter space.

10. the processing device comprising: applying an inverse Fourier transform to the measured Ronchigram to calculate a corresponding reciprocal spatial Ronchigram; mapping the corresponding reciprocal space Ronchigram to a corresponding one of the plurality of reference Ronchigrams; 2. The assistive device of claim 1, configured to convert the measured Ronchigram into the input token by: providing as the input token a token representing the corresponding one of the plurality of reference Ronchigrams in the plurality of tokens.

11. The transformer, an encoder including a first multi-head self-attention mechanism and a first positionally fully connected feedforward neural network; 2. The assistance device of claim 1, further comprising: a second multi-head self-attention mechanism; a multi-head attention mechanism on an output of the encoder and an output of the second multi-head self-attention mechanism; and a decoder including a second fully connected feedforward neural network.

12. 1. An automated method implemented via a computing device for providing assistance to a scientific instrument, the method comprising: converting a measured Ronchigram into input tokens for a transformer, the measured Ronchigram being received via data acquired using an electron microscope column of the scientific instrument; generating output tokens based on tokenized sentences ending with the input tokens using the transformer; determining one or more adjustments to one or more control signals for electron beam optics of the electron microscope column based on the input tokens and the output tokens; the input tokens and the output tokens belong to a plurality of tokens representing a plurality of reference Ronchigrams sampling an alignment parameter space of the electron beam optics; An automated method in which the transformer implements an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a respective path through the alignment parameter space to a target alignment state of the electron beam optics.

13. 13. The automated method of claim 12, further comprising iteratively updating the tokenized sentences by appending to the tokenized sentences the input tokens representing the measured Ronchigrams obtained after a last adjustment of the one or more control signals.

14. The automated method of claim 12 , wherein different dimensions of the alignment parameter space represent different respective components of geometric aberrations.

15. The automated method of claim 12 , wherein each respective path is linear in the alignment parameter space.

16. 13. The automated method of claim 12, wherein any two adjacent tokens of the tokenized sentence correspond to opposite ends of a respective linear segment of the respective path through the alignment parameter space.

17. 17. The automated method of claim 16, wherein each linear segment is parallel to a first dimension of the alignment parameter space representing two-fold astigmatism or parallel to a second dimension of the alignment parameter space representing defocus.

18. The automated method of claim 16 , wherein the one or more adjustments are determined based on a length of the respective linear segment in the alignment parameter space.

19. The converting step comprises: applying an inverse Fourier transform to the measured Ronchigram to calculate a corresponding reciprocal spatial Ronchigram; mapping the corresponding reciprocal space Ronchigram to a corresponding one of the plurality of reference Ronchigrams; and providing as the input token a token representing the corresponding one of the plurality of reference Ronchigrams in the plurality of tokens.

20. A non-transitory computer-readable medium storing instructions that, when executed by a computing device, cause the computing device to perform operations including the automated method of claim 12.

Images