Microscope aberration correction

Abstract

Scientific instrument support systems, and related methods, apparatuses, computing devices, and computer readable media are disclosed herein. In some embodiments, a support apparatus for a scientific instrument includes an interface device and a processing device. An interface device receives a Lenchi map acquired with a scientific instrument and transmits a control signal for electron beam optics of the scientific instrument. The processing device converts the measured Lengchi map into an input token for the transformer, generates an output token based on a tokenized sentence ending with the input token, and determines an adjustment to the control signal based on the input token and the output token. The input token and the output token belong to a plurality of tokens representing a reference Lengchi map that samples an alignment parameter space of the electron beam optics. The transformer implements an autoregressive masking language model trained on a corpus of reference sentences representing a path through an alignment parameter space to a target alignment state of the electron beam optics.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. non-provisional application No. 18 / 345,675, filed on June 30, 2023, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] Various exemplary embodiments relate generally, but not exclusively, to electron microscopy components, instruments, systems, and methods. Summary of the invention

[0004] The resolution of an electron microscope can be limited by imperfections in the electron beam lens. The results of such imperfections are referred to as aberrations due to their similarity to aberrations observed in optical microscopes. For example, before performing experimental measurements with a scanning transmission electron microscope (STEM), an operator typically needs to tune the electron beam lens of the microscope so that at least some of the aberrations are corrected. In some cases, such tuning needs to be repeated, for example several times a day, to counteract detuning caused by various drift processes in the circuits associated with the electron beam lens. This type of tuning is typically tedious, time consuming, and / or non-optimal when performed manually by an operator.

[0005] Various examples, aspects, features, and embodiments of a scientific instrument are disclosed herein, inter alia, that includes an electron microscope column and associated Ronchi diagram acquisition circuitry. An electronic controller of the scientific instrument employs a transformer to automatically correct at least some geometric aberrations of an electron beam optic in the electron microscope column. According to an exemplary embodiment, the transformer implements an autoregressive masked language model that is trained on a corpus of reference sentences representing various paths through the alignment parameter space of the electron beam optic to its target alignment state. A return loop of the transformer is connected to the Ronchi diagram acquisition circuitry to enable the transformer to automatically drive the alignment process of the scientific instrument toward the target alignment state based on the simultaneously acquired Ronchi diagrams, thereby advantageously eliminating the need for an operator to manually (re)align the electron beam optics.

[0006] One example provides a support device for a scientific instrument, the support device comprising: an interface device configured to receive data representing a Ronchi diagram acquired using an electron microscope column of the scientific instrument, and also configured to transmit one or more control signals for an electron beam optical device of the electron microscope column; and a processing device configured to: convert the measured Ronchi diagram into an input token for a transformer; generate an output token using the transformer based on a tokenized sentence ending with the input token; and determine one or more adjustments to the one or more control signals based on the input token and the output token, wherein the input token and the output token belong to a plurality of tokens representing a plurality of reference Ronchi diagrams sampling an alignment parameter space of the electron beam optical device; and wherein the transformer implements an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a corresponding path through the alignment parameter space to a target alignment state of the electron beam optical device.

[0007] Another example provides an automated method for providing support for a scientific instrument, performed via a computing device, the method comprising: converting a measured Ronchi diagram into an input token for a transformer, the measured Ronchi diagram received via data acquired using an electron microscope column of the scientific instrument; generating output tokens using the transformer based on a tokenized sentence ending with the input token; and determining one or more adjustments to one or more control signals of an electron beam optical device for the electron microscope column based on the input token and the output token; wherein the input token and the output token belong to a plurality of tokens, the plurality of tokens representing a plurality of reference Ronchi diagrams sampling an alignment parameter space of the electron beam optical device; and wherein the transformer implements an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a corresponding path through the alignment parameter space to a target alignment state of the electron beam optical device.

[0008] Yet another example provides a non-transitory computer-readable medium storing instructions that, when executed by a computing device, cause the computing device to perform operations including the automated method described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The foregoing aspects of the present disclosure and many of the attendant advantages will become more readily appreciated when the following detailed description is taken in conjunction with the accompanying drawings.

[0010] Figure 1 is a block diagram illustrating an exemplary scientific instrument according to some embodiments.

[0011] Figure 2 is a diagram showing the Figure 1 Block diagram for obtaining Ronchi plots in scientific instruments.

[0012] FIG. 3A to FIG. 3C is a Ronchi diagram corresponding to a two-dimensional (2D) alignment parameter space of scientific instrument 100 according to some examples.

[0013] 4A to 4H The diagram shows some examples of Figure 1 Generate digital twins of scientific instruments.

[0014] Figure 5 is a diagram illustrating training an artificial intelligence (AI) agent to Figure 1 Flowchart of a method for performing Ronchi diagram alignment in a scientific instrument.

[0015] Figure 6 is shown according to some examples Figure 5 Ronchi diagrams for some operations of the method.

[0016] Figure 7 is a diagram showing a method for performing Figure 1 Block diagram of a transformer that performs Ronchi diagram alignment in a scientific instrument.

[0017] Figure 8 is a diagram showing a combination according to some embodiments Figure 7 Block diagram of the converter's optical device alignment circuit.

[0018] Fig. 9 According to some embodiments, Figure 1 Flowchart of an automated method for aligning electron beam optics in a scientific instrument.

[0019] Fig.10 is a block diagram of an exemplary computing device configured to perform at least some scientific instrument support operations, according to various embodiments. DETAILED DESCRIPTION

[0020] An exemplary STEM alignment method includes two or more sets of alignment operations. One set of such alignment operations (sometimes referred to as Ronchi diagram alignment) involves tuning the electron beam optics based on observed Ronchi diagram patterns. Using Ronchi diagram alignment, both low-order aberrations and high-order aberrations can be corrected. In some implementations, after Ronchi diagram alignment, the focusing lens aperture is inserted, and then at least some residual aberrations are finely corrected to obtain high-quality STEM images. Disadvantageously, at least some users find it difficult to perform Ronchi diagram alignment.

[0021] In this article, the term "Ronchi pattern" refers to the diffraction pattern of a converging electron beam focused on an amorphous specimen. In mathematical terms, the lens aberration function χ(k) can be expressed by formula (1):

[0022]

[0023] where α is the convergence angle; φ is the azimuth angle; λ is the wavelength of the electron beam; in Krivanek notation, C n,m and φ n,m denotes the different components of geometric aberration; m is the degree of aberration; and n is the order of aberration. The observed Ronchi pattern is proportional to the electron probability density g(k) on the diffraction plane, which can be expressed using the transmitted wave function ψ t It can be expressed as the square of the modulus of the Fourier transform (FT) of (x), for example, as follows:

[0024] g(k)=|FT{ψ t (x)}| 2 (2)

[0025] The transmitted wave function ψ t (x) The product of the Fourier transform of the aberration function χ(k) and the amorphous specimen potential. In some models, the amorphous specimen is considered as a randomly generated noisy phase grating that adds a random phase to the electron beam, for example, under the eikonal approximation.

[0026] Important features encoded in the texture of the Ronchi pattern are magnification and symmetry. For a focused beam, the center of the Ronchi pattern has a high local magnification, which represents the coherent, essentially aberration-free part of the electron beam. Further away from the optical axis, aberrations reduce the local magnification. The presence of aberrations affects the structure and symmetry of the Ronchi pattern. For example, asymmetric aberrations (m>0) destroy the rotational symmetry. Double astigmatism (C 12 ) usually stretches the high magnification area unidirectionally and produces obvious fringes. Axial coma (C 21 ) makes the center of the Ronchi diagram shift. Triple astigmatism (C 23 ) produces a three-fold symmetric lobe, etc. When the microscope is aligned, various currents through the electron beam lens can be adjusted to minimize the various manifestations of aberrations in the Ronchi diagram. In a well-aligned microscope, low-order aberrations are almost completely corrected, resulting in large featureless areas of high local magnification, often with a characteristic sextuplet profile attributable to residual high-order aberrations.

[0027] Figure 1 1 is a block diagram illustrating a scientific instrument 100 according to some embodiments. The scientific instrument 100 includes a scanning transmission electron microscope (STEM) column 102 coupled to a vacuum chamber 108. The vacuum chamber 108 houses a movable sample holder 110 therein and can be vacuumed using one or more vacuum pumps (in Figure 1 In an exemplary embodiment, the sample holder 110 is capable of independently moving parallel to the XY coordinate plane and parallel to the Z coordinate axis, wherein the corresponding coordinate system is represented by Figure 1The sample S to be interrogated using the scientific instrument 100 is mounted in the sample holder 110, as shown in FIG. Figure 1 shown.

[0028] In the example shown, the STEM column 102 includes an electron source 112 and two or more electron beam lenses, which are shown for illustration purposes. Figure 1 Only two of them are schematically shown, namely the objective lens 106 and the condenser lens 116. In some examples, a different (other than two) number of such lenses may be used in the STEM column 102. In some embodiments, the objective lens 106 may be an ultra-high resolution (UHR) lens.

[0029] In operation, the electron source 112 generates an electron beam 114 that propagates generally along a longitudinal axis 115 of the STEM column 102. The electron beam lenses 106 and 116 are operated to generate electric and magnetic fields that affect the trajectories of electrons in the electron beam 114. Control signals 152, 156 generated by the electronic controller 150 are used to change the strength and / or spatial configuration of the fields and impart desired characteristics to the electron beam 114. In general, the electron beam lenses 106 and 116, the control signals 152 and 156, and other associated components of the scientific instrument 100 may be used to perform various operations and support various functions, such as beam focusing, aberration mitigation, aperture cropping, filtering, etc. The STEM column 102 also includes a deflection unit 118 that may direct the electron beam 114 in response to a control signal 154 applied thereto by the electronic controller 150. Such beam steering may be used to move a focused portion of the electron beam 114 along a desired path across the sample S, for example, to perform a raster or vector scan thereof.

[0030] The scientific instrument 100 also includes detectors 160, 170, 180 located in the vacuum chamber 108 relatively close to the sample S. In operation, the detectors 160, 170, and 180 generate measurement streams 162, 172, and 182 received by the electronic controller 150. The specific type of detectors 160, 170, 180 depends on the embodiment of the scientific instrument 100, and can generally be selected from a variety of detector types suitable for detecting different types of emissions and / or radiation from the sample S generated in response to the electron beam 114. Example types of emissions / radiation that can be generated in this manner include, but are not limited to, X-rays, infrared light, visible light, ultraviolet light, backscattered electrons, secondary electrons, Auger electrons, elastically scattered electrons, non-scattered (e.g., zero energy loss) electrons, and inelastically scattered electrons. In various embodiments, different (other than three) numbers of such detectors can be used in the scientific instrument 100. In some embodiments, detectors 160, 170, 180 are selected from the group consisting of: a high-angle annular dark field detector, a medium-angle annular dark field detector, an annular bright field detector, a segmented annular detector, a differential phase contrast detector, and a two-dimensional (e.g., pixelated) diffraction pattern detector. Other detectors capable of detecting various of the above-mentioned types of emission / radiation may also be used in various additional embodiments.

[0031] Figure 2 FIG. 2 is a block diagram 200 illustrating acquiring a Ronchi map image 210 in a scientific instrument 100 according to some embodiments. Figure 2 Only one probe forming lens, namely electron beam lens 202, is shown in FIG. Electron beam lens 202 represents the front field of the objective lens, which can actually be formed by more than one electron beam lens (for example, see Figure 1 The compression of the electron beam 114 caused by the objective back field and the effect of any back lens of the specimen are Figure 2 The Ronchi diagram image 210 is captured by a two-dimensional pixelated detector (eg, a CCD camera) 180 .

[0032] In some implementations, prior to Ronchi map acquisition, the sample S and / or the electron beam 114 are positioned so that the electron beam 114 strikes the sample S in an amorphous region of the sample S. In some other implementations, a crystalline region of the sample S may also be used. During acquisition of the Ronchi map images 210, the electron beam 114 remains stationary and the post-specimen intensity of the beam is recorded by the CCD camera 180 or by a functionally equivalent device. To perform the above-described Ronchi map alignment, a series of Ronchi map images 210 are acquired, wherein each Ronchi map image in the series of Ronchi map images 210 is captured after a small change in electrical excitation in a selected component of the probe-forming lens 202.

[0033] The above series of Ronchi diagrams can be used to guide STEM alignment because the intensity of the electron beam 114 formed at the Fraunhofer diffraction plane of the STEM varies significantly with angle, and this variation is a sensitive function of lens aberrations and defocus d. When the excitation of the optical components of the probe forming lens 202 is slightly changed, small misalignments become apparent through shifts in the Ronchi diagram pattern that deviate from circular symmetry. In addition, the presence or absence of interference fringes in the Ronchi diagram pattern is related to the amount of incoherent broadening.

[0034] At large defocus d, the intersection point F of the electron rays is at a relatively large distance from the sample S along the longitudinal axis 115, and the projected image can be observed with the detector 180. When approaching Gaussian focus, an angular dependence on magnification typically occurs, for example due to lens aberrations and the way in which lens aberrations affect the phase of the electron beam 114. At slightly underfocus (i.e., when the intersection point F is slightly below the sample S), azimuthal and sagittal circles of almost infinite magnification can be observed. These are angles where the defocus d and spherical aberration effectively cancel each other. For example, by energizing the astigmatism correction coils in the STEM column 102, the axial astigmatism can be accurately corrected so that those Ronchi diagram features are approximately circularly symmetric. As the electron beam 114 is better focused on the sample S, the low angles in the center show progressively higher magnifications. The coma-free axis is clearly defined in the Ronchi diagram in this position, and all alignment and positioning of the detector and aperture can be advantageously performed relative to this axial point. The direct observability of the coma-free axis in the series of Ronchi diagram images 210 is a significant advantage of Ronchi diagram alignment over some other alignment methods in which, for example, the current or voltage center of the objective is used as a proxy reference, and this type of reference is generally not as accurate as a directly visible reference.

[0035] After alignment of the electron beam 114 with respect to the coma-free axis, control of the intensity distribution of the beam depends on the precise setting of the electron beam lens 202 and the size of the STEM objective aperture, which is usually inserted to exclude peripheral aberrations corresponding to high angles of the beam. For example, low-order aberrations can now be corrected, based on their appearance in the Ronchi map image 210 captured by the detector 180, for example by "turning" the A1 and C1 virtual knobs of the scientific instrument 100. Here, the term "A1 virtual knob" refers to the double astigmatism (C1) that mainly affects the lens aberration function χ(k). 12) component. The term "C1 virtual knob" refers to one or more settings of the electron beam lens 202 that primarily affect the defocus d. The A1 and C1 virtual knobs are typically turned in small increments to navigate from a detuned lens configuration to a configuration where the associated low-order aberrations are substantially corrected. Due to the inherent ambiguity of inferring actual A1 and C1 values ​​from a single Ronchi diagram image 210, a series of Ronchi diagram images 210 are typically required to guide the navigation process, for example, as described below with reference to Figure 3A These inherent ambiguities are manifestations of corresponding symmetries in the lens aberration function χ(k) (see also equation (1)). Various embodiments disclosed herein relate to automating the process of navigating through the alignment parameter space of electron beam optics, for example, as described in more detail below.

[0036] Figure 3A 3 is a Ronchi diagram chart 300 corresponding to the A1-C1 parameter space of the scientific instrument 100 according to one example. The Ronchi diagram chart 300 has a corresponding reference Ronchi diagram 310 for each (A1, C1) value pair including the i-th value of the A1 parameter and the j-th value of the C1 parameter. ij , where i and j are indices. Each Ronchi graph 310 ij is a reciprocal space (k-space) Ronchi map that is related to a corresponding x-space Ronchi map image 210 captured by the detector 180 via a Fourier transform.

[0037] In the diagram 300 , the Ronchi diagram 310 00 This corresponds to a target alignment state of the scientific instrument 100, wherein the relevant low-order aberrations are substantially corrected. For example, according to the Ronchi diagram 310 in the diagram 300 8,-10 ,310 10,-8 and 310 9,-9 Comparison of single Ronchi diagram 310 ij The inherent ambiguity in inferring the actual A1 and C1 parameter values ​​is evident. These individual Ronchi diagrams are very similar to each other, even though each of them corresponds to a different pair of corresponding (A1, C1) values. Thus, in obtaining a Ronchi diagram that looks like 310 8,-10 ,310 10,-8 and 310 9,-9 When a single Ronchi diagram of any Ronchi diagram in is obtained, unambiguous inference of the corresponding actual (A1, C1) parameter value pair is difficult. However, without knowing the current position of the instrument in the A1-C1 parameter space, even the sign of the required A1 and / or C1 parameter adjustment may not be unambiguously determined in some cases.

[0038] Various embodiments disclosed herein use machine learning techniques to address at least some of the above-mentioned problems in the prior art. In a representative example, a deep learning framework is provided to guide the instrument alignment process through the alignment parameter space (defined by Figure 3A 300 ). The words of the alignment language are represented by Ronchi diagrams, and grammatically correct sentences of the alignment language are represented by a sequence of Ronchi diagrams that result in a target alignment state for the scientific instrument 100, such as the Ronchi diagram 310 corresponding to the diagram 300. 00 In some examples, the deep learning architecture relies on: (i) a digital twin of the scientific instrument 100 capable of generating reference Ronchi diagrams for multiple known alignment parameter space settings; (ii) training an AI agent to tune the digital twin toward a target alignment state; and (iii) deploying the trained AI agent with the electronic controller 150 of the scientific instrument 100 for automated control of the Ronchi diagram alignment process.

[0039] As used herein, the term "digital twin" refers to a computer model of the scientific instrument 100 that is configured to output a computed Ronchi diagram in response to input of any valid set of alignment parameter values. Figure 3A In the example shown, the digital twin operates to calculate the corresponding Ronchi diagram 310 in response to receiving as input a valid (A1, C1) parameter value pair consisting of the i-th value of the A1 parameter and the j-th value of the C1 parameter ij . For different embodiments of the STEM column tuning circuit, the digital twin of the scientific instrument 100 can be configured to calculate Ronchi diagrams for other virtual knob combinations, such as combinations that control alignment parameters other than the A1 and / or C1 parameters and / or combinations with a different (other than two) number of virtual knobs in the knob combination used to tune the STEM column.

[0040] FIG. 3B to FIG. 3C Ronchi diagrams 301 and 302 corresponding to the A1-C1 parameter space of the scientific instrument 100 according to another example are shown, respectively. Both Ronchi diagrams 301 and 302 are generated using a digital twin of the scientific instrument 100. Figure 3B ) has a Ronchi graph image 320 ij , each of these Ronchi diagram images is similar to Ronchi diagram image 210. Ronchi diagram chart 302 ( Figure 3C ) has a reciprocal space (k space) Ronchi diagram 330 ij Each of these Ronchi graphs is obtained by Figure 3B The Ronchi diagram image 320 is shown ijThe corresponding Ronchi diagram image in is calculated by Fourier transform. In each of the Ronchi diagram graphs 301 and 302, the horizontal axis represents the C1 parameter and the vertical axis represents the A1 parameter. 00 and Reciprocal Space Ronchi Diagram 330 00 Indicates the target alignment state of the scientific instrument 100 .

[0041] Some embodiments disclosed herein use a Ronchi diagram image to perform Ronchi diagram alignment. The Ronchi diagram image used in these embodiments may be a measured Ronchi diagram image 210 ( Figure 2 ) or similar to the digital twin Renchi diagram image 320 ij Image( Figure 3B ). Some other embodiments disclosed herein use reciprocal space Ronchi diagrams to perform Ronchi diagram alignment. The reciprocal space Ronchi diagrams used in these embodiments may be similar to the reciprocal space Ronchi diagram 310. ij ( Figure 3A ) or reciprocal space Ronchi diagram 330 ij ( Figure 3C ).

[0042] 4A to 4H The performance of a digital twin of a scientific instrument 100 according to some examples is illustrated. More specifically, 4A to 4H Each of the figures shows a corresponding Ronchi diagram 310 for the indicated index value. ij . Lenchi Figure 310 ij Presented in pairs (grouped into FIG. 4A to FIG. 4B , FIG. 4C to FIG. 4D , FIG. 4E to FIG. 4F and Figures 4G to 4H ), where each pair includes the corresponding experimentally measured Ronchi diagram 310 ij and digital twin Renci diagram 310 ij In order to facilitate the comparison between the two. Figure 4A , Figure 4C , Figure 4E and Figure 4G The corresponding digital twin Ronchi diagrams are shown in Figure 4B , Figure 4D , Figure 4F and Figure 4H The comparison of the Ronchi plots within each pair reveals that the experimentally measured Ronchi plots 310 ij Renci diagram 310 with digital twin ij There is some mismatch between the two, which is visually indicated by the difference in the fit of the major and minor axes for some pairs of Ronchi plots.

[0043] In different examples, the AI ​​agent can be used in experimentally measured Ronchi graphs 310 ij On or in the digital twin Renchi diagram 310ij The experimentally measured Ronchi graph 310 ij Typically available than digital twin Renchi diagram 310 ij More accurate AI agent training. However, the improvement in accuracy comes at the cost of inputs targeting those Ronchi graphs 310 ij On the other hand, the use of digital twin Lench diagram 310 ij Some loss of accuracy can be achieved by generating a relatively large number of digital twin Ronchi diagrams 310 when a relatively accurate digital twin of the scientific instrument 100 is available. ij The speed and ease of use make up for it. 4A to 4H The presence of mismatches shown in Figure 310 may generally reduce the Ronchi diagram in the digital twin ij The convergence rate of the trained AI agent towards the target state, but such digital twin Ronchi diagrams still provide experimental Ronchi diagrams for convergence occurrence 310 ij Therefore, depending on the cost-benefit analysis applicable to different use cases, the experimentally measured Ronchi diagram 310 ij Or digital twin Renchi diagram 310 ij Can be used to train AI agents. In some cases, digital twin Ronchi diagram 310 ij The use of may be preferred for training AI agents.

[0044] Figure 5 is a flow chart illustrating a method 500 of training an AI agent to perform Ronchi diagram alignment in a scientific instrument 100 according to some embodiments. For purposes of illustration and without any implied limitation, the method 500 is described with reference to the example A1-C1 alignment parameter space described above. Based on the description provided, one of ordinary skill in the relevant art will be able to make and use other embodiments of the method 500 corresponding to other alignment parameter spaces.

[0045] The operation of block 502 of method 500 includes providing a plurality of reference Ronchi maps 310 ij , which will fully sample the alignment parameter space, such as the A1-C1 alignment parameter space ( FIG. 3A to FIG. 3C In some examples, the plurality of reference Ronchi diagrams 310 provided in block 502 may include: ij In some other examples, the plurality of Ronchi diagrams 310 provided in block 502 may include experimentally measured Ronchi diagrams. ijThe digital twin Ronchi diagram is included. The granularity of the parameter space sampling by reference to the Ronchi diagram is an optional configuration parameter of the method 500. In different examples, the sampling granularity of the A1 and C1 dimensions of the alignment parameter space can be the same or different. In some examples, the sampling granularity of a given dimension of the alignment parameter space can be non-uniform. In some examples of such examples, the closer to the target state 310 of the scientific instrument 100 00 Using a relatively finer sampling granularity, and moving further away from the target state 310 of the scientific instrument 100 00 Use a relatively coarser sampling granularity.

[0046] The operations of block 504 of method 500 include labeling the provided Ronchi diagram 310 ij The individual Ronchi diagrams in the alignment parameter space are separated to reflect the fact that substantially similar Ronchi diagrams can represent different positions in the alignment parameter space. In one example, the operation of block 504 includes giving different corresponding labels to each group of such substantially similar Ronchi diagrams.

[0047] Figure 6 6 is a Ronchi diagram chart 600 illustrating labeling operations of block 504 according to some examples of method 500. Chart 600 is generally similar to chart 300 ( Figure 3A ), but for simplicity, only nine reciprocal space (k-space) Ronchi diagrams 610 are included k (where k = 0, 1, ..., 8). Ronchi diagram 6100 represents the target alignment state of scientific instrument 100 (see also Figure 3A of 310 00 ), and are assigned a label of "0" by the labeling operation of box 504. Ronchi diagrams 6101 and 6105 are substantially similar to each other and are assigned a label of "1". Ronchi diagrams 6102 and 6106 are substantially similar to each other and are assigned a label of "2". Ronchi diagrams 6103 and 6107 are substantially similar to each other and are assigned a label of "3". Ronchi diagrams 6104 and 6108 are substantially similar to each other and are assigned a label of "4".

[0048] The operation of block 506 of method 500 includes converting the labeled Ronchi graph 310 generated by the labeling operation of block 504 into ij Tokenization and embedding. The resulting embedding is a labeled Ronchi graph 310 ij In some examples, the corresponding vectors are real-valued and are constructed to align the labeled Ronchi graph 310 in the parameter space. ij The "meaning" of the scientific instrument 100 is encoded in the target state 310 00 In other words, the labeled Ronchi graphs 310 with embeddings that are relatively close to each other in the vector space ijIt is expected to have a target state 310 relative to the graph 300 00 . In various embodiments, the operation of block 506 can be implemented using one or more of a neural network, dimensionality reduction, a probabilistic model, an interpretable knowledge base method, and / or other methods commonly used for these purposes in natural language processing (NLP) models. In some other examples, the embedding does not fully reflect the A / C parameters or maintain their distance in the original parameter space. In such examples, the Ronchi diagrams are encoded in a manner that reflects their FFT shapes, because there will be multiple AC combinations that cause (substantially) the same Ronchi diagram. The encoder exploits this symmetry and relabels the corresponding Ronchi diagrams accordingly.

[0049] The operations of block 508 of method 500 include constructing a training text corpus by: (i) generating a plurality of different alignment sequences, each of which results in a target state 310 of scientific instrument 100; 00 or 6100, and (ii) representing each alignment sequence with a corresponding tokenized sentence. Figure 6 , an exemplary alignment sequence for such a training text corpus is indicated by arrows 612 and 614. The initial alignment state of the example sequence corresponds to Ronchi diagram 6101. Arrow 612 represents a "turn" of the C1 virtual knob of the scientific instrument 100, which causes the instrument to transition from the initial alignment state to an intermediate alignment state corresponding to Ronchi diagram 6102. Arrow 614 represents a "turn" of the A1 virtual knob of the scientific instrument 100, which causes the instrument to transition from the intermediate alignment state to a target alignment state corresponding to Ronchi diagram 6100. A tokenized sentence representing the exemplary alignment sequence is "X1X2X0", where X1, X2, and X0 are tokens corresponding to the labeled Ronchi diagrams 6101, 6102, and 6100, respectively. In the example shown, the alignment path (612, 614) is linear in parameter space and has segments oriented parallel to one of the parameter space axes because each segment of the alignment path represents a turn of a single knob. In other examples, each of the alignment path segments may represent the rotation of a single knob or the rotation of two knobs simultaneously. In such examples, the alignment path is still linear, but its segments are not necessarily parallel to one of the parameter space axes.

[0050] In some examples, the training text corpus constructed in block 508 includes tokenized sentences of various lengths, where (i) each tokenized sentence ends with a token X0, and (ii) any two adjacent tokens of the tokenized sentences are connected by a one-dimensional (1D) tuning operation. As used herein, the term "1D tuning operation" refers to a tuning operation during which a single virtual knob of the scientific instrument 100 is turned. Figure 6In the example shown, each of arrows 612 and 614 represents a corresponding 1D tuning operation. For arrow 612, the corresponding 1D tuning operation includes turning the C1 virtual knob but not turning the A1 virtual knob. For arrow 614, the corresponding 1D tuning operation includes turning the A1 virtual knob but not turning the C1 virtual knob. The path through the A1-C1 alignment parameter space represented by arrows 612 and 614 is straight. Typically, any two adjacent tokens of the tokenized sentence correspond to the opposite ends of the corresponding linear segments of the straight path in the parameter space. Some of the linear segments are parallel to the first (e.g., A1) dimension of the parameter space. Some other linear segments are parallel to the second (e.g., C1) dimension of the alignment parameter space. In some other examples, the AI ​​agent is configured to implement some of the tuning steps by tuning both the A1 and C1 knobs in a 2D tuning operation.

[0051] The operation of block 510 of method 500 includes training an autoregressive masked language model on the training text corpus constructed in block 508. In an exemplary embodiment, the training and subsequent use of the autoregressive masked language model is performed using the following reference Figures 7 and 8 After the training of the model is completed, the AI ​​agent can be operated to predict and select the next alignment state of the scientific instrument 100 based on the previous sequence of measured Ronchi diagrams. Incremental tuning of the scientific instrument 100 through such a sequence of next states will eventually lead to a target alignment state, for example corresponding to the Ronchi diagram 6100 ( Figure 6 )’s target alignment status.

[0052] Figure 7 is a block diagram illustrating a transformer 702 that performs Ronchi graph alignment in a scientific instrument 100 according to some embodiments. In the example shown, the transformer 702 is part of a training circuit 700 for training the transformer 702 according to the method 500. In the training circuit 700, the transformer 702 is configured to receive a reference sentence 704 selected from the text corpus constructed in block 508 of the method 500, for example, as described above with reference to Figures 5 and 6 In operation, the selected reference sentence 704 is applied to the input preprocessing module 710 of the transformer 702. In one example, the input preprocessing module 710 performs embedding and position encoding operations on the received reference sentence 704. The output 712 of the input preprocessing module 710 is a representation of (x1, ..., x n ), which is applied to the encoder 720. The encoder operates to represent (x1, ..., x n ) is mapped to the sequence 712 representing z=(z1,…,z n) sequence 722. In various examples, the encoder 720 is implemented as a single encoder or an encoder stack, wherein each such encoder includes two corresponding sublayers. The first of the two sublayers includes a corresponding multi-head self-attention mechanism for the input vector. The second of the two sublayers includes a corresponding positionally fully connected feedforward neural network. In other examples, other suitable transformer configurations may also be used.

[0053] Given z, decoder 740 operates to generate a sequence of symbols (y1, ..., y m ) output vector 742. In some examples, decoder 740 is implemented using a decoder stack, where each decoder in the stack includes three corresponding sublayers. A first sublayer of the three sublayers is a corresponding masked multi-head self-attention mechanism for the output vector of the previous iteration. A second sublayer of the three sublayers is a corresponding multi-head attention mechanism for the output 722 of encoder 720 and the output of the first sublayer. A third sublayer of the three sublayers is a corresponding positionally fully connected feedforward neural network.

[0054] The output vector 742 of the decoder 740 is applied to the output post-processing module 750. In a representative example, the output post-processing module 750 includes a linear layer, followed by a softmax layer. The linear layer is usually implemented using a fully connected neural network, which projects the output vector 742 into a larger vector commonly referred to as a logits vector. The softmax layer then operates to convert the logits vector into a set of probabilities (all positive values, which add up to 1). The unit with the highest probability is selected, and the token associated with the unit is generated as the output 752 of the time step. The token of the output 752 is guided back to the box 706 via a return loop 754, wherein a sequence of such tokens is generated by placing each next token 752 to the right of the corresponding previous tag 752 until the <end of sentence> symbol is generated by the decoder 740. Each incomplete sentence of the box 706 is applied to the output pre-processing module 730 to start the next decoding iteration. The output pre-processing module 730 is similar in function to the input pre-processing module 710, and performs embedding and position encoding operations on the sentence 706.

[0055] After the <end of sentence> symbol is generated, the completed sentence of block 706 is compared to the corresponding original sentence 704 in a loss function module 708. In some examples, the loss function module 708 guides the training of the transformer 702 based on cross entropy and Kullback-Leibler divergence. When the completed output sentence 706 does not reconstruct the input sentence 704 well, the loss function module 708 applies a corresponding penalty. Various parameters of the neural network used in the transformer 702 are iteratively adjusted until the penalty applied by the loss function module 708 to any of the reference sentences 704 from the text corpus constructed in block 508 of method 500 becomes less than a selected threshold.

[0056] Figure 8 is a block diagram illustrating an optical alignment circuit 800 used in the electronic controller 150 of the scientific instrument 100 according to some embodiments. The optical alignment circuit 800 is configured to connect the (trained) transducer 702 to the electron beam optics of the STEM column 102 and to the two-dimensional pixelated detector 180, e.g., as Figure 8 Note that the optical alignment circuit 800 has a return loop 754 inserted into the transducer 702 (see also Figure 7 ) so that the latter can drive a Ronchi diagram alignment process of the scientific instrument 100, for example, as described in more detail below.

[0057] In operation, the two-dimensional pixelated detector 180 captures the Figure 2 The detector 180 then transmits the captured Ronchi map 210 to a Fourier transform (FT) module 810 of the circuit 800, for example, via the measurement stream 182. The FT module 810 applies an inverse Fourier transform to the received Ronchi map 210 to generate a corresponding reciprocal space (k-space) Ronchi map 310, which is then applied to the mapper 820. In some examples, the Fourier transform operation is omitted.

[0058] Note that the Ronchi map 310 applied to the mapper 820 belongs to the continuous space of Ronchi maps, where the patterns in the experimentally measured Ronchi map 210 can generally change smoothly. However, as described above, using the Ronchi map 310 that discretely samples the continuous space of Ronchi maps ij The finite set of Ronchi graphs is used to train the transformer 702. Therefore, the mapper 820 is used to map the Ronchi graph from the continuous space of Ronchi graphs to the Ronchi graph 310. ij on a finite set of, on which transformer 702 has been trained via method 500.

[0059] In one embodiment, the mapper 820 comprises a variational autoencoder that operates to convert any continuous space Ronchi graph 310 to the closest corresponding finite set Ronchi graph 310 ij In some examples, the conversion process includes the following two sets of operations. During the first set of operations, the encoder portion of the variational autoencoder maps the continuous space Ronchi map 310 to the corresponding latent representation. During the second set of operations, the decoder portion of the variational autoencoder projects the latent representation onto the reference Ronchi map 310. ij The mapper 820 then operates to output a token 752' which represents the reference Ronchi map 310 selected in this manner by the variational autoencoder. ij The output token 752' is input to block 706, as described above with reference to Figure 7 A copy of the token 752 ′ is also provided to an increment (Δ) circuit 830 .

[0060] In some embodiments, the functionality of both the FT module 810 and the mapper 820 may be implemented using a (differently trained) variational autoencoder. In such embodiments, the encoder portion of the variational autoencoder is trained to map the Ronchi graph 210 to the corresponding latent representation, with or without performing a Fourier transform as an explicit processing step. The decoder portion of the variational autoencoder is trained to project the latent representation onto the reference Ronchi graph 310. ij The corresponding one in the above.

[0061] Next, the transformer 706 operates to process the current unfinished sentence 706, thereby outputting the next token 752 for the sentence, for example, as described above with reference to Figure 7 The increment circuit 830 then operates to process the tokens 752 and 752' to determine the alignment parameter space (eg, FIG. 3A to FIG. 3C The parameter difference 832 between the tokens in the (A1, C1) alignment parameter space is shown. Figure 3A In some examples of the parameter space corresponding to Figure 3A In some other examples of the parameter space of FIG. 8 , the parameter difference 832 has a non-zero C1 component. The determined parameter difference 832 is applied to a knob turning circuit 840, which is configured to convert the received parameter difference 832 into a set of one or more corresponding adjustments 8421-842 N , for the set of control signals applied by the electronic controller 150 to the electron beam optics of the STEM column 102. Figure 1 In the embodiment shown, one or more adjustments 8421-842 may be used. Nto change one or both of the control signals 152 and 156. The control signal adjustment accordingly changes the alignment state of the electron beam optics in the STEM column 102. The latter change will then be manifested in the next Ronchi plot 210 captured by the detector 180.

[0062] Fig. 9 is a flow chart of an automated method 900 for aligning electron beam optics in a scientific instrument 100 according to some embodiments. In some examples, the method 900 is implemented using the optical alignment circuit 800. Figure 2 and Figures 7 to 9 To describe method 900.

[0063] The operations of block 902 of method 900 include operating the scientific instrument 100 to measure a Ronchi diagram 210 corresponding to a current alignment state of the electron beam optics of the STEM column 102. Examples of such measurements are described above, for example, with reference to Figure 2 The operation of block 904 of method 900 includes converting the measured Ronchi map 210 into corresponding tokens 752' from a plurality of reference tokens with which the transformer 702 has been trained. In some examples, the FT module 810 and the mapper 820 (see also Figure 8 ) to perform the conversion operation of block 904. The operation of block 906 of method 900 includes operating transformer 702 to process the current unfinished sentence 706 and output the next token 752 of the sentence, for example, as described above with reference to Figure 7 described.

[0064] The operation of decision block 908 includes determining whether the target alignment state of the scientific instrument 100 is reached. In one embodiment, the target alignment state is indicated by the converter 702 via the generation of the <end of sentence> symbol. When it is determined that the target alignment state is reached ("yes" at decision block 908), the operation of method 900 terminates. When it is determined that the target alignment state is not reached ("no" at decision block 908), method 900 proceeds to the operation of execution block 910.

[0065] The operation of block 910 includes adjusting one or more control signals applied to the electron beam optics in the STEM column 102 based on the tokens 752 and 752'. In some examples, the incremental circuit 830 and the knob rotation circuit 840 (see also Figure 8 ) to perform adjustment of one or more control signals. After completing the operations of block 910, method 900 loops back to block 902.

[0066] Fig.10is a block diagram of an exemplary computing device 1000 configured to perform at least some scientific instrument support operations according to various embodiments. For example, in some embodiments, the computing device 1000 executes the electronic controller 150 (see also Figure 1 ) In various embodiments, the support modules of the scientific instrument 100 may be implemented by a single computing device 1000 or by multiple computing devices 1000.

[0067] Fig.10 The computing device 1000 is shown as having multiple components, but any one or more of these components may be omitted or repeated as appropriate for the application and setting. In some embodiments, some or all of the components included in the computing device 1000 may be attached to one or more motherboards and packaged in a housing. In some embodiments, some of these components may be manufactured onto a single system on a chip (SoC) (e.g., the SoC may include one or more processing devices 1002 and one or more storage devices 1004). Additionally, in various embodiments, the computing device 1000 may not include Fig.10 The computing device 1000 may include one or more of the components shown, but may include interface circuitry for connecting to the one or more components using any suitable interface (e.g., a universal serial bus (USB) interface, a high-definition multimedia interface (HDMI) interface, a controller area network (CAN) interface, a serial peripheral interface (SPI) interface, an Ethernet interface, a wireless interface, or any other suitable interface). For example, the computing device 1000 may not include the display device 1010, but may include a display device interface circuit (e.g., a connector and a driving circuit) capable of connecting to the external display device 1010.

[0068] The computing device 1000 includes a processing device 1002 (e.g., one or more processing devices). As used herein, the term "processing device" refers to any device or portion of a device that processes electronic data from registers and / or memory to convert the electronic data into other electronic data that can be stored in registers and / or memory. In various embodiments, the processing device 1002 may include one or more digital signal processors (DSPs), application specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), server processors, or any other suitable processing devices.

[0069] The computing device 1000 also includes a storage device 1004 (e.g., one or more storage devices). In various embodiments, the storage device 1004 may include one or more memory devices, such as a random access memory (RAM) device (e.g., a static RAM (SRAM) device, a magnetic RAM (MRAM) device, a dynamic RAM (DRAM) device, a resistive RAM (RRAM) device, or a conductive bridge RAM (CBRAM) device), a hard drive-based memory device, a solid-state memory device, a network drive, a cloud drive, or any combination of memory devices. In some embodiments, the storage device 1004 may include a memory that shares a die with the processing device 1002. In such an embodiment, the memory may be used as a cache memory and include, for example, an embedded dynamic random access memory (eDRAM) or a spin-transfer torque magnetic random access memory (STT-MRAM). In some embodiments, the storage device 1004 may include a non-transitory computer-readable medium having instructions thereon that, when executed by one or more processing devices (e.g., the processing device 1002), cause the computing device 1000 to perform any appropriate method or portions of such methods disclosed herein below.

[0070] The computing device 1000 also includes an interface device 1006 (e.g., one or more interface devices 1006). In various embodiments, the interface device 1006 may include one or more communication chips, connectors, and / or other hardware and software to manage communications between the computing device 1000 and other computing devices. For example, the interface device 1006 may include circuitry for managing wireless communications that transmit data to and from the computing device 1000. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communications channels, etc. that can transmit data via modulated electromagnetic radiation through a non-solid medium. The term does not imply that the associated device does not contain any wires, although in some embodiments it may not contain any wires. The circuitry included in the interface device 1006 for managing wireless communications may implement any of a plurality of wireless standards or protocols, including but not limited to Institute of Electrical and Electronics Engineers (IEEE) standards, including Wi-Fi (IEEE 802.11 series), IEEE 802.16 standards, Long Term Evolution (LTE) projects, and any amendments, updates, and / or revisions (e.g., Advanced LTE projects, Ultra Mobile Broadband (UMB) projects (also known as “3GPP2”), etc.). In some embodiments, the circuitry included in the interface device 1006 for managing wireless communications may operate according to a Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE network. In some embodiments, the circuitry included in the interface device 1006 for managing wireless communications may operate in accordance with Enhanced Data for GSM Evolution (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). In some embodiments, the circuitry included in the interface device 1006 for managing wireless communications may operate in accordance with Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Telecommunications (DECT), Evolution Data Optimized (EV-DO) and derivatives thereof, as well as any other wireless protocols designated as 3G, 4G, 5G and higher.In some embodiments, the interface device 1006 may include one or more antennas (eg, one or more antenna arrays) configured to receive and / or transmit wireless signals.

[0071] In some embodiments, interface device 1006 may include a circuit for managing wired communication (such as electricity, optics or any other suitable communication protocol). For example, interface device 1006 may include a circuit for supporting communication according to Ethernet technology. In some embodiments, interface device 1006 may support both wireless communication and wired communication, and / or may support multiple wired communication protocols and / or multiple wireless communication protocols. For example, a first group of circuits of interface device 1006 may be dedicated to close range wireless communication (such as Wi-Fi or Bluetooth), and a second group of circuits of interface device 1006 may be dedicated to long-range wireless communication (such as global positioning system (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO or other). In some other embodiments, a first group of circuits of interface device 1006 may be dedicated to wireless communication, and a second group of circuits of interface device 1006 may be dedicated to wired communication.

[0072] Computing device 1000 also includes battery / power circuitry 1008. In various embodiments, battery / power circuitry 1008 may include one or more energy storage devices (e.g., batteries or capacitors) and / or circuitry for coupling components of computing device 1000 to an energy source separate from computing device 1000 (e.g., AC line power).

[0073] The computing device 1000 also includes a display device 1010 (e.g., one or more separate display devices). In various embodiments, the display device 1010 may include any visual indicator, such as a head-up display, a computer monitor, a projector, a touch screen display, a liquid crystal display (LCD), a light emitting diode display, or a flat panel display.

[0074] The computing device 1000 also includes additional input / output (I / O) devices 1012. In various embodiments, the I / O devices 1012 may include one or more data / signal transmission interfaces, audio I / O devices (e.g., microphones or microphone arrays, speakers, headphones, earbuds, alarms, etc.), audio codecs, video codecs, printers, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors, vibration sensors, etc.), image capture devices (e.g., one or more cameras), human interface devices (e.g., keyboards, cursor control devices such as mice, styluses, trackballs, or trackpads), and the like.

[0075] Depending on the specific implementation of the scientific instrument 100 and / or the instrument portion 200, the various components of the interface device 1006 and / or the I / O device 1012 can be configured to output appropriate control signals (e.g., 152, 154, 156) for the various components of the scientific instrument 100, receive appropriate control / telemetry signals from the various components of the scientific instrument 100, and receive measurement streams (e.g., 162, 172, 182) from the various detectors of the scientific instrument 100. In some examples, the interface device 1006 and / or the I / O device 1012 include one or more analog-to-digital converters (ADCs) for converting received analog signals into digital form suitable for operations performed by the processing device 1002 and / or the storage device 1004. In some additional examples, the interface device 1006 and / or the I / O device 1012 include one or more digital-to-analog converters (DACs) for converting digital signals provided by the processing device 1002 and / or the storage device 1004 into analog form suitable for being communicated to the corresponding components of the scientific instrument 100.

[0076] According to one example disclosed above, for example, in the Summary section and / or with reference to Figures 1 to 10 According to any one or any combination of some or all of the figures in the text, there is provided a support device for a scientific instrument, the support device comprising: an interface device configured to receive data representing a Ronchi diagram acquired using an electron microscope column of the scientific instrument, and also configured to transmit one or more control signals for an electron beam optical device of the electron microscope column; and a processing device configured to: convert the measured Ronchi diagram into an input token for a converter; generate an output token using the converter based on a tokenized sentence ending with the input token; and determine one or more adjustments to one or more control signals based on the input token and the output token, wherein the input token and the output token belong to a plurality of tokens representing a plurality of reference Ronchi diagrams sampling an alignment parameter space of the electron beam optical device; and wherein the converter implements an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a corresponding path through the alignment parameter space to a target alignment state of the electron beam optical device. As used herein, the term "Ronchi map" should be interpreted as covering Ronchi map images (ie, x-space Ronchi maps) and reciprocal space (k-space) Ronchi maps.

[0077] In some examples of the above support apparatus, the plurality of reference Ronchi diagrams includes one or more Ronchi diagrams generated via a computer model of the scientific instrument.

[0078] In some examples of any of the aforementioned support devices, the plurality of reference Ronchi maps include one or more Ronchi maps acquired using an electron microscope column of a scientific instrument.

[0079] In some examples of any of the supporting apparatuses described above, the processing device is configured to iteratively update the tokenized sentence by appending thereto input tokens representing measured Ronchi diagrams obtained after last adjustment of the one or more control signals.

[0080] In some examples of any of the above supporting devices, different dimensions of the alignment parameter space represent different corresponding components of the geometric aberration.

[0081] In some examples of any of the above support devices, each respective path is a straight line in the alignment parameter space.

[0082] In some examples of any of the supporting devices described above, any two adjacent tokens of the tokenized sentence correspond to opposite ends of respective linear segments of respective paths through the alignment parameter space.

[0083] In some examples of any of the above supporting devices, the corresponding linear segment is parallel to a first dimension of the alignment parameter space representing double astigmatism, or parallel to a second dimension of the alignment parameter space representing defocus.

[0084] In some examples of any of the support devices described above, the one or more adjustments are determined based on lengths of corresponding linear segments in the alignment parameter space.

[0085] In some examples of any of the supporting devices described above, the processing device is configured to convert a measured Ronchi map into an input token by: applying an inverse Fourier transform to the measured Ronchi map to calculate a corresponding reciprocal space Ronchi map; mapping the corresponding reciprocal space Ronchi map to a corresponding reference Ronchi map in a plurality of reference Ronchi maps; and providing a token in a plurality of tokens that represents a corresponding reference Ronchi map in a plurality of reference Ronchi maps as an input token.

[0086] In some examples of any of the supporting devices described above, the transformer includes: an encoder comprising a first multi-headed self-attention mechanism and a first positionally fully connected feedforward neural network; and a decoder comprising a second multi-headed self-attention mechanism, a multi-headed attention mechanism on an output of the encoder and an output of the second multi-headed self-attention mechanism, and a second fully connected feedforward neural network.

[0087] According to another example disclosed above, for example, in the Summary section and / or with reference to Figures 1 to 10Any one or any combination of some or all of the figures in the text, provides an automated method for providing support for a scientific instrument performed via a computing device, the method comprising: converting a measured Ronchi diagram into an input token for a transformer, the measured Ronchi diagram being received via data acquired using an electron microscope column of the scientific instrument; generating an output token using the transformer based on a tokenized sentence ending with the input token; and determining one or more adjustments to one or more control signals of an electron beam optical device for the electron microscope column based on the input token and the output token; wherein the input token and the output token belong to a plurality of tokens, the plurality of tokens representing a plurality of reference Ronchi diagrams sampling an alignment parameter space of the electron beam optical device; and wherein the transformer implements an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a corresponding path through the alignment parameter space to a target alignment state of the electron beam optical device.

[0088] In some examples of the automated method described above, the automated method further includes iteratively updating the tokenized sentence by appending thereto input tokens representing measured Ronchi diagrams obtained after last adjustment of the one or more control signals.

[0089] In some examples of any of the automated methods described above, different dimensions of the alignment parameter space represent different respective components of the geometric aberration.

[0090] In some examples of any of the automated methods described above, each respective path is a straight line in the alignment parameter space.

[0091] In some examples of any of the automated methods described above, any two adjacent tokens of the tokenized sentence correspond to opposite ends of respective linear segments of respective paths through the alignment parameter space.

[0092] In some examples of any of the automated methods described above, the corresponding linear segment is parallel to a first dimension of the alignment parameter space representing double astigmatism, or parallel to a second dimension of the alignment parameter space representing defocus.

[0093] In some examples of any of the automated methods described above, the one or more adjustments are determined based on lengths of corresponding linear segments in the alignment parameter space.

[0094] In some examples of any of the above-described automated methods, the conversion includes: applying an inverse Fourier transform to a measured Ronchi map to calculate a corresponding reciprocal space Ronchi map; mapping the corresponding reciprocal space Ronchi map to a corresponding reference Ronchi map in a plurality of reference Ronchi maps; and providing a token in a plurality of tokens representing a corresponding reference Ronchi map in a plurality of reference Ronchi maps as an input token.

[0095] According to another example disclosed above, for example, in the Summary section and / or with reference to Figures 1 to 10 Any one or any combination of some or all of the figures in the drawings provides a non-transitory computer-readable medium storing instructions that, when executed by a computing device, cause the computing device to perform operations including any of the above automated methods.

[0096] It should be understood that the above description is intended to be illustrative rather than restrictive. By reading the above description, many specific implementations and applications other than the examples provided will be apparent. The scope should not be determined with reference to the above description, but should be determined with reference to the full scope of the appended claims and the equivalents to which these claims are entitled. It is anticipated and intended that future developments will occur in the technology discussed herein, and the disclosed systems and methods will be incorporated into these future examples. In short, it should be understood that the present application is capable of modification and variation.

[0097] All terms used in the claims are intended to be given their broadest reasonable interpretations and their ordinary meanings as understood by those skilled in the art described herein, unless an explicit indication to the contrary is made herein. Specifically, use of singular articles such as "a," "the," "said," etc. should be understood to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary.

[0098] The abstract is provided to allow the reader to quickly determine the nature of the technical disclosure. It should be observed that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing detailed description, it can be seen that various features are grouped together in various examples for the purpose of streamlining the disclosure. The method of the present disclosure should not be interpreted as reflecting the following intention: the claimed subject matter includes more features than those explicitly stated in each claim. Rather, as reflected in the following claims, the subject matter of the present invention lies in less than all the features of a single disclosed example. Therefore, the following claims are hereby incorporated into the detailed description, with each claim acting on its own as a separately claimed subject matter.

[0099] Unless expressly stated otherwise, each numerical value and range should be interpreted as approximate as if the value or range was preceded by the word "about" or "approximately".

[0100] Although elements in the following method claims, if any, are recited in a specific order with corresponding labels, these elements are not necessarily intended to be limited to being implemented in that specific order unless the claim recitation otherwise implies a specific order for implementing some or all of these elements.

[0101] Unless otherwise specified herein, the use of ordinal adjectives "first," "second," "third," etc. to refer to one of multiple similar objects merely indicates that different instances of such similar objects are being referred to, and is not intended to imply that the similar objects so referred to must be in a corresponding order or sequence in time, space, rank, or in any other manner.

[0102] Unless otherwise indicated herein, in addition to its ordinary meaning, the conjunction "if" may also or alternatively be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting", and its interpretation may depend on the corresponding specific context. For example, the phrase "if it is determined" or "if [the condition] is detected" may be interpreted to mean "upon determination" or "in response to determination" or "upon detection of [the condition or event]" or "in response to detecting [the condition or event]".

[0103] Also for the purpose of this specification, the term "coupled", "connected" refers to any means known in the art or later developed, wherein energy is allowed to be transferred between two or more elements, and the insertion of one or more additional elements is contemplated, although not required. Conversely, the term "directly coupled", "directly connected", etc. implies that there are no such additional elements.

[0104] The functions of the various elements shown in the drawings, including any functional blocks marked as "processors" and / or "controllers", can be provided by using dedicated hardware and hardware capable of executing software in association with appropriate software. When provided by a processor, the function can be provided by a single dedicated processor, by a single shared processor, or by multiple separate processors (some of which can be shared). In addition, the explicit use of the term "processor" or "controller" should not be interpreted as exclusively referring to hardware capable of executing software, but may implicitly include but is not limited to digital signal processor (DSP) hardware, network processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), read-only memories (ROMs), random access memories (RAMs) and non-volatile storage devices for storing software. Other conventional and / or customized hardware may also be included. Similarly, any switches shown in the figure are conceptual only. Their functions can be performed by the operation of program logic, by dedicated logic, by the interaction of program control and dedicated logic, or even manually, as more specifically understood from the context, and specific techniques can be selected by the implementer.

[0105] As used in this application, the term "circuitry" may refer to one or more or all of the following: (a) a hardware circuit implementation only (such as an implementation only in analog and / or digital circuits); (b) a combination of hardware circuitry and software, such as (if applicable): (i) a combination of analog and / or digital hardware circuitry and software / firmware, and (ii) a hardware processor and software (including a digital signal processor), any portion of software and memory, which work together to cause a device such as a mobile phone or server to perform various functions; and (c) a hardware circuit and / or processor, such as a microprocessor or a portion of a microprocessor, which requires software (e.g., firmware) to operate, but when software is not required for operation, the software may not be present. This definition of circuitry applies to all uses of the term in this application, including in any claims. As another example, as used in this application, the term circuitry also covers an implementation of a hardware circuit or processor (or multiple processors) or a portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term "circuitry" also covers, for example and if applicable to a particular claim element, a baseband integrated circuit or a processor integrated circuit for a mobile device or a similar integrated circuit in a server, cellular network device, or other computing or networking device.

[0106] Those skilled in the art will appreciate that any block diagrams herein represent conceptual diagrams of illustrative circuits embodying the principles of the present disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocodes, etc. represent various processes that can be substantially represented in a computer-readable medium and thus executed by a computer or processor, regardless of whether such a computer or processor is explicitly shown.

Claims

1. A support device for a scientific instrument, the support device comprising: an interface device configured to receive data representing a Ronchi diagram acquired using an electron microscope column of the scientific instrument and further configured to transmit one or more control signals for electron beam optics of the electron microscope column; and A processing device, the processing device being configured to: converting the measured Ronchi diagram into input tokens for the transformer; generating, using the transformer, an output token based on a tokenized sentence ending with the input token; as well as determining one or more adjustments to the one or more control signals based on the input token and the output token, wherein the input token and the output token belong to a plurality of tokens representing a plurality of reference Ronchi diagrams sampling an alignment parameter space of the electron beam optics; and wherein the transformer implements an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a respective path through the alignment parameter space to a target alignment state of the electron beam optical device.

2. The support device of claim 1, wherein the plurality of reference Ronchi diagrams comprises one or more Ronchi diagrams generated via a computer model of the scientific instrument.

3. The support device of claim 1, wherein the plurality of reference Ronchi diagrams comprises one or more Ronchi diagrams acquired using the electron microscope column of the scientific instrument.

4. The support apparatus according to claim 1, wherein the processing device is configured to iteratively update the tokenized sentence by appending thereto the input token representing the measured Ronchi diagram obtained after the last adjustment of the one or more control signals.

5. A support device according to claim 1, wherein different dimensions of the alignment parameter space represent different corresponding components of geometric aberrations.

6. A support device according to claim 1, wherein each corresponding path is a straight line in the alignment parameter space.

7. The support device according to claim 1, wherein any two adjacent tokens of the tokenized sentence correspond to opposite ends of corresponding linear segments of the corresponding path through the alignment parameter space.

8. The support device according to claim 7, wherein the corresponding linear segment is parallel to a first dimension of the alignment parameter space representing double astigmatism, or parallel to a second dimension of the alignment parameter space representing defocus.

9. The support device of claim 8, wherein the one or more adjustments are determined based on the lengths of the corresponding linear segments in the alignment parameter space.

10. The support apparatus according to claim 1, wherein the processing device is configured to convert the measured Ronchi graph into the input token by: applying an inverse Fourier transform to the measured Ronchi diagram to calculate a corresponding reciprocal space Ronchi diagram; Mapping the corresponding reciprocal space Ronchi diagram to a corresponding reference Ronchi diagram among the plurality of reference Ronchi diagrams; and A token among the plurality of tokens representing the corresponding reference Ronchi graph among the plurality of reference Ronchi graphs is provided as the input token.

11. The support device according to claim 1, wherein the automatic encoder comprises: an encoder comprising a first multi-head self-attention mechanism and a first positionally fully connected feed-forward neural network; and A decoder comprising a second multi-head self-attention mechanism, a multi-head attention mechanism on the output of the encoder and the output of the second multi-head self-attention mechanism, and a second fully connected feedforward neural network.

12. An automated method performed via a computing device for providing support to a scientific instrument, the method comprising: converting a measured Ronchi diagram received via data acquired using an electron microscope column of the scientific instrument into an input token for a transformer; generating, using the transformer, an output token based on a tokenized sentence ending with the input token; as well as determining one or more adjustments to one or more control signals for electron beam optics of the electron microscope column based on the input token and the output token, wherein the input token and the output token belong to a plurality of tokens representing a plurality of reference Ronchi diagrams sampling an alignment parameter space of the electron beam optics; and wherein the transformer implements an autoregressive masked language model trained on a corpus of reference sentences, each of the reference sentences representing a respective path through the alignment parameter space to a target alignment state of the electron beam optical device.

13. The automated method of claim 12, further comprising iteratively updating the tokenized sentence by appending thereto the input token representing the measured Ronchi diagram obtained after a last adjustment of the one or more control signals.

14. An automated method according to claim 12, wherein different dimensions of the alignment parameter space represent different corresponding components of geometric aberrations.

15. The automated method of claim 12, wherein each respective path is linear in the alignment parameter space.

16. An automated method according to claim 12, wherein any two adjacent tokens of the tokenized sentence correspond to opposite ends of respective linear segments of the respective paths through the alignment parameter space.

17. The automated method of claim 12, wherein the corresponding linear segments are parallel to a first dimension of the alignment parameter space representing double astigmatism, or parallel to a second dimension of the alignment parameter space representing defocus.

18. The automated method of claim 12, wherein the one or more adjustments are determined based on lengths of the corresponding linear segments in the alignment parameter space.

19. The automated method of claim 12, wherein converting comprises: applying an inverse Fourier transform to the measured Ronchi diagram to calculate a corresponding reciprocal space Ronchi diagram; Mapping the corresponding reciprocal space Ronchi diagram to a corresponding reference Ronchi diagram among the plurality of reference Ronchi diagrams; as well as A token among the plurality of tokens representing the corresponding reference Ronchi graph among the plurality of reference Ronchi graphs is provided as the input token.

20. A non-transitory computer-readable medium storing instructions that, when executed by a computing device, cause the computing device to perform operations including the automated method of claim 12.

Images