Method for determining one or more state parameters of a multi-beam charged particle imaging system, method for restoring an image acquired by a multi-beam charged particle imaging system, and corresponding computer program and system

Abstract

The invention relates to a method for determining state parameters of a multi-beam charged particle imaging system (1) comprising: generating a plurality of secondary beamlets emitted from a sample; for each of at least one primary beamlet: generating a subimage from a signal recorded by the detector for a corresponding secondary beamlet generated in response; and determining state parameters of the multi-beam charged particle imaging system using a machine learning model trained to map at least one subimage of an image generated by at least one primary beamlet and / or characteristics thereof to one or more state parameters. The invention also relates to an image restoration method that maps an aberrated subimage and corresponding beamlet-specific aberration information to a restored subimage. The invention also refers to corresponding computer programs, computer-readable media, machine learning training and data generation methods and corresponding systems.

Description

[0001] 11.02.2026 h - se - 1 -

[0002] Method for determining one or more state parameters of a multi-beam charged particle imaging system, method for restoring an image acquired by a multibeam charged particle imaging system, and corresponding computer program and system

[0003] Related Applications

[0004] This application claims benefit of German patent application No. 10 2025 106373.7 filed on February 20th, 2025, which is hereby incorporated by reference in its entirety.

[0005] Field of the Invention

[0006] The invention relates to methods and systems for determining state parameters of a multi-beam charged particle imaging system e.g., focus parameters, astigmatism parameters or higher order aberration parameters. The determined state parameters can be used to control parameters of the multi-beam charged particle imaging system, e.g., focus parameters or a stigmation alignment. In this way, a multi-beam charged particle imaging system can be quickly, easily and automatically calibrated to improve image quality. The invention also relates to methods and systems for restoring images acquired by a multi-beam charged particle imaging system. The invention particularly concerns the correction of beamlet-specific optical aberrations to improve image quality and uniformity across a multi-beam field of view.

[0007] Background of the Invention

[0008] Using multi-beam charged particle imaging systems such as multi-beam scanning electron microscopes, samples can be analyzed on a microscopic scale. Images of a surface of a sample, for example of a semiconductor wafer, a photolithography mask or a biological sample, can be recorded using these multi-beam scanning electron microscopes. While in a single-beam charged particle imaging system a single beam of charged particles, such as, for example, electrons or ions, is used to analyze the sample, in a multi-beam charged particle imaging system, a plurality of particle beam-lets is used for this purpose. The plurality of the particle beamlets is directed at the surface of the sample at the same time. In this way, a significantly larger area of the surface of the sample can be sampled and analyzed within the same period of timeas compared to a single-beam charged particle imaging system. A multi-beam scanning electron microscope is disclosed, for example, in US 7244949 and in US 20190355544.

[0009] In a multi-beam electron microscope, a sample is irradiated by an array of electron beamlets, comprising for example 4 up to 10,000 electron beamlets, as primary radiation, whereby each electron beam is separated by a distance of 1 - 200 micrometers from its neighboring electron beamlet. For example, a multi-beam charged particle microscope has about 100 separated electron beamlets, arranged on a hexagonal array, with the electron beamlets separated by a distance of about 10 pm. The charged particles can, for example, be electrons or ions.

[0010] The plurality of primary beamlets is focused by a common objective lens on a surface of a sample under investigation, for example a semiconductor wafer. During the illumination of the sample surface with primary beamlets, interaction products, e.g. secondary electrons, originate from the plurality of intersection points formed by the focus points of the primary beamlets, while the amount and energy of interaction products depend on the material composition and topography of the sample surface. The interaction products form a plurality of secondary beamlets, which are collected by the common objective lens and guided onto a detector arranged at a detector plane by a projection imaging system of the multi-beam charged particle imaging system. The detector comprises a plurality of detection areas with each comprising a plurality of detection pixels and detects an intensity distribution for each of the plurality of secondary beamlets, and an image patch of, for example, 100 pm x 100 pm is obtained. To adjust focus position and stigmation of the plurality of secondary beamlets, the multi-beam charged particle imaging systems comprises adjustable electrostatic and magnetic elements.

[0011] The image quality depends, among others, on the characteristics of the primary beam-lets in the sample plane, in particular on the size and shape of the focus points of the primary beamlets in the sample plane. The best image quality can be obtained using radially symmetric focus points with minimal radius. The size and shape of the focus points can be optimized by positioning the focus along the propagation axis of each electron beamlet in the sample plane and by removing aberrations, such as astigmatism and higher order aberrations, using electron-optical elements, such as lenses and multipole fields.The alignment of each primary electron beamlet can be done manually. This is, however, inaccurate and time-consuming and, thus, not suitable for a fast and efficient operation of the multi-beam charged particle imaging system.

[0012] Therefore, image-quality based autofocus and autostigmatism methods have been introduced to accomplish the alignment task. These methods commonly acquire a series of images with different amplitudes of aberrations. From these images, the image quality is deduced using an image quality metric, e.g. resolution or sharpness. To minimize the aberrations, the amplitude with the best image quality is chosen. However, for high throughput applications such as, for example, wafer inspection, such automatic alignment methods still require too much computation time and often form the bottleneck of automatic inspection.

[0013] Approaches for automatic alignment and control in charged particle microscopy are known in the state of the art. DE 10 2019 120 279 B3 and US 2022 / 0262595 A1 disclose methods for determining system state parameters and controlling electro-optical components of charged particle imaging systems in order to improve imaging performance or alignment. Scientific publications such as Advancing electron microscopy using deep learning by K. Chen et al., Journal of Physics: Materials 7 (2024), p.

[0014] 022001.1 - 022001.27, and Automated and autonomous experiments in electron and scanning probe microscopy by S. V. Kalinin et al., Automated and autonomous experiments in electron and scanning probe microscopy, ACS nano 15.8 (2021): p: 12604 – 12627, describe the use of machine-learning techniques for image enhancement, analysis, and automated operation in electron microscopy systems. While these approaches enable improved alignment, automation, or image processing, they do not address the problem of reducing computational effort for automatic alignment of primary beamlets in multi-beam charged particle imaging systems.

[0015] It is, therefore, an objective of this invention to provide a method for automatic alignment of the primary beamlets of multi-beam charged particle imaging systems that requires even less computation time.

[0016] Even with automated alignment methods, the image quality obtained in multi-beam charged particle imaging systems is often non-uniform across the multi-field-of-view. This non-uniformity arises because all primary beamlets are typically operated usingglobal electro-optical settings, such as a global focus or global stigmation setting, while individual beamlets experience different aberrations depending on their position within the beamlet array. In particular, beamlets arranged at different radial distances from the optical axis are affected differently by field curvature, astigmatism, and higher-order aberrations.

[0017] As a consequence, when the multi-beam charged particle imaging system is operated at a global focus or global stigmation setting, only a subset of the beamlets can be optimally focused and stigmatized, while other beamlets remain partially defocused or astigmatic. This leads to a dispersion of spatial information across the multi-field-of-view and results in variations in resolution and image quality between different image patches generated by different beamlets.

[0018] To compensate for these effects, hardware-based correction approaches have been proposed, such as local focus offset elements, field curvature correctors, or beamlet-specific stigmation elements. However, such hardware-based solutions significantly increase the complexity of the charged particle imaging system. In particular, as the number of beamlets increases, the design and integration of beam let-specific correction elements become increasingly challenging due to limitations in electrical routing density, manufacturing tolerances, yield, robustness against contamination or electrical arcing, and overall system cost. These limitations restrict the scalability of multibeam charged particle imaging systems toward larger numbers of beamlets and beamlet shells.

[0019] Furthermore, the amount of spatial information dispersion depends on the alignment state of the system and varies across the multi-field-of-view. In aligned multi-beam systems, both beamlets close to the optical axis and beamlets at large radial distances may experience increased aberrations due to alignment compromises, while beam-lets located at intermediate radial positions may exhibit comparatively better image quality. This makes it difficult to achieve uniform image quality across all beamlets using alignment and hardware correction alone.

[0020] In addition to hardware-based alignment and correction approaches, image restoration methods based on machine learning have been proposed for charged particle microscopy. For example, WO 2024 / 188982 A1 discloses machine learning-basedenhancement of scanning electron microscope images, and de Haan et al., Resolution enhancement in scanning electron microscopy using deep learning (Scientific Reports 9, 12050, 2019), describe deep-learning approaches for improving image resolution by training neural networks on degraded and higher-quality images.

[0021] However, such approaches generally operate on image content alone and implicitly learn degradation characteristics from training data, without explicitly taking into account beam-specific optical transfer behavior of the imaging system. In particular, these approaches do not explicitly utilize beamlet-specific aberration information, such as point spread functions measured at a detector plane, which characterize the optical properties of individual beamlets in a multi-beam charged particle imaging system.

[0022] As a consequence, the applicability of such image-only machine learning approaches to multi-beam charged particle imaging systems is limited. In multi-beam systems, individual beamlets experience different aberrations depending on their position within the beamlet array, leading to non-uniform image quality across the multi-field-of-view. Image-only restoration methods are not inherently adapted to compensate such beamlet-specific and shell-dependent aberrations and may therefore provide limited robustness with respect to changes in alignment state, drift, or system configuration.

[0023] At the same time, the unique design of multi-beam charged particle imaging systems provides access to system-internal information that characterizes the beam quality of individual beamlets. In particular, beamlet-specific intensity distributions or beam profiles can be measured at a detector plane, for example using a scintillator, and contain information about the optical transfer behavior of the system for each beamlet. Nevertheless, conventional alignment and correction approaches primarily use such information for system calibration and hardware adjustment rather than for compensating image quality variations in the image domain.

[0024] As a result, there remains a need for approaches that improve image quality and uniformity across the multi-field-of-view of multi-beam charged particle imaging systems without further increasing hardware complexity or computational bottlenecks, and that enable scalable operation with a large number of beamlets while maintaining high throughput and high resolution.It is therefore an objective of the invention to provide methods and systems for operating multi-beam charged particle imaging systems that improve image quality and uniformity across the multi-field-of-view while reducing computational effort and avoiding increased hardware complexity.

[0025] This objective is achieved by the invention specified in the independent claims. Advantageous embodiments and further developments of the invention are specified in the dependent claims.

[0026] Summary of the invention

[0027] Embodiments of the invention concern methods for determining state parameters of primary beamlets in multi-beam charged particle imaging systems, corresponding systems, computer implemented methods for training machine learning models for this purpose, computer programs and computer-readable media. Further embodiments of the invention refer to image restoration methods for multi-beam charged particle imaging systems, corresponding systems and computer implemented methods for training machine learning models for this purpose, computer programs and computer-readable media.

[0028] A first embodiment involves a method for determining one or more state parameters of a multi-beam charged particle imaging system comprising means for generating a plurality of primary beamlets in a raster configuration for the parallel scanning of a sample, and a detector arranged in or close to a focal plane of a plurality of secondary beamlets emitted from the sample, the method comprising: generating, by irradiation of a sample with a plurality of primary beamlets, a plurality of secondary beamlets that are emitted from the sample; for each of at least one primary beamlet: generating a subimage of an image from a signal recorded by the detector for a corresponding secondary beamlet generated in response to an irradiation of the sample with this primary beamlet; and determining one or more state parameters of the multi-beam charged particle imaging system, by analyzing the at least one generated subimage of the image using a machine learning model trained to map at least one subimage of an image generated by at least one primary beamlet in a multi-beam charged particle imaging system or characteristics thereof to one or more state parameters of the multi-beam charged particle imaging system.Another embodiment of the invention involves a method for determining one or more state parameters of a multi-beam charged particle imaging system comprising means for generating a plurality of primary beamlets in a raster configuration for the parallel scanning of a sample, and a detector arranged in or close to a focal plane of a plurality of secondary beamlets emitted from the sample, the method comprising: generating, by irradiation of a sample with a plurality of primary beamlets, a plurality of secondary beamlets that are emitted from the sample; for each of at least one primary beamlet: generating a subimage of an image from a signal recorded by the detector for a corresponding secondary beamlet generated in response to an irradiation of the sample with this primary beamlet, and determining one or more state parameters of this primary beamlet by analyzing the respective generated subimage of the image using a machine learning model trained to map at least one subimage of an image generated by at least one primary beamlet in a multi-beam charged particle imaging system or characteristics thereof to one or more state parameters of the at least one primary beamlet.

[0029] By determining state parameters of the multi-beam charged particle imaging system, in particular of primary beamlets, automatically from subimages, state parameters of the multi-beam charged particle imaging system, in particular of each primary beam-let, can be determined separately, quickly, reliably and automatically. The determined state parameters can be automatically controlled (e.g., corrected or adjusted) using electro-optical components of the multi-beam charged particle imaging system. By determining state parameters of each primary beamlet, which may slightly differ due to different reasons, the state parameters can be estimated more accurately. To this end, machine learning models are used, since machine learning models automatically derive features and rules for an optimal solution of their task from training data without a human having to define rules by hand, which requires less time, expert knowledge and is more accurate. Thus, the accuracy of the predictions is improved and the user effort and required time for setting up the system are reduced.

[0030] In an example, the method further comprises controlling the one or more determined state parameters of the multi-beam charged particle imaging system, in particular of each of the at least one primary beamlet, using electro-optical components of the multi-beam charged particle imaging system. By controlling each primary beamlet independently, different advantages can be achieved: a sample independent image quality can be achieved, sample charging and sample drift can be reduced, hexagondistortions, pitch distortions or beam tilt can be reduced, system uptime can be improved and system reliability parameters can be determined more accurately. Depending on the number of illuminations of the sample required for acquiring the image, the computation time for aligning the primary beamlets can be drastically reduced. Preferably, only a single illumination of the sample is required.

[0031] A multi-beam charged particle imaging system refers to a system that generates, by irradiation of a sample with a plurality of primary beamlets, a plurality of secondary beamlets that are emitted from the sample and received by a detector. The plurality of primary beamlets is scanned over the sample in parallel, a plurality of timely resolved intensity signals is recorded by the detector, and the intensity signals from all scan positions of each primary beamlet are used to form a subimage of an image. The primary and secondary beamlets consist of charged particles such as ions or electrons.

[0032] State parameters comprise parameters that describe states of the multi-beam charged particle imaging system, in particular states of a primary beamlet of the multibeam charged particle imaging system, e.g., focus parameters, astigmatism parameters, higher order aberration parameters, pitch parameters, beamlet current parameters, point spread function parameters, beamlet intensity parameters, illumination field curvature parameters, illumination field inclination parameters, stigmation parameters. State parameters of the multi-beam charged particle imaging system comprise state parameters of the primary beamlets of the multi-beam charged particle imaging system. Thus, some of the state parameters can refer to state parameters of single primary beamlets of the multi-beam charged particle imaging system, e.g., focus, astigmatism, higher order aberration, beamlet current, beamlet intensity, point spread function parameters, etc. These can be estimated from a single subimage generated by the respective beamlet. Other state parameters can refer to state parameters of the multi-beam charged particle imaging system that can be estimated from two or more, in particular neighboring, subimages generated by primary beamlets, e.g., pitch, illumination field curvature, illumination field inclination, stigmation parameters.

[0033] An image can refer to a multi-beam charged particle imaging system image that is acquired by the multi-beam charged particle imaging system in a full scan of the object, or to a detector image that is acquired directly from the detector using a single illumination of the sample and comprises point spread functions of the secondarybeamlets. A detector image can, for example, be acquired from the visible light emitted by an electron to photon conversion unit (e.g., a scintillator), e.g., using a scintillator image acquisition unit.

[0034] A "scintillator" refers to an electron-to-photon conversion unit. It refers to any material or device capable of emitting photons, typically in the visible or UV spectrum, in response to being struck by high-energy charged particles, such as electrons.

[0035] A subimage of an image refers to a portion of the image that is generated using a single primary beamlet of the multi-beam charged particle imaging system, e.g., during a single illumination of a sample (e.g., in case of a detector image) or during a scan of the sample (e.g., in case of a multi-beam charged particle imaging system image).

[0036] Characteristics of an image or a subimage refer to any kind of features in the image or subimage (e.g., edges, filter responses, etc.) or any kind of data that can be computed from the image or subimage (e.g., properties of objects such as point spread functions, e.g., diameters, brightness, a fitted function, etc.).

[0037] According to an example, each subimage of the image comprises a point spread function recorded by the detector for a secondary beamlet, emitted from the sample in response to an irradiation of the sample with a primary beamlet. By using a detector image, the characteristics of the plurality of secondary beamlets generated by an illumination of the sample with the plurality of primary beamlets become directly visible. In this way, the state parameters can be determined in a highly accurate way.

[0038] In an example, the image is generated during a single irradiation of the sample with the plurality of primary beamlets. By using an image that is acquired using a single illumination of the sample with the plurality of primary beamlets, the computation time for determining the state parameters of the multi-beam charged particle imaging system can be strongly reduced.

[0039] In an example, the image is generated during two or more irradiations of the sample with the plurality of primary beamlets, wherein during each irradiation of the sample a different part of the sample is imaged. In this way, two, three, multiple or all subimages generated by a primary beamlet can be averaged during a scan. By averaging thesubimages, a dependence of the point spread function on the specific sample is reduced. The point spread function is, thus, less dependent on the sample surface. In addition, image properties such as brightness, contrast and noise level are improved by averaging. In this way, the state parameters can be estimated more accurately. A few can, for example, mean two, three, four or five irradiations or less than five, ten or twenty irradiations. A few means less than half the number of positions of a single primary beamlet during a scan of the sample. By using only a few illuminations of the sample, the computation time for determining the state parameters of the multi-beam charged particle imaging system can be strongly reduced.

[0040] In another example, a few images are generated in response to a few irradiations of the sample with the plurality of primary beamlets using different imaging parameters, and the one or more state parameters of the multi-beam charged particle imaging system are determined from the at least one subimage of the few generated images. By acquiring two or more images using different imaging parameters, ambiguous point spread functions can be resolved, e.g., in case the same point spread function results from over-focus and under-focus. By using only a few illuminations of the sample, the computation time for determining the state parameters of the multi-beam charged particle imaging system can be strongly reduced.

[0041] According to an aspect of the invention, the few generated images are averaged, and the one or more state parameters of the multi-beam charged particle imaging system are determined from the at least one subimage of the average image. By averaging, noise, contrast or brightness variations are reduced in the average image and the accuracy of the determined state parameters of the multi-beam charged particle imaging system is improved.

[0042] According to an example, the characteristics of the at least one subimage comprise characteristics of point spread functions of one or more secondary beamlets within the at least one subimage of the image. In particular, the one or more state parameters of a primary beamlet are determined by analyzing characteristics of the point spread function of the respective secondary beamlet within the respective subimage of the image. The characteristics can, for example, comprise statistical properties or measurements of the point spread function such as diameters.According to an example, the image is generated by scanning the plurality of primary beamlets over the sample in parallel, recording a plurality of timely resolved intensity signals provided by the detector and combining the intensity signals from all scan positions of each primary beamlet to form a different subimage of the image. In this way, a multi-beam charged particle imaging system image is generated. Each subimage is generated by a single primary beamlet and can be used to determine state parameters of this beamlet. By using a multi-beam charged particle imaging system image for automatically determining state parameters of primary beamlets, no additional hardware and software is required, e.g., no scintillator image acquisition unit, no specific sensitive optics in the detection path from scintillator to scintillator image acquisition unit and no separate alignment routines.

[0043] In a preferred example, the one or more state parameters of the multi-beam charged particle imaging system comprise one or more state parameters of each of the at least one primary beamlet, and the one or more state parameters of each of the at least one primary beamlet are determined by analyzing the respective at least one generated subimage using a machine learning model trained to map at least one subimage of an image generated by at least one primary beamlet in a multi-beam charged particle imaging system or characteristics thereof to one or more state parameters of the at least one primary beamlet. Thus, the one or more state parameters of the multibeam charged particle imaging system that are determined from the at least one subimage are state parameters of the primary beamlets used to generate these subimages. In this way, for each primary beamlet specific one or more state parameters can be determined.

[0044] The machine learning model may use two or more subimages of an image as input to determine state parameters of the multi-beam charged particle imaging system, in particular of the respective primary beamlets.

[0045] Alternatively, the machine learning model can be trained to map a single subimage of an image generated by a single primary beamlet or characteristics thereof to one or more state parameters of the multi-beam charged particle imaging system, in particular of the primary beamlet.

[0046] In an example, the one or more state parameters of the multi-beam charged particle imaging system comprise one or more state parameters of each of the at least oneprimary beamlet, and the one or more state parameters of each of the at least one primary beamlet are determined by analyzing the respective at least one generated subimage using a machine learning model trained to map a subimage of an image generated by a primary beamlet in a multi-beam charged particle imaging system or characteristics thereof to one or more state parameters of the primary beamlet. Thus, the machine learning model maps a single subimage obtained by primary beamlet to state parameters of this primary beamlet. The complexity of the machine learning task is hereby reduced and the prediction accuracy is improved. In this way, more accurate and beamlet-specific state parameters can be estimated from the respective subimage of the image.

[0047] In an example, the machine learning model is a neural network, in particular a deep neural network. Neural networks are specifically well suited for learning complex relations between input and output and, thus, increase the accuracy of the predicted state parameters.

[0048] According to an aspect, the machine learning model is a residual neural network. Residual neural networks can be used to efficiently train deep neural networks due to their residual formulation, and, thus, improve the accuracy of the predicted state parameters.

[0049] According to another aspect, the neural network comprises at least one attention mechanism. Attention mechanisms allow for taking into account relationships between different parts of the input data and, thus, improve the prediction accuracy of the state parameters.

[0050] In an example, the machine learning model generates a confidence score estimating the accuracy of the predicted one or more state parameters of the multi-beam charged particle imaging system. Such a confidence score is particularly helpful in case of ambiguous subimages, e.g., ambiguous point spread functions, that can indicate different state parameters. A confidence score can reflect such uncertainties due to ambiguities. In case of an ambiguity, two or more images can be acquired with different state parameters to resolve the ambiguity.

[0051] In an example, the confidence score evaluates the ambiguity of the at least one subimage with respect to the predicted one or more state parameters of the multi-beamcharged particle imaging system. In this way, the confidence score can be used for determining ambiguous state parameter predictions. In such cases, alternative methods can be used for determining state parameters, or additional images can be acquired to resolve the ambiguities.

[0052] According to an example, the primary beamlets are grouped into at least two groups, each group is associated with a group-specific machine learning model trained for mapping at least one subimage of the image generated by at least one primary beam-let of the group to one or more state parameters of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet, and the at least one subimage of the image generated by the at least one primary beamlet is mapped to one or more state parameters of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet, using the one or more trained group-specific machine learning models associated with the one or more groups of the at least one primary beamlet. By grouping primary beamlets and associating a machine learning model with each group, the machine learning task for each group is facilitated, e.g., since less variations occur in the input subimages. In addition, location-specific information is contained in each group-specific machine learning model, since inputs only come from a specific group of primary beamlets. Thus, more accurate predictions of state parameters can be obtained.

[0053] A group of the plurality of primary beamlets comprises a subset of the primary beam-lets but not all of the primary beamlets of the multi-beam charged particle imaging system. Thus, a group can comprise a single primary beamlet, two primary beamlets, or multiple primary beamlets of the plurality of primary beamlets. Primary beamlets can, for example, be grouped by partitioning them into at least two groups. Preferably, but not necessarily, each primary beamlet belongs to exactly one group. There may also be primary beamlets belonging to more than one group, e.g., beamlets on the border of two groups may belong to both groups to ensure consistent state parameter estimation between neighboring subimages. The smaller the groups the more accurate are the state parameter predictions, but the more training time and memory is required. By selecting a group size a tradeoff can be achieved between training effort and memory requirement on the one hand and state parameter prediction accuracy on the other hand.

[0054] Throughout this application, the term “multiple” refers to two or more.The primary beamlets can be grouped according to different criteria. Different grouping criteria can be combined.

[0055] The primary beamlets can, for example, be spatially grouped. Spatially neighboring primary beamlets usually generate similar point spread functions such that a single machine learning model is sufficient to produce highly accurate predictions of state parameters for a group. In this way, the machine learning expenses are reduced. The primary beamlets can, for example, be grouped with respect to the shell to which they belong, wherein a shell comprises a group of primary beamlets arranged at a similar radial distance from an optical axis of the multi-beam charged particle imaging system, surrounding a central primary beamlet. Different groupings of primary beamlets can also be used, e.g., grouping with respect to similar state parameter settings, with respect to a user preference or with respect to similar sample properties.

[0056] In an example, each primary beamlet is associated with a separate machine learning model. Thus, each group comprises a single primary beamlet. In this way, the associated machine learning model can specialize in subimages obtained by the single primary beamlet. The machine learning model is, thus, optimally adapted to the subimages generated by the respective primary beamlet. In this way, an increased state parameter prediction accuracy can be achieved. However, a larger number of machine learning models has to be trained, stored and maintained. By grouping primary beamlets, a tradeoff can be achieved between accuracy and machine learning expenses.

[0057] Thus, according to a preferred example, each primary beamlet is associated with a beamlet-specific machine learning model trained for mapping the respective subimage of an image generated by the primary beamlet to one or more state parameters of the primary beamlet, and the at least one subimage of the image generated by the at least one primary beamlet is mapped to one or more state parameters of the at least one primary beamlet using the at least one corresponding trained beamlet-specific machine learning model. By associating a beamlet-specific machine learning model with each primary beamlet, the machine learning task for each primary beamlet is even more facilitated, e.g., since less variations occur in the input subimages. In addition, location-specific information is contained in each beamlet-specific machinelearning model, since inputs are generated by a specific primary beamlet. Thus, even more accurate predictions of state parameters can be obtained.

[0058] According to an example, the steps of the method are iterated. By iterating the state parameter prediction and state parameter control step the predicted state parameters can be iteratively optimized. In this way, the accuracy of the predicted state parameters can be improved.

[0059] According to an embodiment of the invention, a computer implemented method is used for training a machine learning model for mapping at least one subimage of an image generated by at least one primary beamlet in a multi-beam charged particle imaging system to one or more state parameters of the at least one primary beamlet as described above.

[0060] A computer implemented method for training a plurality of group-specific machine learning models that each map at least one subimage of an image generated by at least one primary beamlet from a group of primary beamlets of a multi-beam charged particle imaging system to one or more state parameters of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet, according to an embodiment of the invention comprises: providing a prototype machine learning model for mapping at least one subimage of an image generated by at least one primary beamlet from a group of primary beamlets of a multi-beam charged particle imaging system to one or more state parameters of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet; for each group of primary beamlets, providing group-specific training data comprising at least one subimage of one or more images obtained by at least one primary beamlet of the group of primary beamlets of the multi-beam charged particle imaging system and corresponding one or more state parameters of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet, and training the corresponding group-specific machine learning model by fine-tuning the prototype machine learning model using the group-specific training data.

[0061] Each of the plurality of trained group-specific machine learning models can be used for mapping at least one subimage of an image generated by a primary beamlet of the respective group of primary beamlets of the multi-beam charged particle imaging system to one or more state parameters of the multi-beam charged particle imagingsystem, in particular of the respective at least one primary beamlet. The prototype machine learning model can be trained using 1) training data from different groups of primary beamlets of a single multi-beam charged particle imaging system, 2) training data from the same group of primary beamlets of different multi-beam charged particle imaging systems and / or 3) training data from different groups of primary beamlets of different multi-beam charged particle imaging systems. During fine-tuning, only training data generated by the multi-beam charged particle imaging system whose state parameters will be predicted is used.

[0062] Using this training method, computation time can be reduced, since the prototype machine learning model only needs to be trained once and only has to be fine-tuned for each group of primary beamlets, in particular for each primary beamlet. Furthermore, less training data is required, since the prototype machine learning model can be trained using training data generated by various primary beamlets, even from different multi-beam charged particle imaging systems, that is available more easily, and only the fine-tuning requires group-specific training data generated by a specific group of primary beamlets, in particular by a specific primary beamlet. In this way, for example, the prototype machine learning model can be generated in a centralized location only once, whereas the fine-tuning can take place at a customer’s site for a specific multi-beam charged particle imaging system, requiring less computation time and training data.

[0063] In an example, each group of primary beamlets of the multi-beam charged particle imaging system comprises a single primary beamlet. Thus, a separate machine learning model can be trained for each primary beamlet.

[0064] According to an example, a computer implemented method for training a plurality of beamlet-specific machine learning models that each map a subimage of an image generated by a primary beamlet of a multi-beam charged particle imaging system to one or more state parameters of the primary beamlet comprises: providing a prototype machine learning model for mapping a subimage of an image generated by a primary beamlet of a multi-beam charged particle imaging system to one or more state parameters of the primary beamlet; providing beamlet-specific training data comprising a subimage of one or more images obtained by a primary beamlet of the multi-beam charged particle imaging system and corresponding one or more state parameters ofthe primary beamlet; and training the corresponding beamlet-specific machine learning model by fine-tuning the prototype machine learning model using the beamlet-specific training data. The prototype machine learning model can be trained using training data generated 1) by different primary beamlets of the multi-beam charged particle imaging system, 2) by the same primary beamlet of different multi-beam charged particle imaging systems and / or 3) by different primary beamlets of different multi-beam charged particle imaging systems. The fine-tuning is carried out using training data from the multi-beam charged particle imaging system whose state parameters will be measured by the machine learning model.

[0065] In an example, the plurality of trained group-specific or beamlet-specific machine learning models is used in a method for determining one or more state parameters of a multi-beam charged particle imaging system according to any of the embodiments, examples or aspects described above.

[0066] A computer program according to an embodiment of the invention comprises instructions which, when the program is executed by a computer, cause the computer to carry out a computer implemented method for training one or more machine learning models as described above.

[0067] A computer-readable medium according to an embodiment of the invention has stored thereon a computer program executable by a computing device, the computer program comprising code for executing a computer implemented method fortraining one or more machine learning models as described above.

[0068] A multi-beam charged particle imaging system according to an embodiment of the invention comprises: means for generating a plurality of primary beamlets in a raster configuration for the parallel scanning of a sample; a detector arranged in a focal plane of a plurality of secondary beamlets emitted from the sample; and a data processing unit for determining state parameters of the multi-beam charged particle imaging system and a control unit for controlling state parameters of the multi-beam charged particle imaging system using a method for controlling the determined state parameters as described above.According to an example, the detector of the multi-beam charged particle imaging system comprises a scintillator and the multi-beam charged particle imaging system comprises a scintillator image acquisition unit.

[0069] According to an embodiment of the invention, a method for restoring an image in a multi-beam charged particle imaging system is provided. The system comprises means for generating a plurality of primary beamlets and a detector arranged in or close to a focal plane of a plurality of secondary beamlets. The method comprises acquiring a plurality of aberrated subimages, wherein each aberrated subimage is generated from signals recorded by the detector and corresponds to an area of a sample scanned by a respective primary beamlet, obtaining beam let-specific aberration information comprising a point spread function or parameters derived therefrom, generating a plurality of restored subimages by applying, for each aberrated subimage, a restoration method that uses the respective aberrated subimage and the corresponding beamlet-specific aberration information as inputs, and generating a restored image from the plurality of restored subimages.

[0070] In this way, spatial information that is dispersed across the multi-field-of-view due to beamlet-specific aberrations, in particular defocus caused by field curvature and astigmatism introduced by beam splitting, is restored individually for each beamlet, even when the system is operated with global focus or global stigmation settings. This leads to a substantially improved uniformity of image quality across the multi-beam field-of-view without requiring local hardware-based aberration correction for each beamlet. Thereby, limitations associated with hardware scalability, manufacturing complexity, alignment effort, and robustness are avoided. As a result, the number of usable beamlets and beamlet shells can be increased while maintaining high and uniform resolution, enabling higher-throughput imaging and improved defect sensitivity in multi-beam charged particle imaging systems.

[0071] According to an example, the plurality of primary beamlets is grouped into at least two groups, and the restoration method is applied using group-specific aberration information. This leads to a reduction in computational complexity while still compensating systematic aberration differences between different groups of beamlets.

[0072] According to a further example, the groups correspond to different shells of the plurality of primary beamlets. Thereby, shell-dependent aberrations inherent to multi-beam charged particle optical systems are compensated in a physically meaningful manner.

[0073] According to an example, the restoration method is applied independently for each primary beamlet. As a result, beamlet-specific aberrations are compensated with high accuracy, enabling highly uniform image quality even for a large number of beamlets.

[0074] According to an example, the method further comprises controlling one or more electro-optical components of the multi-beam charged particle imaging system to partially compensate aberrations prior to restoring the image. In this way, coarse aberrations are reduced by hardware-based correction, while residual and beamlet-specific aberrations are compensated by image restoration.

[0075] According to an example, the restoration method restores spatial frequency information beyond a resolution limit of the aberrated subimages using known or constrained object information. This allows spatial detail beyond the nominal resolution limit to be inferred from the acquired image data.

[0076] According to an example, the detector comprises a scintillator, and the beamlet-specific point aberration information is obtained from a scintillator image acquired by the detector. Thereby, beamlet-specific aberration information is measured directly, in parallel, and at a common sample position.

[0077] According to an example, the restoration method comprises a deconvolution algorithm that uses the beamlet-specific point spread function as a kernel for deconvolving the respective aberrated subimage. In this way, image restoration is performed using physics-based system information, leading to a restoration of high accuracy.

[0078] According to an example, the restoration method comprises a constrained deconvolution algorithm configured to estimate a beamlet-specific point spread function while simultaneously restoring the subimage, wherein the obtained beamlet-specific aberration information is used to provide one or more constraints for estimating the point spread function. This leads to adaptive image restoration while remaining constrained by measured system characteristics, in particular in case the measured point spread function or its parameters are only approximations of the true point spread function.According to an embodiment of the invention, the restoration method comprises a machine learning image restoration model configured to map an aberrated subimage and corresponding beamlet-specific aberration information to a restored subimage. In this way, complex and non-linear aberration effects are compensated without explicit optical modeling. In addition, highly accurate restorations are obtained in a low computation time.

[0079] According to an example, the beamlet-specific aberration information is provided to the machine learning image restoration model as an additional input channel or as conditioning information. Thereby, the machine learning image restoration model adapts its restoration behavior to the current aberration state of the respective beam-let.

[0080] According to an example, the machine learning image restoration model comprises a convolutional neural network, a residual neural network, or a neural network comprising at least one attention mechanism. This enables efficient extraction of object-relevant features while suppressing aberration-induced artifacts.

[0081] According to an example, the convolutional neural network comprises an encoder–decoder architecture. As a result, object information is encoded into a compact representation and reconstructed with high fidelity. At the same time, only the most relevant information is preserved by the network, while redundant information and noise are removed, leading to an efficient and accurate image restoration.

[0082] According to an example, the restoration method comprises a combination of a deconvolution algorithm and a machine learning image restoration model, wherein the machine learning image restoration model maps an aberrated subimage deconvolved using the beamlet-specific point spread function to a restored subimage. In this way, physics-based deconvolution and data-driven learning are combined, leading to highly accurate image restorations.

[0083] According to an embodiment of the invention, a computer implemented method for training a machine learning image restoration model for restoring images in a multibeam charged particle imaging system is provided. The method comprises acquiring training data including aberrated subimages and corresponding beamlet-specific aberration information, providing corresponding reference images representing restoredimage information, and training the machine learning image restoration model using the training data. In this way, the trained model incorporates system-specific aberration characteristics, thereby achieving highly accurate image restorations.

[0084] According to an example, the reference images are generated from images acquired using different imaging conditions of the multi-beam charged particle imaging system. This allows high-quality reference information to be obtained without additional hardware components.

[0085] According to an example, the beam let-specific aberration information is provided to the machine learning image restoration model as an additional input channel or as conditioning information during training. Thereby, the trained model learns to explicitly account for system-specific aberrations.

[0086] According to an example, the plurality of primary beamlets is grouped into at least two groups, and the machine learning image restoration model is trained using groupspecific training data. This leads to efficient training while preserving compensation of group-dependent aberrations.

[0087] According to a further example, the groups correspond to different shells of the plurality of primary beamlets. Thereby, shell-dependent aberration characteristics are incorporated into the training process.

[0088] According to an example, a single machine learning image restoration model is trained for restoring images corresponding to a plurality of primary beamlets. In this way, a common model can be reused across multiple beamlets.

[0089] According to a further example, separate machine learning image restoration models are trained for different primary beamlets or different groups of primary beamlets. As a result, beamlet-specific or group-specific restoration behavior is achieved.

[0090] According to an example, the machine learning image restoration model is trained using a combination of deconvolution-based restoration and machine learning-based refinement as described above. Thereby, physics-based priors and data-driven learning are jointly exploited during training.According to an embodiment of the invention, a computer implemented method for generating training data for training a machine learning image restoration model configured for image restoration in a multi-beam charged particle imaging system is provided. The method comprises acquiring, for at least one primary beamlet (preferably for at least two, at least three, or multiple primary beamlets), a reference subimage and corresponding beamlet-specific aberration information, introducing a predefined aberration in the system, acquiring an aberrated subimage and corresponding aberration information, and associating the aberrated subimage with the reference subimage as a training data pair. In this way, realistic and system-specific training data is generated in a controlled manner.

[0091] According to an example, the generated training data is used for training a machine learning image restoration model as described above. This ensures consistency between training data generation and model training.

[0092] According to an embodiment of the invention, a computer program is provided that, when executed by a data processing system, causes the data processing system to carry out a method for training a machine learning image restoration model and / or generating training data as described above. In this way, the invention can be implemented in software and deployed flexibly.

[0093] According to an embodiment of the invention, a computer-readable medium is provided, storing instructions which, when executed by a data processing system, cause the data processing system to carry out a method for training a machine learning image restoration model and / or generating training data as described above. Thereby, the invention can be distributed, stored, and reused efficiently.

[0094] According to an embodiment of the invention, a multi-beam charged particle imaging system is provided, comprising means for generating a plurality of primary beamlets for scanning a sample, a detector, one or more processing devices, and one or more machine-readable hardware storage devices storing instructions that, when executed, perform a method for restoring an image as described above. In this way, image restoration can be integrated directly into the imaging system.According to an embodiment of the invention, a method is provided comprising controlling one or more electro-optical components of a multi-beam charged particle imaging system based on determined state parameters, and subsequently restoring the image according to a method for restoring an image as described above. Thereby, hardware-based calibration and software- based image restoration are combined into a robust and scalable correction approach.

[0095] The invention described by examples and embodiments is not limited to the embodiments and examples but can be implemented by those skilled in the art by various combinations or modifications thereof.

[0096] Brief Description of the Drawings

[0097] Fig. 1 is a schematic illustration of a multi-beam charged particle imaging system according to an embodiment of the invention;

[0098] Fig. 2a, b illustrate a scanning operation of the plurality of primary beamlets during an image acquisition;

[0099] Fig. 3 shows a detector comprising an electron to photon conversion unit in the form of a scintillator arranged in the focal plane of the plurality of secondary beam focus points;

[0100] Fig. 4 shows a flowchart of a method for determining state parameters of a multibeam charged particle imaging system according to an embodiment of the invention;

[0101] Fig. 5a, b show a multi-beam charged particle imaging system image and a subimage thereof that is analyzed to obtain state parameters of the multibeam charged particle imaging system, in particular of the respective primary beamlet;

[0102] Fig. 6a – d shows images and exemplary subimages that are analyzed to obtain state parameters of the multi-beam charged particle imaging system, in particular of the primary beamlets;Fig 7a shows an image comprising a point spread function for each primary beamlet of the multi-beam charged particle imaging system;

[0103] Fig. 7b, c illustrate the extraction of characteristics of a point spread function generated by a secondary beamlet that is emitted from a sample in response to its illumination by a primary beamlet;

[0104] Fig. 8 shows a flowchart of an iterative method for optimizing state parameters from characteristics of point spread functions;

[0105] Fig. 9a, b illustrate an inference step of the trained machine learning model for an input comprising a single subimage or multiple subimages;

[0106] Fig. 10 illustrates a residual neural network for mapping at least one subimage to one or more state parameters of a multi-beam charged particle imaging system;

[0107] Fig. 11a illustrates an exemplary implementation of an attention mechanism in a machine learning model;

[0108] Fig. 11b illustrates a T ransformer block comprising multiple attention mechanisms;

[0109] Fig. 12 illustrates a flowchart of a method for predicting state parameters using a machine learning model and additionally a confidence score;

[0110] Fig. 13 shows subimages of an image that are spatially grouped in four different groups;

[0111] Fig. 14 illustrates an exemplary method for generating training data and training a machine learning model for mapping a subimage of an image generated by a primary beamlet in a multi-beam charged particle imaging system to one or more state parameters of the multi-beam charged particle imaging system, in particular of the primary beamlet, according to the invention;

[0112] Fig. 15 shows training data for training a machine learning model for state parameter prediction, the training data comprising various subimages of multi-beam charged particle imaging system images and corresponding state parameters;

[0113] Fig. 16 shows training data for training a machine learning model for state parameter prediction, the training data comprising various subimages of images and corresponding state parameters;

[0114] Fig. 17 illustrates a computer implemented method for training a plurality of group-specific machine learning models by first training a prototype machine learning model using subimages of images generated by various primary beamlets and subsequently fine-tuning the prototype machine learning model using group-specific training data for each group of primary beamlets;

[0115] Fig. 18a-c illustrates accuracy results of the method for determining one or more state parameters of a multi-beam charged particle imaging system;

[0116] Fig. 19 illustrates a multi-beam charged particle imaging system according to an embodiment of the invention;

[0117] Fig. 20 illustrates non-uniform image quality across a multi-field-of-view of a multibeam charged particle imaging system when operated with global optical settings;

[0118] Fig. 21 illustrates the dependence of information dispersion on shell number within a multi-field-of-view, showing contributions of different aberrations;

[0119] Fig. 22 shows a hardware-based approach for compensating field-curvature-in- duced focus offsets by adding correction hardware;

[0120] Fig. 23 shows a hardware-based approach for compensating astigmatism by adding stigmation correction hardware;

[0121] Fig. 24 illustrates scalability limitations of hardware-based aberration correction approaches with increasing numbers of beam paths;Fig. 25 shows a flowchart of a method for image restoration in a multi-beam charged particle imaging system according to an embodiment of the invention;

[0122] Fig. 26 illustrates an image restoration method based on deconvolution using measured point spread functions;

[0123] Fig. 27 illustrates a constrained deconvolution method in which constraints are derived from measured point spread functions;

[0124] Fig. 28 illustrates an image restoration method based on a machine learning image restoration model;

[0125] Fig. 29 shows an encoder–decoder based machine learning image restoration model architecture;

[0126] Fig. 30 shows a flowchart of a method for training a machine learning image restoration model;

[0127] Fig. 31 shows a flowchart of a method for generating training data for training a machine learning image restoration model;

[0128] Fig. 32 illustrates exemplary training data configured to train a machine learning image restoration model;

[0129] Fig. 33 illustrates a hybrid method combining hardware-based aberration compensation and image-domain restoration;

[0130] Fig. 34 illustrates improvements in image resolution and uniformity across a multi- field-of-view achieved using a method according to the invention;

[0131] Fig. 35 illustrates resolution as a function of shell number before and after image restoration; and

[0132] Fig. 36 illustrates a combined control and restoration method integrating state parameter determination, hardware control, and image restoration.Detailed Description

[0133] In the following, advantageous exemplary embodiments of the invention are described and schematically shown in the figures. Throughout the figures and the description, same reference numbers are used to describe same features or components. Components similar in function and structure are indicated as far as possible by similar or identical reference numerals. Some array elements, for example the plurality of primary beamlets, are identified by a reference number. Depending on the context, the same reference number may also identify a single element out of the array elements. Each primary beamlet (3.1, 3.2, 3.3) is, for example, one beamlet of the plurality of primary beamlets (3). Dashed lines indicate optional features.

[0134] A "multi-beam charged particle imaging system" refers to any instrument that utilizes a plurality of charged particle beams, scanning in parallel, to acquire an image of a sample. The charged particles may be electrons or ions. A primary example of such a system according to the invention is a multi-beam scanning electron microscope (MultiSEM), which is used for high-throughput inspection of samples such as semiconductor wafers or biological specimens.

[0135] WO 2005 / 024881 A2 discloses a multi-beam charged particle imaging system in the form of a multi-beam electron microscope system which operates with a multiplicity of electron beamlets for the parallel scanning of a sample to be inspected with a bundle of electron beamlets. The bundle of primary electron beamlets is generated by directing a primary charged particle beam onto a multi-beam forming unit, comprising at least one multi-aperture plate, which has a multiplicity of openings. One portion of the electrons of the electron beam is incident onto the multi-aperture plate and is absorbed there, and other portions of the beam transmit the plurality of openings of the multi-aperture plate. Thereby, in the beam path downstream of each opening, a plurality of primary electron beamlets are formed whose cross section is defined by the cross section of the corresponding openings. The plurality of primary electron beam-lets are focused by an objective lens on a surface of a sample and interaction volumes of the primary electrons within the sample are formed at the plurality of focus points.Secondary electrons or backscattered electrons emanate from the interaction volumes. Thereby, a plurality of secondary electron beamlets are emitted from the sample, collected, and imaged onto a detector. Each of the secondary beamlets is incident onto separate detector elements, so that the secondary electron intensities detected therewith provide information relating to the surface of the sample at the location where the corresponding primary beamlet is incident onto the sample. The bundle of primary electron beamlets is scanned systematically over the surface of the sample and an electron microscopic image of the sample is generated. Alternatively, the sample is continuously moved by a sample stage and a scanning image is acquired.

[0136] Fig. 1 is a schematic illustration of a multi-beam charged particle imaging system 1 according to an embodiment of the invention. The multi-beam charged particle imaging system 1 uses a plurality of charged particle beamlets for forming an image of a sample 7. The multi-beam charged particle imaging system 1 generates a plurality of J primary particle beamlets 3 which strike the sample 7 to be examined in order to generate interaction products, e.g. secondary electrons, which emanate from the sample 7 and are subsequently detected. The multi-beam charged particle imaging system 1 is of the scanning electron microscope (SEM) type, which uses a plurality of primary electron beamlets 3 which are incident on a surface of the sample 7 at a plurality of locations and generate there a plurality of primary electron beam focus spots 5, that are spatially separated from one another. The sample 7 to be examined can be of any desired type, e.g., a semiconductor wafer or a semiconductor mask, and can comprise an arrangement of miniaturized elements. The surface 25 of the sample 7 is arranged in a sample plane 101 of an objective lens 102 of a sample irradiation unit 100.

[0137] A diameter of the minimal beam spots or focus spots 5 shaped in the sample plane 101 can be small. Exemplary values of this diameter are below four nanometers, for example three nm or less. The focusing of the primary charged particle beamlets 3 for shaping the focus spots 5 is carried out by the objective lens system 102. In this case, the objective lens system 102 can comprise a magnetic immersion lens. Further examples of focusing means are described in the German patent DE 102020125534 B3, the entire content of which is herewith incorporated in the disclosure.The plurality of focus spots 5 of the primary beamlets 3 form a regular raster arrangement of incidence locations, which are formed in the sample plane 101. The number J of primary beamlets 3 may be five, twenty-five or more. In practice, the number J of beamlets 3, and hence the number of incidence locations or focus spots 5, can be chosen to be significantly greater, such as, for example, J = 10 x 10, J = 20 x 30 or J = 100 x 100. Exemplary values of the pitch P between the incidence locations are 1 micrometer, 10 micrometers, or more, for example 40 micrometers. For sake of simplicity, only three primary beamlets 3.1, 3.2 and 3.3 with corresponding focus points 5.1, 5.2 and 5.3 are shown in Fig 1.

[0138] The primary particles 3 striking the sample 7 generate interaction products, e.g. secondary electrons, back-scattered electrons, which emanate from the surface of the sample 7. The interaction products emanating from the surface of the sample 7 are shaped by the objective lens 102 to form secondary electron beamlets 9. For sake of simplicity, through the disclosure, all the interaction products are collectively described as secondary electrons, forming secondary electron beamlets 9.

[0139] The multi-beam charged particle imaging system 1 provides a detection beam path 13 for guiding the plurality of secondary particle beamlets 9 to a secondary electron imaging system or detection unit 200. The secondary electron imaging system 200 comprises several electro-optical lenses 205.1 to 205.5 for directing the secondary particle beamlets 9 towards a spatially resolving particle detector 600. The detector 600 is arranged in the image plane 225. The detector 600 is comprising a plurality of detection elements. Detection elements can for example be diodes such as PMDs, or CMOS or CCD detection elements, provided with electron-to-light conversion elements, or can be formed as direct electron detection elements. In an example, the detector 600 comprises an electron-to-light conversion element, such as a scintillator 602, by which secondary electrons are converted into light, and a plurality of light detection elements. The combination of the electron-to-light conversion element and the plurality of light detection elements hereby form together a plurality of electron detection elements. In addition to a scintillator 602, the detector may comprise a scintillator image acquisition unit 604 configured to acquire an image of the scintillator, in particular during the emission of visible light. The detector can be used to detect charged particles, photons, ions, myons, etc.The imaging with the secondary electron imaging system 200 is strongly magnifying such that both the raster pitch of the primary beamlets on the wafer surface and the size and shape of focal points of the primary beamlets 3 are imaged in much magnified fashion. By way of example, a magnification is between 10x and 100x such that one nm on the wafer surface is imaged enlarged to between 10 nm and 100 nm. By way of example, a magnification is between 100x and 300x such that one nm on the wafer surface is imaged enlarged to between 100 nm and 300 nm. In an example, an image field of a multi-beam charged particle imaging system with for example 100 µm diameter is enlarged to approximately 30 mm.

[0140] The primary particle beamlets 3 are generated in a beam generation apparatus 300 comprising at least one charged particle emitter 301, at least one collimation lens 303, a multi-aperture arrangement 305 and a first field lens 331 and a second field lens 333. The charged particle emitter 301 is connected to a voltage supply for providing an emitter voltage VK to the emitter 301 and generates at least one diverging charged particle beam 309, which is at least substantially collimated by the at least one collimation lens 303, and which illuminates the multi-aperture arrangement 305. The multiaperture arrangement 305 comprises at least one first multi-aperture or filter plate 304, which has a plurality of J openings formed therein in a first raster arrangement. Particles of the illuminating particle beam 309 pass through the J apertures or openings of the first multi-aperture plate 304 and form the plurality of J primary beamlets 3. Particles of the illuminating beam 309 which strike the first aperture plate 304 are absorbed by the latter and do not contribute to the formation of the primary beamlets 3. A multi-aperture arrangement 305 usually has at least a further multi-aperture plate 306, for example a lens array, a stigmator array or an array of deflection elements. In this example, the particle beam 309 is perfectly collimated by collimation lens 303. However, it is also possible to design the multi-aperture arrangement 305 for a diverging or converging incident particle beam 309.

[0141] Together with the first field lens 331 and a second field lens 333, the multi-aperture arrangement 305 focuses each of the primary beamlets 3 in such a way that focal points are formed in an intermediate image surface 321. Alternatively, the beam foci and the intermediate image surface 321 can be virtual. The intermediate image surface 321 can be curved and tilted to pre-compensate a field curvature and image plane tilt of the charged particle imaging system arranged downstream of the intermediate image surface 321.The at least one field lens 103 and the objective lens 102 provide a first imaging particle optical unit for imaging the surface 321, in which the beam foci are formed, onto the sample plane 101 such that a second raster configuration of focus spots 5 of the primary beamlets is formed there. Typically, the surface 25 of the sample 7 is arranged in the sample plane 101, and the focal points 5 are correspondingly formed on the sample surface 25. The plurality of primary beamlets 3 form a crossover point 108, in the vicinity of which a first scanning deflector 110 is arranged. The first scanning deflector 110 is used to deflect the plurality of primary beamlets 3 collectively and synchronously such that the plurality of focus spots 5 are moved simultaneously over the surface 25 of the sample 7. The first scanning deflector 110 is driven by a scanning control unit 860 such that in an inspection mode of operation, a plurality of two-dimensional image data of the surface is acquired. Additionally, the multi-beam charged particle imaging system 1 can comprise further static deflectors and multipole elements 112 configured to adjust the position and beam shapes of the plurality of the primary charged particle beamlets 3.

[0142] The objective lens 102 and the projection lenses 205 provide a secondary electron imaging system 200 for imaging the sample plane 101 onto the detection plane 225. The objective lens 102 is thus a lens or a lens system that is part of both the first and the second particle optical unit, while the field lenses 103, 331 and 333 belong only to the first particle optical unit 100, and the projection lenses 205 belong only to the secondary electron imaging system 200.

[0143] A beam divider 400 is arranged in the beam path of the first particle optical unit 100 between the field lens 103 and the objective lens system 102. The beam divider 400 is also part of the second optical unit in the beam path between the objective lens system 102 and the projection lenses 205.

[0144] The first scanning deflector 110 is arranged in a primary electron beam path or in a joint electron beam path. In the example shown in Fig. 1, the secondary electron beamlets 9 traverse during use the first scanning deflector 110 in opposite direction and the scanning movement of the secondary beamlets 9 is partially compensated for. The secondary electrons have typically a different kinetic energy compared to the primary electrons. Therefore, the scanning movement of the moving irradiation positions is only partially compensated. The secondary electron imaging system 200,therefore, comprises the second, collective beam deflector 222 which is arranged in the vicinity of a crossover plane or pupil plane 21a of the secondary electron beamlets 9. The second, collective beam deflector 222 is operated synchronously with the first beam deflector 110 and compensates during use a beam deflection of the secondary electron beamlets 9 such that the focus points 15 of the secondary beamlets 9 remain at constant position on the detection plane 225. Thereby, each focus point 15 of each individual secondary beamlet 9 is kept within the area of a set of detection elements, which is assigned to the individual secondary beamlet 9.

[0145] Together with the objective lens 102, the lenses 205 serve to focus the secondary beamlets 9 on the spatially resolving detector 600 and, in the process, compensate the imaging scale and the twist of the plurality of secondary electron beamlets 9 as a result of a magnetic lens such that a third raster arrangement of the focus points 15 of the plurality of secondary electron beamlets 9 remains constant on the detection plane 225. The electron-optical lenses 205.1 to 205.5 are shown as magneto-optical elements but are not limited to magneto-optical elements and can comprise also electro-static lens elements or stigmators. The secondary electron imaging system 200 further comprises exchangeable contrast apertures 284a, 284b, mounted on an aperture filter module 214 at a pupil plane 21 of the secondary electron imaging system 200. With an exchange mechanism (not shown), different aperture stops 284a or 284b can be positioned in the second pupil plane 21 b and aligned with respect to the optical axis 2105 of the secondary electron imaging system 200.

[0146] Further information relating to such multi-beam charged particle imaging systems and components used therein, such as, for instance, particle sources, multi-aperture plate and lenses, can be obtained from the international patent applications WO 2005 / 024881, WO 2007 / 028595, WO 2007 / 028596, WO 2011 / 124352 and WO 2007 / 060017 and the German patent applications having the publication numbers DE 102013 016 113 A1 and DE 102013 014976 A1, the disclosure of which in the full scope thereof is incorporated by reference in the present application.

[0147] The multi-beam charged particle imaging system 1 furthermore comprises a control unit 800 configured both for controlling the individual optical components of the multibeam charged particle imaging system and for evaluating and analyzing the signals obtained by the detector 600. In this case, the control or controller unit 800 can beconstructed from a plurality of individual electronic computers or electronic components. By way of example, the control unit 800 comprises a control operation processor 880, a control module 840 for the control of the electron-optical elements of the secondary electron imaging system 200 and a control module 830 for the control of the electron-optical elements of the primary beamlet generation unit 100. The control unit 800 further comprises a stage control module 850 for positioning the sample surface 25 or sample 7 by stage 500 within the sample plane 101. The control unit 800 further comprises a control module to adjust a sample voltage VS, which is connected to a module 503 for supplying the sample voltage VS to the sample 7, said sample voltage VS also being referred to as extraction voltage. Thereby, during use, an extraction field is generated between the objective 102 and the surface 25 of the sample 7. During use, the extraction field decelerates the primary charged particles of the primary beamlets 3 before the sample surface 25 is reached and generates an additional focusing effect on the plurality of primary beamlets 3. At the same time, the extraction field serves during use to accelerate the secondary particles out of the surface 25 of the sample 7.

[0148] Further, the control unit 800 comprises the scanning control module 860. During an inspection mode of operation, a plurality of focus points 15 of secondary electron beamlets is formed in the detection plane 225, and a plurality of signals is recorded during the scanning operation of the primary beamlets 3 over the surface 25 of the sample 7. The detector 600 comprises a plurality of sets of detection elements with one set of detection elements for each secondary electron beamlet 9. During use, each set of detection elements is configured to record the intensity signal of the assigned secondary electron beamlet 9. The plurality of intensity signals for the plurality of secondary electron beamlets 9 is transferred to the image data acquisition unit 810, where the image data is processed and stored in memory 890. The setup of the secondary electron optical imaging system 200, the detector 600, and the assignment of the sets of detection elements to the focus spots 15 of the secondary electron beam-lets 9 is initially determined and stored in the memory 890 of the control unit 800 of the multi-beam charged particle imaging system 1.

[0149] According to the example of Fig. 1, the multi-beam charged particle imaging system 1 further comprises a retractable monitoring system 230, which can be inserted into the secondary electron beam path in front of the detection plane 225. The monitoringsystem 230 comprises further imaging elements and a high-resolution detector. The monitoring system 230 is connected to a monitoring control unit 820.

[0150] During an image scanning operation step, the control unit 800 is configured to trigger the image sensor 600 to detect in predetermined time intervals a plurality of timely resolved intensity signals from the plurality of secondary electron beamlets 9, and the digital image of an image patch is accumulated and stitched together from all scan positions of the plurality of primary charged particle beamlets 3.

[0151] Fig. 2a, b illustrates a scanning operation of the plurality of primary charged particle beamlets 3 during an image acquisition. The scanning operation control module 860 is configured to provide during use a scanning signal to a scanning deflector 110. Thereby, each primary charged particle beamlet 3 is deflected by the collective multibeam raster scanner 110 such that the corresponding focus spot 5.i is scanned over an image patch 245. i of a single beamlet (Fig. 2a). Each image patch 245. i has a diameter AP of for example 8pm to 10pm. The scanning operation comprises a scanning of a plurality of parallel image scanning lines 241 along scanning direction 143.1 for image acquisition. At the end of each image scanning line 241, each beamlet 3 is moved back to the starting position of a next scanning line, which is also called “flyback” 243. During image acquisition along image scanning lines 241, the scanning operation is controlled to achieve a dwell time of about 50 ns at each image point, with for example 8000 image points per image scanning line 241. The time for flyback 243 can be much shorter, for example 20ns in total. Fig. 2b shows the parallel operation of a plurality of primary charged particle beamlets 3 to acquire an image of a surface area segment 251 of a wafer surface, consisting of a plurality of image patches 245.

[0152] The detector or image sensor 600 comprises a plurality of detector pixels or individual detectors. For each of the plurality of secondary charged particle beam focus points 15, the intensity is detected separately, and the property of the sample surface 25 is detected with high resolution for a large image patch of the sample 7 with high throughput. For example, with a raster of 10 x 10 beamlets with 8pm pitch, an image patch of approximately 88pm x 88pm is generated with one image scan with collective multi-beam raster scanner 110. The image patch is sampled with a pixel spacing of for example 1nm, thus with a pixel number of 8000 pixels per image line for eachbeamlet, such that the image patch generated by 100 beamlets comprises 6.4 gigapixel. The digital image data is collected by control unit 800. Details of the digital image data collection and processing, using for example parallel processing, are described in international patent application WO 2020 / 151904 A2 and in US-Patent US 9,536,702, which are hereby incorporated by reference.

[0153] The image sensor 600 is configured by an array of sensing areas in a pattern compatible to the raster arrangement of the secondary electron beamlets focused by the detection unit 200 onto the image sensor 600. This enables a detection of each individual secondary electron beamlet independent from the other secondary electron beamlets incident on the image sensor 600. The image sensor 600 illustrated in Fig.

[0154] 1 can be an electron sensitive detector array such as a CMOS or a CCD sensor. Such an electron sensitive detector array can comprise an electron to photon conversion unit, such as a scintillator element or an array of scintillator elements. In another embodiment, the image sensor 600 can be configured as electron to photon conversion unit or scintillator plate arranged in the focal plane of the plurality of secondary charged particle beam focus points 15. An example is shown in Fig. 3.

[0155] The image sensor 600 can further comprise a relay optical system comprising collection lenses 605 and zoom lens 611 for imaging and guiding the photons generated by the electron to photon conversion unit 602 at the secondary charged particle beam focus points 15 on dedicated photon detection elements 623, such as a plurality of photomultipliers or avalanche photodiodes. Such an image sensor is disclosed in US 9,536,702, which is cited above and incorporated by reference. The image sensor is further configured with an optionally extractable monitoring system 230, comprising a beam divider mirror 237, an imaging lens 235 and a CMOS sensor 232 with high resolution.

[0156] During an acquisition of an image patch by scanning the plurality of primary charged particle beamlets 3, the stage 500 is preferably not moved, and after the acquisition of an image patch, the stage 500 is moved to the next image patch to be acquired. In an alternative implementation, the stage 500 is continuously moved in a second direction while an image is acquired by scanning of the plurality of primary charged particle beamlets 3 with the collective multi-beam raster scanner 110 in a first direction. Stage movement and stage position is monitored and controlled by sensorsknown in the art, such as Laser interferometers, grating interferometers, confocal micro lens arrays, or similar.

[0157] During an image scan, the control unit 800 is configured to trigger the image sensor 600 to detect in predetermined time intervals a plurality of timely resolved intensity signals from the plurality of secondary electron beamlets, and the digital image of an image patch is accumulated and stitched together from all scan positions of the plurality of primary charged particle beamlets 3.

[0158] The control unit 800 of the multi-beam charged particle imaging system 1 further comprises an imaging control module 810, configured to receive the data streams from the image sensor 600 and to generate a digital image of the surface 25 of the sample 7 during operation.

[0159] Next to the electron to photon conversion unit, e.g., the scintillator, a scintillator image acquisition unit 604 may be arranged. The scintillator image acquisition unit 604 is configured to acquire an image of the scintillator, in particular during the emission of visible light.

[0160] According to an embodiment of the invention illustrated in Fig. 4, a method 27 for determining one or more state parameters of a multi-beam charged particle imaging system comprising means for generating a plurality of primary beamlets in a raster configuration for the parallel scanning of a sample, and a detector arranged in or close to a focal plane of a plurality of secondary beamlets emitted from the sample, comprises: generating, by irradiation of a sample with a plurality of primary beamlets, a plurality of secondary beamlets that are emitted from the sample in a step P1; for each of at least one primary beamlet: generating a subimage of an image from a signal recorded by the detector for a corresponding secondary beamlet generated in response to an irradiation of the sample with this primary beamlet in a step P2, and determining one or more state parameters of this primary beamlet by analyzing the respective at least one generated subimage using a machine learning model trained to map at least one subimage of an image generated by at least one primary beamlet in a multi-beam charged particle imaging system or characteristics thereof to one or more state parameters of the multi-beam charged particle imaging system in a step P3.According to an example, in the final method step, the one or more state parameters of a primary beamlet are determined from the respective generated subimage.

[0161] The "means for generating a plurality of primary beamlets" refers to the electro-optical assembly responsible for creating the array of individual beamlets from a single source beam. This typically comprises a charged particle emitter, a collimation lens, and a multi-aperture arrangement, such as one or more aperture plates, which divides the source beam into the plurality of primary beamlets.

[0162] Fig. 5a shows a multi-beam charged particle imaging system image 30 obtained by a multi-beam charged particle imaging system 1. The image 30 is generated by scanning the plurality of primary beamlets over the sample in parallel, recording a plurality of timely resolved intensity signals provided by the detector and combining the intensity signals from all scan positions of each primary beamlet to form a different subimage 34 of the image 30, as shown exemplarily in Fig. 5b. Thus, each subimage requires multiple illuminations of the sample. From each subimage 34, one or more state parameters of the respective primary beamlet that was scanned over a portion of the sample to obtain this subimage 34 can be determined.

[0163] The term "state parameters" refers to a set of quantitative values that describe the operational condition or optical properties of the multi-beam charged particle imaging system or its individual beamlets. These parameters particularly include, but are not limited to, optical aberration parameters such as focus parameters, astigmatism parameters (e.g., magnitude and orientation), and higher-order aberrations (e.g., coma, spherical aberration). They may also include beamlet current, beamlet intensity, and parameters describing the position or shape of a beamlet's point spread function, or sample charging. According to an example, state parameters of a primary beamlet are from the group comprising focus parameters, astigmatism parameters (e.g., chromatic or spherical astigmatism), higher order aberration parameters, pitch parameters, beamlet current parameters, beamlet intensity parameters, point spread function parameters, illumination field curvature parameters, illumination field inclination parameters, stigmation parameters.

[0164] Alternatively, a detector image 32, 32’, as shown in Fig. 6a and 6c, can be used to derive state parameters of the multi-beam charged particle imaging system, in particular of the respective primary beamlet. According to an embodiment, the multi-beamcharged particle imaging system comprises a scintillator image acquisition unit 604 (shown in Fig. 3). The scintillator image acquisition unit is configured for acquiring a detector image 32, 32’ of the scintillator during the emission of visible light due to its excitement by a plurality of secondary beamlets. A subimage of a scintillator image, thus, requires only a single illumination of the sample. From a subimage 36, 36’ shown in Fig. 6b and 6d of the detector image 32, 32’ state parameters of the multi-beam charged particle imaging system, in particular of the respective primary beamlet, are derived. As only a single illumination of the sample is required to acquire a detector image, the computation time for aligning the different primary beamlets is drastically reduced.

[0165] The detector that may comprise an electron to photon conversion unit, e.g., a scintillator, is arranged in or close to a focal plane of the plurality of secondary beamlets emitted from the sample. Thus, an image 32, 32’ acquired by the detector, e.g., a scintillator image acquisition unit, for a single illumination of the sample with a plurality of primary beamlets shows a cross section of each secondary beamlet emitted from the sample. The cross section of a secondary beamlet is also referred to as a point spread function 38 of the secondary beamlet. The point spread functions 38 are arranged in a hexagonal pattern that corresponds to the arrangement of the primary beamlets. This arrangement follows the geometry of the round lenses to reduce aberrations. The image can, for example, comprise 91 point spread functions generated by an illumination of the sample using 91 primary beamlets.

[0166] Different point spread functions 38 of secondary beamlets are shown in Fig. 6b and 6d. To obtain high quality images, one aims for focused, bright, radially symmetric point spread functions 38 with minimum radius. The point spread function 38 in Fig.

[0167] 6b is bright, almost radially symmetric and of a small radius. It, thus, results in a subimage 36 of a multi-beam charged particle imaging system image of higher quality than the point spread function in Fig. 6d, which is not radially symmetric and much darker. The state parameters of the primary beamlets used to generate the image 32 in Fig. 6a are, thus, better suited to generate high quality images than the state parameters of the primary beamlets used to generate the image 32’ in Fig. 6c.

[0168] According to an embodiment, the image is generated during a single irradiation of the sample with the plurality of primary beamlets. Thus, each primary beamlet generates a single point spread function in the image, from which state parameters of the primarybeamlet or of the whole multi-beam charged particle imaging system can be derived. This procedure is particularly fast, since the state parameters of the primary beamlets can be derived from a single illumination of the sample instead of a complete image that requires raster scanning of the sample.

[0169] Instead of a single irradiation, a few irradiations can also be used. During the irradiations of the sample, the same portions of the sample can be irradiated or different portions of the sample can be irradiated. By using a few irradiations of the sample, imaging effects such as varying brightness, contrast or noise or a varying sample topology can be reduced without notably increasing the acquisition or computation time.

[0170] According to another embodiment, each subimage of the image is generated by averaging the detector signal while scanning the respective primary beamlet of the plurality of charged particle beamlets over the sample. In this way, the point spread functions generated by a single secondary beamlet are averaged while scanning the respective portion of the sample. The averaging reduces the dependency of the point spread function from the topology of the sample surface and at the same time improves the quality of the image by reducing noise, contrast or brightness variations. The averaging of the point spread functions of a single secondary beamlet can be carried out over two, several, multiple or all of the scan points of the respective primary beamlet. In this way, a tradeoff can be achieved between the quality of the averaged point spread function and the computation time required for determining the state parameters of the multi-beam charged particle imaging system, in particular of the primary beamlets.

[0171] The term "imaging parameters" refers to the controllable settings of the electro-optical components of the system that influence the formation and trajectory of the charged particle beams. Examples include the voltage or current settings for lenses, deflectors, and stigmators, which are adjusted by a control unit to operate the system.

[0172] Instead of acquiring an image (or an averaged image) for a single setting of imaging parameters of the primary beamlets, two or more images can be acquired for different settings of imaging parameters of the primary beamlets. In this way, ambiguities can be resolved, e.g., ambiguities that result from point spread functions that are charac-teristic for different state parameters of the multi-beam charged particle imaging system, in particular of the respective primary beamlet. For example, the same point spread function can be characteristic for two different focus parameters of a primary beamlet (e.g., over focus and under focus). Thus, according to an embodiment, for each primary beamlet, two or more subimages of two or more images are generated in response to two or more irradiations of the sample with this primary beamlet using different imaging parameters, e.g., focus parameters, and wherein the one or more state parameters of the primary beamlet are determined by analyzing the respective two or more subimages.

[0173] The one or more state parameters of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet of the multi-beam charged particle imaging system, can be determined from the respective at least one generated subimage in different ways, e.g., by analyzing characteristics of the point spread functions.

[0174] The term "point spread function" or " PSF" refers to the two-dimensional intensity distribution of a single beamlet as recorded by the detector, representing the system's response to a point source. Characteristics or parameters derived from a point spread function refer to specific scalar or vector values calculated from the PSF. Such derived characteristics or parameters include, but are not limited to, the dimensions of the PSF (e.g., full-width at half-maximum), its ellipticity, its orientation, its total or peak intensity, moments of its intensity distribution (e.g., mean, variance, covariance), or coefficients from a mathematical fit to the PSF.

[0175] According to an embodiment illustrated in Fig. 7a to c, the characteristics of the at least one subimage used as input of the machine learning model comprise characteristics of the at least one point spread function of the respective at least one secondary beamlet within the respective subimage of the image. Fig. 7a shows an image comprising point spread functions 38 of a plurality of secondary beamlets generated in response to an illumination of a sample with a plurality of corresponding primary beamlets. The point spread functions show differing characteristics, e.g., different intensities and different shapes. For example, the point spread functions in the lower right corner are diagonally elongated, whereas the point spread functions in the center are of a lower intensity and rather circular. Spatially neighboring point spread functions often show similar characteristics. From these characteristics state parametersof the multi-beam charged particle imaging system, in particular of the respective primary beamlets, can be determined. This can be done using a machine learning model, but it can also be done in other ways without using a machine learning model.

[0176] Fig. 7b shows three secondary beamlets 9 that generate point spread functions 38. Fig. 7c illustrates characteristics of the point spread functions 38 from which state parameters of the multi-beam charged particle imaging system, in particular of the respective primary beamlets, are determined. The characteristics can, for example, comprise the diameter dltd2 of the point spread function in one or more dimensions. The characteristics can comprise parameters of a fitted function, e.g., moments of a fitted probability distribution such as moments of a Gaussian (mean, variance or covariance or higher order moments). The characteristics can comprise an intensity, an average intensity, a maximum intensity or an intensity distribution of the point spread function. The characteristics can comprise shape descriptors such as a measure for roundness, circularity, ellipticity, eccentricity, convexity, size, area, radius, diameter, centroid, principal axes, etc. The characteristics can also comprise parameters obtained using spectral shape analysis, Fourier analysis, Wavelet analysis, etc. The characteristics can also comprise measurements taken from two or more point spread functions 38, e.g., an average characteristic, an average distance between centers of point spread functions (to estimate pitch or stigmation parameters), or a function fitted to two or more point spread functions (to estimate illumination field curvature or illumination field inclination). Instead of or in addition to the at least one subimage of the image used as input of the machine learning model, one or more of these characteristics of the point spread functions can be used as input of the machine learning model.

[0177] Apart from machine learning models, to determine state parameters from characteristics of a point spread function, tables can be used to map one or more characteristics of point spread functions to corresponding state parameters of the multi-beam charged particle imaging system, in particular of the respective primary beamlet. Such a table can, for example, include a range of one or more diameters of a point spread function, an eccentricity and an average intensity to one or more state parameters of the respective primary beamlet. Such tables can be established by professionals or from example data or statistics.Alternatively, a function mapping point spread function characteristics to state parameters of the multi-beam charged particle imaging system, in particular of primary beamlets, can be obtained by fitting a function such as a polynomial, Bezier curve, Spline or a sequence of basis functions to example data. Example data can comprise tuples of one or more point spread function characteristics and one or more state parameters. They can be generated by selecting a set of state parameters in a multibeam charged particle imaging system and measuring characteristics of the resulting point spread functions in an image.

[0178] A trained machine learning model is used to obtain state parameters of a multi-beam charged particle imaging system, in particular of primary beamlets. The trained machine learning model can use at least one subimage and / or characteristics thereof as input. For example, a machine learning model can be trained that uses characteristics of a point spread function as input that are mapped to one or more state parameters of the multi-beam charged particle imaging system, in particular of the primary beam-let. The machine learning model can be trained using training data comprising characteristics of point spread functions and corresponding state parameters of the multibeam charged particle imaging system, in particular of primary beamlets, used to generate the point spread functions. Different machine learning models can be used here, for example, (linear) regression, k-means, support vector machines, decision trees, random forests, diffusion models, neural networks such as multi-layer perceptrons, transformers or deep learning approaches.

[0179] Machine learning models are trained using training data, i.e., examples, and, thus, independently derive their knowledge from the training data instead of requiring a user to define rules for state parameter determination. In this way, optimal results with respect to a loss function that is minimized during training can be obtained automatically in a data-driven way. Thus, the use of machine learning methods increases the quality of the predictions of the state parameters and at the same time reduces the required user effort.

[0180] An iterative method illustrated in Fig. 8 can be used to optimize the state parameters based on the point spread functions in the subimages. In an image acquisition step 64 an image comprising point spread functions is acquired and subimages comprising a single point spread function are extracted. In an analysis step 66, characteristics of the point spread functions are determined as described above. In a comparison step68, the characteristics are compared to corresponding characteristics of an optimal point spread function, e.g., an optimal width, an optimal intensity, an optimal distribution, etc. In case the determined characteristics are sufficiently close to the respective characteristics of the optimal point spread function, (“optimum reached” 70) the optimization is finished (“exit step” 72). Otherwise (“optimum not reached” 74), in a correction step 76, state parameters of the multi-beam charged particle imaging system, in particular of the respective primary beamlet, are controlled (corrected) depending on the determined characteristics of the point spread function. The state parameters can be corrected using, for example, electro-optical components of the multi-beam charged particle imaging system. Then the optimization process is repeated 78.

[0181] This iterative procedure can also be used in case of ambiguous point spread functions that can indicate different state parameters, for example different focus parameters (e.g., the same amount of over-focus and under-focus may lead to the same point spread function). Using two or more iterations, ambiguities can be resolved. For example, in case of ambiguous sets of state parameters, a first set can be selected and subsequently corrected using electro-optical components of the multi-beam charged particle imaging system. If the characteristics of the point spread function get worse, the other set of state parameters can be selected and the iterations started from here.

[0182] The term "electro-optical components" refers to any element within the charged particle optical column that uses electric or magnetic fields to influence the trajectory, focus, or shape of the charged particle beams. This includes, but is not limited to, lenses (magnetic and / or electrostatic), deflectors, stigmators, and multipole correctors.

[0183] Instead of characteristics of the point spread functions, the subimages themselves can be used to determine state parameters.

[0184] According to the invention, the one or more state parameters 42 of the multi-beam charged particle imaging system, in particular of each of the primary beamlets, are determined by analyzing the at least one respective subimage 34, 36 of the image using a machine learning model 40 trained to map at least one subimage 34, 36 of an image 30, 32 generated by at least one primary beamlet in a multi-beam charged particle imaging system or characteristics thereof to one or more state parameters 42 of the multi-beam charged particle imaging system, in particular of the at least oneprimary beamlet. Instead of or in addition to using the at least one subimage as input to the machine learning model, characteristics 39 can be extracted from the at least one subimage 34, 36 and used as input to the machine learning model. The at least one subimage 34, 36 of the image can be contiguous (neighboring) subimages of the image, or they can be from different parts of the image.

[0185] Fig. 9a, b illustrate an inference step of the trained machine learning model. An image (multi-beam charged particle imaging system image or detector image) is acquired and subimages 34, 36 are extracted for one or more primary beamlets. In an example, the machine learning model is trained to map a single subimage of an image generated by a single primary beamlet or characteristics thereof to one or more state parameters of the multi-beam charged particle imaging system, in particular of the primary beamlet. In Fig. 9a a single subimage 34, 36 is used as input of the machine learning model 40, whereas in Fig. 9b four contiguous subimages of the image are used as input of the machine learning model 40. Alternatively, non-contiguous subimages could be used as input of the machine learning model.

[0186] The at least one subimage in Fig. 9a and 9b can be from a multi-beam charged particle imaging system image as shown on the right in Fig. 9a and 9b or from a detector image as shown on the left in Fig. 9a and 9b. The diagonal line indicates that either at least one subimage 34 of the multi-beam charged particle imaging system image or at least one subimage 36 of the detector image is used as input of the machine learning model. Each extracted at least one subimage 34, 36 is presented as input to the trained machine learning model 40 that predicts one or more state parameters 42 (e.g., focus, astigmatism, etc.) of the multi-beam charged particle imaging system, in particular of the respective at least one primary beamlet, used to generate the at least one input subimage 34, 36. Finally, in an optional control step 44, the at least one respective primary beamlet is corrected by modifying its settings to subtract the predicted aberrations using electro-optical components of the multi-beam charged particle imaging system.

[0187] According to an embodiment, the method further comprises controlling the one or more determined state parameters of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet, in a control step 44. In this way, for example, sample charging, sample drift, primary beamlet parameters, hexagon distortion, pitch, pitch distortion, beam tilt, illumination field inclination, illuminationfield curvature, stigmation parameters, uptime or system reliability parameters can be controlled. The control step can be carried out automatically by the multi-beam charged particle imaging system. Alternatively, the control step can notify a user of the corrections proposed for the state parameters, and the user can confirm these corrections.

[0188] According to an embodiment, the steps of the method including the control step are iterated 45. In this way, iteratively, subimages 34, 36 of an image are obtained of a sample (or characteristics 39 thereof), one or more state parameters 42 of the multibeam charged particle imaging system, in particular of at least one primary beamlet, are determined from at least one subimage of the image using the machine learning model 40, the determined state parameters are automatically controlled (adjusted or corrected) in the control step 44, and then subimages 34, 36 of a newly acquired image are obtained of the sample, and so on.

[0189] The input of the machine learning model 40 can be the at least one subimage 34, 36 itself, or any kind of characteristics 39 extracted from the at least one subimage 34, 36. Thus, optionally characteristics 39 can be extracted from the at least one subimage 34, 36 and used as input of the machine learning model 40. Characteristics comprise, for example, SIFT features, filter responses (e.g., Garbor filters, edge detectors, filters derived from layers of trained neural networks), representations with respect to basis functions such as Fourier transform features, Wavelet transform features, principal component analysis features, sparse coding features, etc.

[0190] The trained machine learning model 40 can be from the group comprising neural networks, diffusion models, support vector machines, random forests, k-nearest neighbors, etc. Among neural networks, in particular deep learning models are advantageous, for example convolutional neural networks. Particularly important due to their increased prediction accuracy, are neural networks including at least one attention mechanism, e.g., Transformers.

[0191] Deep learning is a class of machine learning that uses artificial neural networks with numerous hidden layers between the input layer and the output layer. Due to this complex internal structure the networks are able to progressively extract higher-level features from the raw input data. Each level learns to transform its input data into a slightly more abstract and composite representation, thus deriving low and high levelknowledge from the training data. The hidden layers can have differing sizes and tasks such as convolutional or pooling layers.

[0192] According to an embodiment exemplarily illustrated in Fig. 10, a residual neural network 54 is used for mapping the at least one subimage 34, 36 and / or characteristics thereof to the one or more state parameters 42 of the multi-beam charged particle imaging system, in particular of at least one primary beamlet. The at least one subimage 34, 36 can be from a multi-beam charged particle imaging system image (shown on the right in Fig. 10) or from a detector image (shown on the left in Fig. 10). The diagonal line indicates that either a subimage 34 of the multi-beam charged particle imaging system image or a subimage 36 of the detector image is used as input. Two or more subimages, that are contiguous or non-contiguous, can also be used as input to the machine learning model as shown in Fig. 9a, b. In this way, for example, pitch parameters can be determined from two or more subimages.

[0193] A residual neural network 54 (also referred to as ResNet) is a deep learning architecture in which the layers or blocks of layers learn residual functions with reference to their input. Thus, instead of learning a complete mapping of the input to an output, the layers or blocks of layers only learn a modification of the input x -> F(x) + x, where x is the input of a layer or block of layers and F(x) the function the layer or block of layers performs. The operation of “+%” is implemented via a skip connection 62 that performs an identity mapping to connect the input of the subnetwork with its output. This connection is referred to as a "residual connection". Residual neural networks are advantageous, since they facilitate training of deep architectures. Due to their residual formulation implemented by the skip connections, they mitigate the vanishing gradient problem that is common in case of a large number of layers. Thus, a higher accuracy of the predictions is achieved for deep neural networks.

[0194] The exemplary residual neural network 54 in Fig. 10 contains multiple 3x3 convolution layers 56, followed by a max pooling layer 58 and a fully connected layer 60. Skip connections 62 (residual connections) connect the input of blocks of layers to their output. The output of the residual neural network are the one or more state parameters 42 of the multi-beam charged particle imaging system, in particular of the at least one respective primary beamlet, that was used to generate the at least one subimage 36, 40 that was presented as input to the residual neural network. The skip connections model the residual formulation that allows for training deeper neural networks.Deep neural networks achieve higher accuracy due to their ability to model complex, non-linear relationships, learn hierarchically structured features, scale effectively with large amounts of data, and leverage modern computational resources.

[0195] According to an embodiment of the invention, the neural network comprises at least one attention mechanism.

[0196] The term “attention mechanism” refers to a computational method that is part of a machine learning method that transforms input data to output data. The computational method is used for recognizing relationships between parts of the input data that are relevant for the transformation. To recognize relationships between parts of the input data, the attention mechanism can transform an element of the input data into a new representation, thereby making use of one or more other elements of the input data and their similarity to the element.

[0197] Attention mechanisms in deep learning are used to help the model focus on the most relevant parts of an input when making a prediction. In many problems, the input may be very large and complex, and it can be difficult for the model to process all of it. Therefore, attention mechanisms allow the model to selectively focus on the parts of the input that are most important for making a prediction, and to ignore the less relevant parts. This can help the model to make more accurate predictions and to run more efficiently at recued computation time. As an example, attention mechanisms can be used to analyze text data or audio data (which is sequential), image data (which is spatially ordered as a 2D matrix or 3D matrix), or graph data (which is spatially un-ordered).

[0198] The attention mechanism helps preserve the context of every section in an input by assigning an attention weight relative to many or even to all other sections. This way, even if the input is large, the model can preserve the contextual importance of each section.

[0199] To recognize relationships between parts of the input data, the attention mechanism can transform an element of the input data into a new representation, thereby making use of one or more other elements of the input data and their similarity to the element. The transformation can comprise a similarity function and an aggregation function, wherein the similarity function assesses the similarity of an element to one or moreother elements in the input data, and the aggregation function maps the element and the one or more other elements and their similarities to the new representation of the element. The aggregation function can generate the new representation of the element using a weighted combination of the one or more other elements in the input data, wherein the weights depend on the similarities of the element to the one or more other elements. An attention mechanism can have at least one trainable parameter, preferably for pre-processing elements of the input data such that the similarity function is applied to the pre-processed elements. The at least one trainable parameter can define at least one projection matrix which is used to pre-process the elements of the input data. Throughout the aforementioned definition of the term “attention mechanism”, instead of elements of the input data, representations of the elements of the input data can be processed.

[0200] In contrast to convolutional layers or fully-connected layers, e.g., in CNNs, the weights applied to the elements of the input data depend on the input data, more precisely on the similarity of each element to the other elements of the input data, instead of being fixed after training. Furthermore, in contrast to convolutional operations, the attention mechanism does not require a fixed sequence of the elements in the input data. Instead, context windows of dynamic or global size can be implemented instead of using context windows of fixed size as in case of convolutions. Finally, in contrast to fully-connected layers, the attention mechanism does not require a fixed number of elements in the input data but can be applied to input data sets of arbitrary size. An attention mechanism can, thus, be understood as a location-dependent convolution with input data dependent weights and a context window of arbitrary size. For example, the context window can comprise the complete input data. In case of an imaging dataset, the input data can, for example, comprise a sequence of patches that forms a partition of the imaging dataset, or a sequence of overlapping patches.

[0201] Fig. 11a illustrates an exemplary implementation of an attention mechanism in a machine learning model. Attention mechanisms 900 can be configured as a specific kind of neural network layer that can be added to deep learning models. They allow the model to focus on specific parts of the input by assigning different weights to different parts of the input. This weighting is typically based on the relevance of each part of the input to the task at hand. The attention mechanism transforms a token 902 as input to an attention-based representation 910 of the token 902 using contextual information 902’.The attention mechanism 900 processes individual parts (called “token” 902) of the input. For example, tokens 902 can be words, sub-words, or characters as part of a text, patches or pixels as part of an image or derived representations or image characteristics. In this case, tokens 902 can either refer to subimages 34, 36, in particular to contiguous subimages 34, 36, as shown in Fig. 11a, or to portions of a single subimage 34 obtained by partitioning the subimage 34 into smaller portions. The corresponding image can either be a multi-beam charged particle imaging system image or a detector image as indicated by the diagonal line. The processing of each token 902 considers local or global context, possibly even the entire input, or context from a second data source. This contrasts with, e.g., convolutional filters in CNNs, which always focus on local context only. The “amount of” context which is considered and the processing operations for this consideration depend both on the current token 902. This also contrasts with convolution filters in CNNs, which have fixed size and fixed weights (i.e., fixed operations for each local structure) after being trained.

[0202] During the processing, the attention mechanism 900 transforms a representation in the form of a feature vector of a token 902 into an attention-based representation 910 while considering local or global context 902’. The attention-based representation is an aggregation of values derived from multiple tokens 902, 902’, weighted by the result of a comparison between these tokens 902’ and the token 902 currently being processed, or between derived feature vectors or further derived representations.

[0203] Attention mechanisms 900 comprise self-attention mechanisms and cross-attention mechanisms. Self-attention mechanisms transform the input to a new representation of the input called attention-based representation 910, thereby paying attention to different sections of the input itself. Cross-attention mechanisms transform the input into the new representation called attention-based representation 910 by paying attention to another data source. For example, the other data source can be a reference image that is paid attention to while consecutively transforming each subimage of an image, or the other data source can be a sentence in a third language that is paid attention to while consecutively translating each of the words of a first language into a second language.

[0204] A possible realization of a self-attention mechanism as illustrated in Fig. 11a can be described as follows: let T c BDTdenote a set of tokens 902, 902’ represented bymultivariate feature vectors of dimensionality DT. The attention mechanism 900 a: IRDTIR6transforms the token 902 into an attention-based representation 910 using a similarity function 906 s:

[0205]

[0206] x IRDfcHR yielding an attention distribution 906’ as a result, and an aggregation function 908 m-. P(RDv) -> IRS:

[0207] a

[0208]

[0209] (t) = ■ v(u) | u E T})

[0210] that maps attention weighted values 912 to attention-based representations 910 of the input tokens 902, 902’. Here, J> denotes the power set. The functions q IR07]RDfc, I RDT-> ]RDfc, and v. IRDTIRDuare local feature transformations called query, key and value function that map a token t to a so-called query 914, key 916 and value 918. The query, key and value function can, for example, comprise a trainable parameter, e.g., one or more projection matrices. The comparison function s measures the similarity between the query 914 q(t) for the current token t and the key 916 k(u) for all tokens u e T. Thus, similarities between different tokens in the input can be measured by the function s. The value 918 can be understood as a representation of the token t. The aggregation function m, thus, aggregates all values of the input weighted by their similarity to the query of the current token t. Thus, instead of using fixed values as weights as in case of convolutions, the weights depend on the input data (tokens).

[0211] An example for an attention mechanism called “scaled dot-product attention” or “soft-max attention” defines the similarity function and the aggregation function as follows:

[0212] S

[0213]

[0214] (t’U) =Zu,e / exp(<tix / ^-)’=ZxEXX- In this implementation, the functions k,q,v are typically realized by a learned linear transformation (projection matrices) or a small multilayer perceptron 904. Further attention mechanisms can also be used, e.g., “additive attention” or “not-scaled dotproduct attention”. The aforementioned attention mechanisms are described in “Attention is all you need, A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, I. Polosukhin, Advances in Neural Information Processing Systems, vol. 30. 2017”.Similarly, a cross-attention mechanism can be implemented. In this case, two sets of tokens 902 T ⊂ ℝDand U ⊂ ℝDare given. For example, in case of language translation T and U represent the tokens of each language stream. The attention mechanism 900 a: ℝD→ ℝD̃is defined as follows:

[0215] a

[0216]

[0217] (t) = k(u)] ■ v(u) | u e [ / }) using the query, key and value functions k: ℝD→ ℝD, q: ℝD→ ℝD, and v: ℝD→ ℝDIn this case, the query q(t) of a token 902 in T is compared to all keys 916 obtained from the tokens 902 in set U. Thus, the attention-based representation 910 of token t depends on the second set of tokens U, thereby implementing cross-attention.

[0218] In case the attention mechanism involves learned parameters, multiple attention mechanisms 900 with different learned parameters can be applied to the same input tokens and the attention-based representation 910 can be concatenated for further processing. This process is called multi-head attention mechanism.

[0219] Using attention mechanisms 900 has the following advantages: first, the attentionbased representations 910 of the tokens 902 take into account relationships between tokens 902 from the entire input sequence instead of only from a local neighborhood as, for example, in a CNN. Second, the processing operations applied to the tokens 902 depend on the tokens instead of being fixed as, for example, in convolutions of CNNs.

[0220] Attention mechanisms are typically used as one building block combined with other operations, for example in a Transformer block 920 as illustrated in Fig. 11b. The Transformer block 920 uses an input comprising a set of tokens 902 and positional encodings 928 that encode the position of each token within the input sequence. A multi-head attention mechanism 922 comprising multiple attention mechanisms 900 transforms the input to an attention-based representation 910 of the input. In an add & norm layer 924 the input is added to the attention-based representation 910 in order to prevent vanishing gradients during training, and the result is normalized to improve convergence of the model. A feed-forward neural network 926 is applied to transform the result into a form that can be used by the following layer. Compared to an attention mechanism, a Transformer block, therefore, provides for more stable learning due to the normalization in the add & norm layer 924 and allows for further non-linearities bymeans of the feed-forward neural network 926. Thus, more accurate predictions are achieved.

[0221] By using at least one attention mechanism in the machine learning model, the structural context of the subimages within the image can be taken into account for state parameter estimation. In fact, some state parameters can often be detected only by considering the spatial context of the subimage. By using attention mechanisms, this spatial context is not limited to the local receptive field of a convolution but can comprise large contexts or even the complete input data, i.e., the global context. Thus, by using at least one attention mechanism, the accuracy of the predicted state parameters is improved.

[0222] Apart from a residual network or a neural network comprising at least one attention mechanism, further neural network architectures can be used, e.g., an Inception or Squeezenet architecture.

[0223] According to an embodiment illustrated in Fig. 12, the machine learning model 40 generates a confidence score 80 estimating the accuracy of the predicted one or more state parameters, or how likely the predicted one or more state parameters are the true state parameters of the multi-beam charged particle imaging system, in particular of the primary beamlet. The confidence score can, for example, comprise a number between 0 and 1 that represents the likelihood that the output of the machine learning model is correct, or an estimated accuracy of the prediction. Alternatively, the confidence score can be a value between 0 and co, where 0 indicates a high confidence in the correctness of the prediction and higher values a lower confidence. A confidence score can be normalized to an interval or it can be mapped to the interval [0,1],

[0224] According to an embodiment, the confidence score evaluates the ambiguity of the at least one subimage, in particular the ambiguity of at least one point spread function of the at least one subimage, with respect to the predicted state parameters of the multi-beam charged particle imaging system. For example, in case of ambiguous subimages, e.g., point spread functions, that can indicate different sets of state parameters, the confidence score can be low. This can, for example, be the case for a secondary beamlet that is symmetric with respect to the detector, e.g., the scintillator plate. Thus, the same point spread function can indicate two different focus parameters. In case of a non-ambiguous point spread function, e.g., in case of a secondarybeamlet that is asymmetric with respect to the detector, the confidence score can be high. A point spread function is non-ambiguous, if the corresponding secondary beam profile is not just a simple conic section, but broken in symmetry in the direction of beam propagation (longitudinally) and perpendicular to the direction of beam propagation (transversally).

[0225] A confidence score can be inherent to the machine learning model, e.g., in case of Bayesian models or Gaussian process ensembles. A confidence score can also be derived from the value of the loss function. The higher the value of the loss function fora given subimage and predicted state parameters, the lower the confidence score. A confidence score can also be obtained by presenting one or more slightly modified versions of an input subimage to the machine learning model and deriving the confidence from the variation of predicted state parameters. In case of large variations, the confidence score can be low, in case of small variations, the confidence score can be high. Statistical measures such as a variance of the resulting distribution of predicted state parameters can be used as confidence score here as well. Alternatively, a second machine learning model can be trained to predict a confidence score given the input and output of a first machine learning model. As training data, pairs of subimages and predicted state parameters can, for example, be used, and the deviation of the predicted state parameters from true state parameters can be used to derive a confidence score.

[0226] Instead of training a single machine learning model to map at least one subimage of an image generated by at least one primary beamlet to one or more state parameters of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet, multiple machine learning models can be trained that are used for different primary beamlets. In an example, the primary beamlets are grouped into at least two groups, and one machine learning model is trained for each group of primary beamlets. In another example, for each primary beamlet a separate machine learning model is trained (group of size one). In this way, the learning task becomes less complex, such that the machine learning models can be better adapted to the specific characteristics of the associated primary beamlet or the associated group of primary beamlets in order to increase the prediction accuracy. By selecting a group size, a tradeoff can be found between prediction accuracy on the one hand and training time and memory requirements on the other hand.The primary beamlets can be grouped according to different criteria. The primary beamlets of a group can be adjacent or non-adjacent, and the corresponding subimages can be contiguous or non-contiguous. The groups of primary beamlets can contain the same number of primary beamlets or differing numbers. The groups can be defined by a user, or they can be generated automatically, for example by tessellation of the image. The number of groups can, for example, be defined by a user or determined automatically. Each primary beamlet can belong to maximally one group. There may be primary beamlets that do not belong to any group. Preferably, each primary beamlet belongs to exactly one group. Some primary beamlets may belong to more than one group, e.g., primary beamlets at the border of two groups. In this way, smooth transitions between the predicted state parameters of neighboring groups can be achieved.

[0227] For example, the primary beamlets can be spatially grouped. Thus, primary beamlets that are adjacent or close to a point (e.g., a selected group center) or that generate contiguous subimages belong to the same group. A spatial grouping can, for example, be obtained by distributing a specified number of group centers over the image and assigning each subimage and corresponding primary beamlet to the group center that is closest to the center of the subimage. Alternatively, a Voronoi tessellation of the image can be computed and each subimage be assigned to the group center, whose Voronoi cell has maximum overlap with the subimage. A spatial grouping is illustrated in Fig. 13. Fig. 13 shows subimages 36 of an image generated by different primary beamlets that are spatially grouped in four different groups 82, 82’, 82”, 82”’. In this way, the prediction accuracy is improved, since primary beamlets that are close to each other usually have similar state parameters. This can, for example, be seen in Fig. 13, where spatially close point spread functions show similar characteristics. Such a spatial grouping of primary beamlets is, of course, not limited to detector images but can be applied to subimages of multi-beam charged particle imaging system images as well.

[0228] In another example, the primary beamlets can be grouped by the similarity of their selected state parameters. In this way, the prediction accuracy is improved, since primary beamlets that have the same or similar selected state parameters usually also have similar measured state parameters.In another example, the primary beamlets are grouped by the similarity of sample properties with respect to the section of the sample that is scanned by the primary beamlet. Such sample properties can, for example, comprise sample height, sample topography, sample location (e.g., center or boundary of sample) or sample type (e.g., type of tissue or of semiconductor structures). By grouping with respect to sample similarity, the subimages of the primary beamlets within the same group are obtained under similar sample conditions and, thus, likely to be similar.

[0229] In another example, primary beamlets are grouped according to imaging properties. Imaging properties of a primary beamlet comprise properties that can be measured from a respective subimage obtained by the primary beamlet, for example, contrast, brightness, sharpness, resolution, signal-to-noise ratio (SNR), structural similarity (SSIM) with respect to some reference image, etc. By grouping with respect to imaging properties, subimages with similar imaging properties and, thus, probably similar state parameters are processed by the same group-specific machine learning model. This reduces the complexity of the machine learning model and allows the machine learning model to better adapt to the task at hand. Therefore, the state parameter predictions are of a higher accuracy.

[0230] Apart from the described groupings, any further groupings are conceivable, for example user-defined groupings or random groupings.

[0231] The number of groups can be between two and the number of primary beamlets. The number of groups can be determined by a user, or it can be determined automatically according to some criterion. Such a criterion can, for example, be a maximum allowable distance between each two primary beamlets within a group, a maximum allowable deviation of one or more parameters of two primary beamlets or of imaging properties of two subimages within a group, etc. Smaller numbers of groups require less trained group-specific machine learning models and, thus, less memory space, training time and effort for providing a sufficient amount of training data. Larger numbers of groups reduce the complexity of the learning task and allow the group-specific machine learning models to better adapt to the training data, which improves the image quality of the adjusted image.An embodiment of the invention comprises a computer implemented method fortraining a machine learning model for mapping at least one subimage of an image generated by at least one primary beamlet in a multi-beam charged particle imaging system to one or more state parameters of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet, as described above.

[0232] Fig. 14 illustrates an exemplary method for generating training data and training a machine learning model for mapping a subimage of an image generated by a primary beamlet in a multi-beam charged particle imaging system to one or more state parameters of the multi-beam charged particle imaging system, in particular of the primary beamlet, according to the invention. In a state parameter selection step 46 state parameters 42 comprising, for example, a focus value

[0233]

[0234] and two astigmatism values aXi, aYi, are selected for one or more primary beamlets of one or more multi-beam charged particle imaging systems. In this way, for example, a known amount of aberration (focus, astigmatism, etc.) can be introduced into the image using electro-optical components of the multi-beam charged particle imaging system. In an image acquisition step 48 one or more images (multi-beam charged particle imaging system images or detector images) are acquired, and subimages 34 of a multi-beam charged particle imaging system image or subimages 36 of a detector image are extracted for one or more primary beamlets. These steps 46, 48 are iterated 52 until a sufficient amount of training data comprising tuples of a subimage and corresponding one or more state parameters of the multi-beam charged particle imaging system, in particular of the respective primary beamlet, is generated. In a training step 50, the generated training data is then used to train the machine learning model.

[0235] The machine learning model for mapping a subimage of an image generated by a primary beamlet in a multi-beam charged particle imaging system to one or more state parameters of the multi-beam charged particle imaging system, in particular of the primary beamlet can, for example, be trained using training data illustrated in Fig.

[0236] 15 and 16. Fig. 15 shows training data comprising various subimages 34 of multibeam charged particle imaging system images and corresponding state parameters comprising a focus value ), an astigmatism value in a first direction aXt and an astigmatism value in a second direction aYi, for i e {1,,.,n} of the corresponding primary beamlet, where n is the number of training samples. Fig. 16 shows corresponding training data comprising various subimages 36 of images. An exemplary focus rangeof the training images is + / - 15 pm, an exemplary stigmation range of the training images is + / - 1.2 units.

[0237] The subimages 34, 36 are derived from multiple images (either multi-beam charged particle imaging system images or detector images) that are obtained using the multibeam charged particle imaging system whose one or more state parameters are to be predicted by the machine learning model.

[0238] During training of the machine learning model, a loss function is optimized. The loss function can, for example, measure the deviation of predicted state parameters from correct state parameters in the training data. The optimization can be carried out using, e.g., a variant of the backpropagation algorithm or the adaptive moment estimation (Adam) optimization algorithm. Adam is an optimization algorithm that builds upon the strengths of two other popular techniques: AdaGrad and RMSProp. It is an adaptive learning rate algorithm that dynamically adjusts the learning rate for each individual parameter within a machine learning model, rather than using a single global learning rate.

[0239] In a specific example, the following parameters were used for training of a residual network including approximately 41,000 degrees of freedom:

[0240] Parameter Value

[0241] Subimage size 32 x 32 px

[0242] Number of residual blocks 4

[0243] Number of epochs 1000

[0244] Number of training images 1500

[0245] Training steps per epoch 5

[0246] Learning rate 0.0003

[0247] Optimizer Adam

[0248] Loss function Mean squared error loss Convolution kernel size 3 x 3

[0249] Activation function Tanh

[0250] Pooling size 2 x 2According to an embodiment illustrated in Fig. 17, a computer implemented method 84 for training a plurality of group-specific machine learning models that each map at least one subimage 36 of an image 30 generated by at least one primary beamlet from a group of primary beamlets of a multi-beam charged particle imaging system to one or more state parameters of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet, comprises: providing a prototype machine learning model for mapping at least one subimage of an image generated by at least one primary beamlet from a group of primary beamlets of a multi-beam charged particle imaging system to one or more state parameters of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet, in a pre-training step 86; for each group of primary beamlets, here exemplarily shown for the group 82’ comprising the primary beamlets generating the point spread functions in the upper right corner of the image 32, providing group-specific training data 92’ comprising at least one subimage 36’ of one or more images obtained by at least one primary beamlet of the group of primary beamlets of the multi-beam charged particle imaging system and corresponding one or more state parameters 42’ of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet, and training the corresponding machine learning model for the group of primary beamlets by fine-tuning the prototype machine learning model using the groupspecific training data in a fine-tuning step 90. Each of the plurality of trained groupspecific machine learning models can be used for mapping at least one subimage of an image generated by a primary beamlet of the respective group of primary beamlets of the multi-beam charged particle imaging system to one or more state parameters of the multi-beam charged particle imaging system, in particular of the respective at least one primary beamlet.

[0251] The prototype machine learning model can be trained using training data 92 comprising at least one subimage 36 of one or more images, each subimage being generated by a primary beamlet of the multi-beam charged particle imaging system, and corresponding state parameters 42 of the multi-beam charged particle imaging system, in particular of the at least one primary beamlet.

[0252] By using the two-step training procedure illustrated in Fig. 17, the multiple group-specific machine learning models can be trained very efficiently. In the pre-training step, subimages of all primary beamlets can be used for training. The subimages can be generated by the same multi-beam charged particle imaging system, or by differentmulti-beam charged particle imaging systems. For example, the subimages can be generated in a centralized place using a prototype multi-beam charged particle imaging system. Then, in the fine-tuning step 90, the prototype machine learning model can be fine-tuned to specific groups of primary beamlets of a specific machine by using only group-specific training data 92’. When comparing the example subimages in the training data 92 to the example subimages in the group-specific training data 92’ in Fig. 17, the subimages in the training data 92 contain much more variance than in the group-specific training data 92’. Thus, the learning task is simplified for the group-specific machine learning model and, thus, the training procedure can be carried out more efficiently and the prediction accuracy is improved. Furthermore, the generation of training data is simplified as less group-specific training data 92’ is required for fine-tuning, and the generation of large amounts of training data 92 for training the prototype machine learning model is simplified.

[0253] According to an example of the method for training a machine learning model, each group of primary beamlets comprises only a single primary beamlet. Thus, each primary beamlet has its own separate machine learning model that is fine-tuned using training data comprising only subimages acquired using the specific primary beamlet.

[0254] Fig. 18a to c illustrates accuracy results of the method for determining one or more state parameters of a multi-beam charged particle imaging system according to the invention for a sample primary charged particle beamlet in 25 sample images using a residual neural network for subimages of an image. Fig. 18a to c show on the horizontal axis an image index and on the vertical axis the predicted respective state parameter in the form of a cross sign (focus value 93 normalized to the interval [-1,1] in Fig. 18a, astigmatism value in X direction 94 in Fig. 18b, and astigmatism value in Y direction 96 in Fig. 18c) and the corresponding ground truth value in the form of a plus sign. The experiments show that the method for determining one or more state parameters of a multi-beam charged particle imaging system using a neural network is sufficiently accurate. Compared to standard calibration methods, the computation time could be reduced by a factor of about 27.

[0255] A multi-beam charged particle imaging system 1 according to an embodiment of the invention is illustrated in Fig. 19. The multi-beam charged particle imaging system comprises: means 305 for generating a plurality of primary beamlets in a raster configuration for the parallel scanning of a sample 7; a detector 600 arranged in a focalplane of a plurality of secondary beamlets emitted from the sample 7; a data processing unit 700 for determining state parameters of the multi-beam charged particle imaging system, in particular of the primary beamlets, and a control unit 702 configured for controlling state parameters of the multi-beam charged particle imaging system 1 using a method for controlling the one or more determined state parameters as described above. The data processing unit 700 is used for determining state parameters of the multi-beam charged particle imaging system 1 from at least one subimage of an image, in particular of a multi-beam charged particle system image 30 or of a detector image 32. In case of a detector image 32, the multi-beam charged particle imaging system 1 may comprise a detector including a scintillator and a scintillator image acquisition unit 604. The state parameters are controlled by the control unit 702. To this end, the at least one respective primary beamlet can be corrected by modifying its settings to subtract the predicted aberrations using electro-optical components of the multi-beam charged particle imaging system 1.

[0256] While the preceding figures illustrate compensating aberrations by controlling state parameters of electro-optical hardware, the embodiments illustrated starting from Fig.

[0257] 20 address a software-based solution of restoring image information in the image domain based on beam let-specific aberration information derived from the imaging system.

[0258] Fig. 20 schematically illustrates non-uniform image quality in a multi-field-of-view 160 of a multi-beam charged particle imaging system when operating at a global focus setting. The multi-field-of-view 160 is composed of a plurality of image patches 245 arranged in concentric shells 162, 162' around a central region. The shading of the image patches 245 indicates variations in image quality or resolution across the multi-field-of-view 160. As illustrated, image quality varies across the multi-field-of-view, indicating the presence of shell-dependent aberrations. Depending on the alignment state and operating conditions of the system, different shells 162, 162’ can be affected to different extents. In some cases, image patches 245 associated with inner shells and / or outer shells may exhibit stronger aberrations than image patches located at intermediate shell positions, while in other cases a different distribution may occur, e.g., the aberrations may increase with increasing shell number as shown in Fig. 20 and 21.Fig. 21 schematically illustrates the dependence of information dispersion 164 on shell number 166 across the multi-field-of-view 160. The horizontal axis represents the shell number 166, corresponding to increasing radial distance from the center of the multi-field-of-view 160, while the vertical axis represents the amount of information dispersion 164. Two curves are shown, one corresponding to defocus 168 and the other corresponding to astigmatism 170. As illustrated, information dispersion 164 caused by defocus 168 increases strongly with shell number 166, approximately following a quadratic behavior, whereas information dispersion 164 caused by astigmatism 170 increases approximately linearly. The total information dispersion 164 experienced by a given shell 162 results from the combined contribution of multiple aberrations.

[0259] Together, Fig. 20 and Fig. 21 illustrate that global optical alignment leads to shelldependent information dispersion 164 and non-uniform image quality across the multi-field-of-view 160.

[0260] Fig. 22 and 23 illustrate hardware-based solutions to the problem of non-uniform image quality of images acquired by a multi-beam charged particle imaging system.

[0261] Fig. 22 schematically illustrates an approach for compensating field-curvature-in-duced focus offsets in a sample irradiation unit 100 by adding hardware 152 to the optical system. The sample irradiation unit 100 comprises a beam shaper plate 144 and an objective lens system 102 for focusing beam paths toward a sample plane. Due to the optical properties of the objective lens system 102, field curvature is introduced, resulting in a focus offset 150 that varies across the irradiated field.

[0262] The left-hand side of Fig. 22 illustrates operation at a global focus setting, in which the focus offset 150 leads to a curved focal surface and variations in focus and resolution across the irradiated field. The right-hand side of Fig. 22 illustrates a solution in which a field curvature corrector 146 is introduced downstream of the beam shaper plate 144 as part of the objective lens system 102 in order to locally influence the beam paths and reduce the focus offset 150.

[0263] However, this hardware-based approach increases system complexity and does not inherently ensure uniform focus across the entire irradiated field, especially for large numbers of beam paths.Fig. 23 schematically illustrates a further hardware-based approach for compensating astigmatism in a sample irradiation unit 100. As in Fig. 22, the sample irradiation unit 100 comprises a beam shaper plate 144 and an objective lens system 102 for focusing beam paths toward a sample plane. In the illustrated configuration, astigmatism introduced by beam shaping and beam splitting elements leads to stigmatized beam profiles 156 at the sample plane when operating at a global stigmation setting. The beam profiles exhibit elliptical shapes with varying orientations, resulting in non-uni-form resolution and image quality.

[0264] The right-hand side of Fig. 23 illustrates a solution in which additional hardware is added to the optical system, as indicated by reference sign 152. In particular, a stigmation corrector 154 is introduced downstream of the beam shaper plate 144 as part of the objective lens system 102 in order to locally compensate astigmatism and obtain corrected beam profiles 158.

[0265] While such hardware-based correction can reduce aberrations for selected beam paths as illustrated in Fig. 22 and Fig. 23, this approach introduces fundamental limitations with respect to scalability toward larger numbers of beam paths.

[0266] Fig. 24 schematically illustrates the scalability limitations of hardware-based aberration correction in multi-beam charged particle imaging systems. The figure shows the relationship between the number of beams 174 and the resulting production cost and system complexity 172 associated with hardware-based correction approaches. As illustrated, increasing the number of beams 174 requires a rapidly increasing number of correction channels and associated electro-optical components, leading to drastic increases in electrical circuit density and design complexity. In addition, densely integrated correction hardware is subject to reduced manufacturing yield due to tight micro-manufacturing tolerances, reduced robustness against environmental influences such as particle contamination or electrical arcing, and substantially increased development, manufacturing, calibration, and maintenance effort. Fig. 24 therefore illustrates that hardware-based correction approaches scale unfavorably toward larger beam numbers 174 and inherently limit achievable system scalability.Against the background of the scalability limitations of hardware- based correction illustrated in the preceding figures, Fig. 25 schematically illustrates a method for restoring an image in a multi-beam charged particle imaging system according to an embodiment of the invention.

[0267] Fig. 25 schematically illustrates a method 180 for restoring an image in a multi-beam charged particle imaging system. In a first step S1, a plurality of aberrated subimages is acquired, wherein each aberrated subimage is generated from signals recorded by a detector and corresponds to an area of a sample scanned by a respective primary beamlet. In a second step S2, beamlet-specific aberration information is obtained for each of the plurality of primary beamlets, the beamlet-specific aberration information comprising a point spread function or parameters derived therefrom. In a third step S3, a plurality of restored subimages is generated by applying, for each aberrated subimage, a restoration method that uses the respective aberrated subimage and the corresponding beamlet-specific aberration information as inputs, and a restored image is generated from the plurality of restored subimages.

[0268] An "aberrated subimage" is a subimage that contains distortions or artifacts resulting from non-ideal optical properties of the imaging system. These aberrations manifest visually as, for example, a general blurring (defocus), a directional stretching or compression (astigmatism), or more complex shape distortions (e.g., coma).

[0269] An "image restoration method" refers to a computational process that takes an aberrated subimage and corresponding beamlet-specific aberration information as inputs, and processes these inputs to generate a restored subimage in which the effects of the aberrations are reduced or removed, leading to an improved image quality.

[0270] In an example of the image restoration method described above, the plurality of primary beamlets is not treated as a homogeneous set but is structured according to their spatial arrangement within the multi-beam system. In particular, the primary beamlets may be grouped into two or more groups, wherein beamlets within a group share similar optical characteristics and aberration behavior. Such grouping may be performed based on the radial distance of the beamlets from the optical axis, resulting in groups that correspond to different shells of the beamlet array.By applying the restoration method using group-specific aberration information, the computational effort required for image restoration can be reduced while still accounting for systematic aberration differences between groups of beamlets. Beamlets belonging to the same shell typically experience similar aberrations, such as comparable amounts of defocus or astigmatism, such that a shared representation of aberration information can be used for restoring subimages associated with that shell.

[0271] In further examples, the plurality of primary beamlets may be grouped according to criteria other than radial position. For example, beamlets may be grouped based on similarity of beamlet-specific aberration information, dominant aberration type, residual aberration after hardware-based correction, detector region, scan direction, noise characteristics, or system symmetry. Such groupings allow flexible trade-offs between restoration accuracy, computational effort, and robustness to system variations.

[0272] According to another example, the restoration method is applied independently for each primary beamlet. In such configurations, beamlet-specific aberration information is obtained and used individually for restoring the corresponding aberrated subimage. This allows a highly accurate compensation of beamlet-specific aberrations and is particularly advantageous for systems with strong beamlet-to-beamlet variations or for applications requiring maximum image fidelity.

[0273] According to an aspect of the invention, the detector used for acquiring the aberrated subimages comprises a scintillator, and the beamlet-specific point spread function is obtained from a scintillator image acquired by the detector, as, for example, shown in Fig. 6 and 7. The scintillator image provides direct information about the spatial intensity distribution of the secondary beamlets at the detector plane, which reflects the optical transfer behavior of the system for the respective primary beamlet. Using scintillator images to obtain beamlet-specific point spread functions enables parallel acquisition of aberration information for a plurality of beamlets and allows the image restoration method to be closely tied to the actual optical state of the system during image acquisition.

[0274] According to the similarity of point spread functions in the scintillator image, groupings of beamlets may be derived. For example, the spatial intensity distribution of the secondary beamlets in the scintillator image may be compared using image processingtechniques, e.g., correlation, to determine their similarity. Beamlets with a high similarity may be grouped.

[0275] In addition or alternatively, clustering methods may be used to group beamlets with similar spatial intensity distribution in the scintillator image. For each beamlet, a spatial intensity distribution corresponding to the secondary beamlet is extracted from the scintillator image and represented by characteristic features, such as moments, symmetry measures, width parameters, coefficients of a parametric point spread function model or some feature vector. The beamlets are then clustered based on similarity of these features using clustering techniques, for example distance-based clustering, hierarchical clustering, or density-based clustering, such as k-means clustering or DBSCAN. Beamlets assigned to the same cluster are considered to exhibit similar optical transfer behavior and are processed using common aberration information during image restoration. In this way, clustering of scintillator-based spatial intensity distributions enables data-driven grouping of beamlets that reflects the actual optical state of the system while reducing computational complexity.

[0276] Fig. 26 schematically illustrates a restoration method 181 based on deconvolution using measured point spread functions. A sample 182 is imaged by the charged particle imaging system. The image formation process is represented as a convolution of the sample 182 with aberration information 184 in the form of a beam let-specific point spread function (PSF) that characterizes the optical transfer behavior of the respective beamlet. During image acquisition, noise 186 is added, resulting in an aberrated image 188. The aberrated image 188 is processed by a deconvolution method 190. The deconvolution method 190 uses the PSF 184 as a known deconvolution kernel to restore image information of the sample 182. In example implementations, the deconvolution method 190 comprises an iterative deconvolution algorithm, such as a Richardson-Lucy algorithm or a related maximum-likelihood-based deconvolution algorithm.

[0277] In multi-beam charged particle imaging systems, the deconvolution method 190 can be applied independently and in parallel to aberrated images 188 corresponding to different beamlets using their respective PSFs 184, thereby enabling efficient restoration of a plurality of subimages acquired simultaneously.Fig. 26 therefore illustrates a restoration method 181 in which beam let-specific aberration information 184 in the form of measured PSFs is exploited for parallelized image restoration.

[0278] Fig. 27 schematically illustrates a restoration method 181 based on constrained deconvolution. As in Fig. 26, a sample 182 is imaged and degraded by convolution with a PSF 184, and noise 186 is added, resulting in an aberrated image 188. The aberrated image 188 is processed by a constrained deconvolution method 192. The constrained deconvolution method 192 is configured to restore image information while simultaneously estimating or refining a PSF subject to one or more constraints 194.

[0279] In the example illustrated in Fig. 27, the constraints 194 are primarily derived from a measured PSF 184 obtained from the imaging system, for example from a scintillator image acquired for the corresponding beamlet. The measured PSF 184 provides prior information about the expected shape, extent, symmetry, or parameter ranges of the PSF and constrains the admissible solutions during deconvolution.

[0280] In further examples, additional constraints 194 may be derived from other sources, such as parametric models of the optical system, calibration data, known physical properties of the imaging system, or bounds on aberration parameters. These constraints may, for example, limit the width, orientation, or higher-order components of the estimated PSF or enforce smoothness and stability of the solution.

[0281] The constrained deconvolution method 192 outputs a restored representation of the sample 182 while ensuring consistency between the restored image and the constrained PSF estimate. As in Fig. 26, the constrained deconvolution method 192 can be applied independently and in parallel for a plurality of beamlets. Fig. 27 therefore illustrates a restoration method 181 in which measured beamlet-specific aberration information in the form of measured beam-let specific PSFs 184 are used as a primary source of constraints 194 for constrained deconvolution, optionally supplemented by further system-derived constraints.

[0282] Fig. 28 schematically illustrates a generalized image restoration approach based on a machine learning image restoration model 404. An input is provided to the machine learning image restoration model 404. The input comprises an aberrated image 188and beamlet-specific aberration information 184. The beam let-specific aberration information 184 may, for example, be derived from measured beam profiles obtained from a scintillator image and represents system-specific information characterizing the optical transfer behavior of the imaging system. By incorporating measured beam-let-specific aberration information 184 directly into the machine learning image restoration model 404, the restoration process is adapted to the actual optical state of the system. This enables a generalized restoration approach that is versatile with respect to different aberration types and system configurations and that can be applied efficiently to a plurality of beamlets. As a result, the approach illustrated in Fig. 28 overcomes limitations of scalability associated with hardware-based correction and iterative alignment methods in multi-beam charged particle imaging systems.

[0283] The use of a machine learning image restoration model 404 enables highly accurate restoration of image information because the restoration behavior is derived from examples rather than from predefined analytical rules. During training, the machine learning image restoration model 404 learns complex and non-linear relationships between aberrated image patterns and restored image patterns that are difficult to capture using explicit physical models or manually defined heuristics. As a result, the restoration accuracy can be improved, in particular in the presence of combined or higher-order aberrations.

[0284] Further, once trained, the machine learning image restoration model 404 enables image restoration with short inference times, as the restoration process is performed by evaluating the trained model rather than by executing iterative optimization or deconvolution algorithms. This allows efficient and high-throughput processing of images acquired by multi-beam charged particle imaging systems. In addition, the use of a trained machine learning image restoration model 404 reduces or eliminates the need for expert knowledge during operation, as restoration parameters and rules do not need to be manually defined or adjusted by a user.

[0285] According to some examples, the machine learning image restoration model 404 is combined with grouping of beamlets based on their optical characteristics or spatial arrangement. For example, beamlets may be grouped according to shells, similarity of beamlet-specific aberration information, or other grouping criteria as described above. In such embodiments, a separate machine learning image restoration model 404 may be trained for each group of beamlets using group-specific training data,allowing the restoration behavior to be adapted to the aberration characteristics of the respective group. In this way, highly accurate restoration is possible while reducing the required effort for training and storing the machine learning image restoration models 404.

[0286] In an alternative example, a separate machine learning image restoration model 404 is trained for each individual beamlet, enabling highly beamlet-specific restoration behavior that leads to high image quality. In further examples, a single machine learning image restoration model 404 is trained using training data from a plurality of beamlets or beamlet groups, allowing the model to generalize across different beamlets while still accounting for beamlet-specific aberration information provided as input. These embodiments allow flexible trade-offs between restoration accuracy, training effort, computational complexity, and scalability in multi-beam charged particle imaging systems.

[0287] Fig. 29 schematically illustrates an exemplary structure of a machine learning image restoration model described in connection with Fig. 28.

[0288] As illustrated, the machine learning image restoration model 404 may be configured as an image-to-image architecture for restoring aberrated images. In particular, the machine learning image restoration model 404 exploits a convolutional neural network structure, here a U-Net, to identify object- re leva nt features in the input. The convolutional neural network comprises an encoder-decoder architecture 406 comprising an encoder 408, a bottleneck 414, and a decoder 410. The encoder 408 processes the input, which includes an aberrated image 188 and beamlet-specific aberration information 184, and extracts object information in the bottleneck 414. The encoded state in the bottleneck 414 represents a compact representation of image features that are relevant for image reconstruction while being less sensitive to system-induced aberrations or noise. The decoder 410 then reconstructs a restored image 182 from the encoded state in the bottleneck 414, thereby compensating aberrations present in the input.

[0289] In the embodiment illustrated in Fig. 29, the machine learning image restoration model 404 is implemented as a U-Net architecture comprising an encoder 408 and a decoder410 arranged in a symmetric structure. The encoder 408 progressively extracts object-relevant features from the input at different spatial scales, while the decoder 410 reconstructs a restored image 182 from the encoded state 414.

[0290] The U-Net architecture further comprises skip connections 409 that directly connect corresponding layers of the encoder 408 and the decoder 410. The skip connections 409 transfer feature information from the encoder 408 to the decoder 410 at matching resolution levels. In this way, fine spatial details extracted in early encoder layers are preserved and reused during reconstruction, while higher-level semantic information extracted at deeper layers contributes to robust restoration of object structures.

[0291] By combining multi-scale feature extraction in the encoder 408 with skip connections 409 and hierarchical reconstruction in the decoder 410, the U-Net-based machine learning image restoration model 404 enables accurate restoration of aberrated image patterns while preserving both global context and local structural detail. This architecture is particularly well suited for image-to-image restoration tasks in multi-beam charged particle imaging systems, where aberrations may vary spatially across the image.

[0292] According to an example, the machine learning image restoration model comprises a residual neural network architecture, as illustrated in Fig. 10. Residual neural networks enable the learning of restoration behavior as a correction relative to an input image, which is particularly effective for compensating moderate aberrations while preserving fine image details. The use of residual connections further improves training stability and allows efficient implementation of deep models with high restoration accuracy.

[0293] In further examples, the machine learning image restoration model 404 comprises one or more attention mechanisms, as described with respect to Fig. 11a, b. An attention mechanism is a computational mechanism configured to dynamically weight different parts of input data based on their relevance to an output being generated. In the context of image restoration, an attention mechanism enables the model to selectively emphasize spatial regions, feature channels, or combinations thereof that are most relevant for reconstructing object information, while reducing the influence of noise- or aberration-dominated regions.Attention mechanisms may operate on image pixels, image patches, feature maps, or encoded representations generated by an encoder.

[0294] In example implementations, the attention mechanism is implemented using a Transformer architecture or components derived therefrom. Transformer architectures employ attention operations that model relationships between elements of the input by computing interactions between query, key, and value representations. In imagebased applications, Transformer-based attention mechanisms can capture long-range spatial dependencies and global contextual information that are difficult to model using local convolution operations alone. In further embodiments, convolutional neural network components, such as encoder-decoder or U-Net architectures, are combined with Transformer-based attention mechanisms, for example by integrating attention modules within the encoder, the decoder, or the skip connections. Such hybrid architectures enable the machine learning image restoration model 404 to jointly exploit local feature extraction and global context modeling, thereby improving robustness and accuracy of image restoration in the presence of spatially varying aberrations.

[0295] In a further example, the machine learning image restoration model comprises a generative machine learning model. Generative models are configured to model a distribution of restored images and to generate restored image information conditioned on input data. In the context of image restoration, the generative machine learning model may generate a restored image conditioned on an aberrated image and beamlet-spe-cific aberration information, wherein the aberration information is provided as conditional information that influences the generative process. Example implementations include conditional variational autoencoders, conditional generative adversarial networks, diffusion-based generative models, or autoregressive generative models. By incorporating beam let-specific aberration information as conditioning input, the generative machine learning model can adapt the generated image to the actual optical state of the imaging system, enabling accurate and flexible restoration of image information even in the presence of complex or spatially varying aberrations.

[0296] The beam let-specific aberration information 184 is provided to the machine learning image restoration model 404 as an additional input channel or as conditioning information. In such embodiments, the aberration information 184 influences the internalprocessing of the machine learning image restoration model 404, for example by modulating feature extraction, normalization, or attention weights within the encoder 408, the decoder 410, or the skip connections 409. In this way, the machine learning image restoration model 404 adapts its restoration behavior to the current aberration state of the imaging system.

[0297] In the context of machine learning, "conditioning information" refers to data that is provided to a model to guide or modify its behavior on the primary input data. In this invention, the beamlet-specific aberration information can act as conditioning information, provided to the model as a separate input channel or vector, to inform the model about the specific type and magnitude of aberration present in the subimage being processed. Further conditioning information may, for example, comprise determined state parameters of the multi-beam charged particle imaging system, or settings of hardware components, e.g., of hardware components for global or beam-specific corrections.

[0298] As illustrated in Fig. 29, the beamlet-specific aberration information 184 can be used as additional input in the input layer of the neural network. Alternatively, it can be used as input in intermediate layers of the neural network, e.g., in an intermediate layer of the encoder or at the bottleneck 414 in a U-Net, represented by reference sign 412, or in an intermediate layer of a diffusion model, etc. Providing beamlet-specific aberration information to an intermediate layer of a neural network enables the machine learning image restoration model to condition higher-level image features on the aberration state of the imaging system. In this way, object-relevant features can first be extracted largely independently of aberrations, while the restoration behavior is subsequently adapted based on the aberration information. This improves robustness of the restoration with respect to varying aberration strengths and reduces the risk of confusing aberration-induced artifacts with actual object structures. Conditioning at deeper layers further enables efficient adaptation of global restoration characteristics, such as anisotropy compensation or resolution enhancement, without affecting low-level feature extraction. As a result, restoration accuracy and generalization across different beamlets, shells, and operating conditions are improved while maintaining low inference complexity.In a further embodiment, the restoration algorithm comprises a combination of a deconvolution method and a machine learning image restoration model. In such embodiments, an aberrated image is first processed by a deconvolution method using beam-let-specific aberration information, such as a point spread function, to remove a substantial portion of system-induced blur in a physics-based manner. The intermediate, deconvolved image is then provided as input to the machine learning image restoration model, which is configured to further restore image information by compensating residual aberrations, noise amplification, and artifacts introduced by the deconvolution process. This hybrid approach combines the interpretability and physical consistency of deconvolution with the flexibility and expressive power of machine learning, enabling improved restoration accuracy and robustness compared to either approach alone. In particular, the machine learning image restoration model can learn to correct systematic deconvolution artifacts and to adapt restoration behavior to beamlet-spe-cific aberration characteristics, while maintaining short inference times suitable for high-throughput multi-beam charged particle imaging systems.

[0299] The training of the machine learning image restoration model may be performed using a structured training regime that exploits different system states and image acquisition conditions. In such a regime, training data is acquired by operating the multi-beam charged particle imaging system in different optical configurations, for example by varying focus, stigmation, or other electro-optical parameters, and by acquiring corresponding aberrated images together with beamlet-specific aberration information, e.g., scintillator images. Reference images representing restored image information may be obtained from images acquired under optimized imaging conditions, from images acquired after alignment, from images using hardware-corrections, or from images processed using physics-based restoration methods. In this way, the training data reflects the actual optical behavior of the system across a range of aberration states and enables the machine learning image restoration model to learn a mapping from aberrated image information and aberration information to restored image information.

[0300] " Imaging conditions" refer to the complete set of parameters and states of the multibeam system and the sample at the time of image acquisition. This encompasses not only the imaging parameters (lens settings, etc.) but also factors such as beam energy, sample type and topography, and detector settings.To reduce the overall training effort and improve scalability, pre-training can be exploited. In particular, a single machine learning image restoration model may first be pre-trained using training data obtained from all beamlets, from multiple beamlets, or from different groups of beamlets. The pre-trained model captures general relationships between aberrated image patterns and restored image patterns that are common across beamlets. Subsequently, the pre-trained model may be adapted, for example by fine-tuning, using training data from a specific group of beamlets or from a single beamlet. This staged training regime enables efficient reuse of learned representations while allowing adaptation to beamlet-specific or group-specific aberration characteristics.

[0301] Such a training regime allows flexible trade-offs between training effort, model generalization, and restoration accuracy. In particular, pre-training across multiple beamlets improves robustness and reduces the amount of beamlet-specific training data required, while subsequent adaptation enables accurate compensation of systematic aberration differences between beamlets or beamlet groups.

[0302] Fig. 30 schematically illustrates a method for training a machine learning image restoration model. In a first step T1, training data is acquired for each of a plurality of primary beamlets, the training data comprising at least one aberrated subimage generated from signals recorded by a detector and corresponding beamlet-specific aberration information comprising a point spread function measured by the detector or parameters derived therefrom, wherein the aberrated subimage and the beamlet-specific aberration information are provided as input data. In a second step T2, for each aberrated subimage, a corresponding reference image representing restored image information is provided as output data. In a third step T3, the machine learning image restoration model is trained using the training data.

[0303] Fig. 31 schematically illustrates a method for generating training data for training a machine learning image restoration model. In a first step G1, for at least one primary beamlet, a reference subimage and corresponding beamlet-specific aberration information are acquired. In a second step G2, a predefined aberration is introduced in the multi-beam charged particle imaging system. In a third step G3, an aberrated subimage and corresponding beamlet-specific aberration information are acquired. In a fourth step G4, the aberrated subimage is associated with the reference subimage as a training data pair.In the first step G1, a reference subimage is acquired for at least one primary beamlet. A "reference subimage" is a subimage that represents the desired "ground truth" output for a corresponding aberrated subimage in a training data pair. Preferably, ist is of high quality and substantially aberration-free or it at least contains reduced aberrations. It is used during the training of a machine learning model as the target to which the model's output is compared. The reference subimage may represent image information acquired under optimized or nominal imaging conditions, for example after alignment of focus and stigmation or under system settings known to produce high image quality. Corresponding beam let-specific aberration information may be additionally obtained from measured beam profiles, such as spatial intensity distributions recorded at a scintillator, and characterizes the optical transfer behavior of the beam-let during acquisition of the reference subimage.

[0304] In the second step G2, a predefined aberration is intentionally introduced into the multi-beam charged particle imaging system. The predefined aberration may be introduced, for example, by modifying one or more electro-optical settings of the system, such as applying a defined focus offset, stigmation offset, or higher-order aberration using lenses, deflectors, or multipole elements. In further examples, the predefined aberration may be introduced by adjusting control signals of electro-optical components or by superimposing a known aberration pattern onto existing system settings. The predefined aberration is selected such that its magnitude and type are known or controllable.

[0305] In the third step G3, an aberrated subimage and corresponding beam let-specific aberration information are acquired while the predefined aberration is present. The aberrated subimage corresponds to the same or a comparable sample region as the reference subimage and reflects the image degradation caused by the introduced aberration. The corresponding beam let-specific aberration information is acquired under the same aberrated system state, enabling a direct association between image degradation and aberration characteristics.

[0306] In the fourth step G4, the aberrated subimage is associated with the reference subimage and the aberration information to form a training data triplet. The training data triplet represents a mapping between aberrated image information and restored image information under known aberration conditions. By repeating steps G1 to G4 fordifferent predefined aberrations, beamlets, or system states, a training data set is generated that captures the relationship between aberrations, aberration information, and corresponding image degradation. This training data set can then be used for training a machine learning image restoration model as described above.

[0307] The training data generation method described above is particularly well suited for multi-beam charged particle imaging systems. In such systems, a plurality of beamlets is available simultaneously, allowing reference subimages and aberrated subimages to be acquired in parallel for different beamlets under controlled system states. This enables efficient generation of large and diverse training data sets that capture beam-let-specific and shell-dependent aberration behavior without significantly increasing acquisition time. As a result, the training data generation method scales efficiently with increasing beamlet numbers and supports training of machine learning image restoration models for high-throughput multi-beam imaging applications.

[0308] The training data generated by the training data generation method described in connection with Fig. 31 is used as training data for the training method described in connection with Fig. 30. In particular, the aberrated subimages and associated beamlet-specific aberration information obtained in steps G1 to G4 are provided as input data, and the associated reference subimages are provided as reference images, for training the machine learning image restoration model used in the restoration methods described above. In this way, the machine learning image restoration model employed for image restoration is trained using training data generated under controlled and known aberration conditions.

[0309] Fig. 32 illustrates a excerpt of an exemplary training data set 419 generated using the method for generating training data as described above. Each triplet contains an aberrated subimage 188, aberration information 184 and a corresponding resetored image 182.

[0310] Fig. 33 schematically illustrates an embodiment in which image restoration is combined with control of electro-optical components 420, 420’ of a multi-beam charged particle imaging system 1 to partially compensate aberrations prior to image restoration.The multi-beam charged particle imaging system 1 comprises one or more electro-optical components 420, 420’ that are controlled by a control unit 800 to adjust the optical state of the system before image acquisition. The electro-optical components 420, 420’ may be controlled to compensate coarse aberrations, such as global defocus or global astigmatism affecting a plurality of beamlets, or to compensate beamlet-specific aberrations individually for different beamlets or beamlet groups. Such control may be based on predetermined settings, calibration data, or beamlet-specific aberration information 184.

[0311] After the hardware-based aberration compensation, an image 30 is acquired by a detector 600. Although the hardware-based correction reduces the magnitude of aberrations, residual aberrations typically remain, in particular beamlet-specific aberrations that cannot be fully compensated by hardware alone. The acquired image 30 is therefore subsequently processed by an image restoration method as described above, using beamlet-specific aberration information 184 to compensate the residual aberrations in the image domain.

[0312] In this way, Fig. 33 illustrates a hybrid approach in which hardware-based aberration compensation performed globally and / or on a beamlet-specific basis is combined with image-domain restoration. The hardware-based compensation reduces the overall aberration magnitude, while the subsequent image restoration improves image quality and uniformity across the image 30 without requiring fully beamlet-specific hardware correction.

[0313] In further examples, the image restoration method is combined with beamlet-specific hardware-based aberration compensation. One or more electro-optical components of the multi-beam charged particle imaging system are controlled on a beamlet-specific basis to compensate coarse aberrations prior to image acquisition, while residual aberrations are subsequently compensated in the image domain using beamlet-specific aberration information. In this way, the image restoration operates on partially corrected subimages, improving robustness of the restoration process and reducing the dynamic range of aberrations to be compensated, while enabling a flexible tradeoff between hardware complexity and software-based correction.

[0314] Fig. 34 schematically illustrates results obtained using the method 180 applied to a multi-field-of-view 160 of a multi-beam charged particle imaging system. The multi-field-of-view 160 comprises a plurality of subimages corresponding to different beam-lets or beamlet groups.

[0315] As illustrated, the method 180 improves non-uniform image resolution in a horizontal direction 424 and in a vertical direction 426 across the multi-field-of-view 160. In contrast to the shell-dependent and non-uniform resolution characteristics shown on the left and illustrated in earlier figures, the resolution obtained after application of the method 180 is more uniform across the multi-field-of-view 160.

[0316] Fig. 34 therefore demonstrates that the method 180 effectively compensates the shelldependent aberrations and information dispersion illustrated in the earlier figures and achieves improved image quality and resolution uniformity across the multi-field-of-view 160.

[0317] Fig. 35 schematically illustrates the dependence of image resolution on shell number 166 before and after application of the image restoration method. The horizontal axis represents the shell number 166, corresponding to increasing radial distance within the multi-field-of-view, while the vertical axis represents the resolution in a horizontal direction 424.

[0318] A first curve represents a resolution before restoration 428. As illustrated, the resolution before restoration 428 varies with shell number 166 and degrades for shells located farther from an optimal region, reflecting shell-dependent aberrations and information dispersion. This behavior limits the attainable resolution when operating the system using global optical settings.

[0319] A second curve represents a resolution after restoration 430 obtained by applying the image restoration method. As illustrated, the resolution after restoration 430 is increased relative to the resolution before restoration 428 for a plurality of shell numbers 166 and exhibits reduced dependence on shell number 166, indicating improved uniformity of resolution across different shells.

[0320] Fig. 35 further illustrates that resolution enhancement beyond the resolution before restoration 428 can be achieved when information about the object is available. The object information refers to any a priori information about the true structure of the sample being imaged that can be used to regularize or guide the restoration processto achieve super-resolution. The object information may comprise prior knowledge about object structure, sparsity, repetitiveness or a specific repeating pattern (as in a semiconductor device), or statistical properties learned from training data, knowledge that the object has sharp edges, positivity constraints (i.e., intensity values cannot be negative), or a known range of possible intensity values. Such super-resolution behavior may be obtained, for example, by deconvolution methods that exploit known or estimated system transfer characteristics, although the achievable improvement is limited by signal-to-noise ratio. By combining deconvolution-based restoration with machine-learning-based restoration, the results of deconvolution can be further refined, enabling the resolution after restoration 430 to be increased beyond what is achievable by deconvolution alone. As a result, Fig. 35 demonstrates that the combined restoration approach not only improves resolution uniformity across shell numbers 166 but also raises the overall level of image quality.

[0321] According to an embodiment illustrated in Fig. 36, a combined control and restoration method 432 is performed as a single coordinated workflow. In a first phase, state parameters 42 characterizing the optical state of the multi-beam charged particle imaging system are determined in steps P1 to P3, as described with respect to the embodiment illustrated in Fig. 4. The state parameters are derived from image data, such as subimages 34 of a multi-beam charged particle image or detector subimages 36, and may represent, for example, focus parameters or stigmation parameters.

[0322] Based on the determined state parameters 42, one or more electro-optical components of the system are controlled in a control step C to partially compensate aberrations in the hardware domain. Subsequently, partially corrected image data 434 is acquired while the system is operated under the adjusted optical state.

[0323] In a next phase, beamlet-specific aberration information 436 is obtained from the acquired data, for example in the form of a point spread function or parameters derived therefrom, in steps S1 to S3 as described with respect to the embodiment illustrated in Fig. 25. In step S3, image restoration is performed on the acquired image data using the beamlet-specific aberration information 436 in order to compensate residual aberrations in the image domain and generate a restored image 438.Fig. 36 thus illustrates a continuous workflow in which determination of state parameters, hardware-based aberration compensation, image acquisition, and image-do-main restoration are combined to improve image quality and uniformity while reducing alignment effort and avoiding fully beamlet-specific hardware correction.

[0324] Fig. 1 also schematically illustrates a multi-beam charged particle imaging system 1 according to a further embodiment of the invention. The system 1 comprises a beam generation apparatus including a multi-aperture arrangement 305 for generating a plurality of primary beamlets for scanning a sample arranged on a stage 500. The primary beamlets are guided toward the sample by an objective lens system and associated electro-optical components, and interaction products generated at the sample are detected by a detector 600 arranged in or close to a focal plane of a plurality of secondary beamlets.

[0325] The multi-beam charged particle imaging system 1 further comprises one or more processing devices and one or more machine-readable hardware storage devices arranged in a control unit 800. The machine-readable hardware storage devices store instructions that are executable by the one or more processing devices to perform operations comprising any one of the image restoration methods described above. In particular, the processing devices are configured to process image data acquired by the detector 600 and to perform image restoration using beamlet-specific aberration information, deconvolution algorithms, machine-learning-based restoration models, or combinations thereof, as described in connection with the preceding figures.

[0326] According to some examples, image restoration is performed purely in the software domain, without modifying the optical configuration of the multi-beam charged particle imaging system during image acquisition. In other examples, image restoration is combined with hardware-based aberration compensation, wherein one or more electro-optical components are arranged in one or more columns of the imaging system and are controlled to correct aberrations globally for a plurality of beamlets or locally for individual beamlets or beamlet groups. In such hybrid embodiments, the hardwarebased correction reduces the overall magnitude of aberrations prior to or during image acquisition, while the software-based image restoration compensates residual aberrations in the image domain. This combination enables flexible adaptation of the imaging system to different operating conditions while avoiding the need for fully beam-let-specific hardware correction.Reference throughout this specification to “an embodiment” or “an example” or “an aspect” means that a particular feature, structure or characteristic described in connection with the embodiment, example or aspect is included in at least one embodiment, example or aspect. Thus, appearances of the phrases “according to an embodiment”, “according to an example” or “according to an aspect” in various places throughout this specification are not necessarily all referring to the same embodiment, example or aspect, but may. Furthermore, the particular features or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more embodiments.

[0327] Furthermore, while some embodiments, examples or aspects described herein include some but not other features included in other embodiments, examples or aspects combinations of features of different embodiments, examples or aspects are meant to be within the scope of the claims, and form different embodiments, as would be understood by those skilled in the art.

[0328] Embodiments, examples and aspects of this invention can be described by the following clauses:

[0329] 1. A method 27 for determining one or more state parameters 42 of a multi-beam charged particle imaging system 1 comprising means 305 for generating a plurality of primary beamlets 3 in a raster configuration for the parallel scanning of a sample 7, and a detector 600 arranged in or close to a focal plane of a plurality of secondary beamlets 9 emitted from the sample 7, the method comprising:

[0330] Generating, by irradiation of a sample 7 with a plurality of primary beamlets 3, a plurality of secondary beamlets 9 that are emitted from the sample 7; For each of at least one primary beamlet 3: generating a subimage 34, 36 of an image 30, 32 from a signal recorded by the detector 600 for a corresponding secondary beamlet 9 generated in response to an irradiation of the sample 7 with this primary beamlet 3; and

[0331] Determining one or more state parameters 42 of the multi-beam charged particle imaging system 1 by analyzing the generated at least one subimage 34, 36 using a machine learning model 40 trained to map at least one subimage 34, 36 of an image 30, 32 generated by at least one primary beamlet 3 in a multi-beam charged particle imaging system 1 and / or characteristicsthereof to one or more state parameters 42 of the multi-beam charged particle imaging system 1.

[0332] The method of clause 1, wherein each subimage 36 of the image 32 comprises a point spread function 38 recorded by the detector 600 for a secondary beamlet 9 emitted from the sample 7 in response to an irradiation of the sample 7 with a primary beamlet 3.

[0333] The method of clause 2, wherein the image 32 is generated during a single irradiation of the sample 7 with the plurality of primary beamlets 3.

[0334] The method of clause 2, wherein a few images 32 are generated in response to a few irradiations of the sample 7 with the plurality of primary beamlets 3 using different imaging parameters, and wherein the one or more state parameters 42 of the multi-beam charged particle imaging system 1 are determined from the at least one subimage of the few generated images 36.

[0335] The method of clause 4, wherein the few generated images 32 are averaged, and wherein the one or more state parameters of the multi-beam charged particle imaging system 1 are determined from the at least one subimage of the average image.

[0336] The method of any one of clauses 2 to 5, wherein the characteristics of the at least one subimage comprise characteristics of point spread functions 38 of one or more secondary beamlets 9 within the at least one subimage 36 of the image 32.

[0337] The method of clause 1, wherein the image is generated by scanning the plurality of primary beamlets 3 over the sample 7 in parallel, recording a plurality of timely resolved intensity signals provided by the detector 600 and combining the intensity signals from all scan positions of each primary beamlet 3 to form a different subimage 34 of the image 30.

[0338] The method of any one of the preceding clauses, wherein the one or more state parameters 42 of the multi-beam charged particle imaging system 1 are from the group comprising focus parameters, astigmatism parameters, higher order aberration parameters, pitch parameters, beamlet current parameters, point spreadfunction parameters, beamlet intensity parameters, illumination field curvature parameters, illumination field inclination parameters, stigmation parameters.

[0339] 9. The method of any one of the preceding clauses, wherein the one or more state parameters of the multi-beam charged particle imaging system 1 comprise one or more state parameters of each of the at least one primary beamlet 3, and wherein the one or more state parameters of each of the at least one primary beamlet 3 are determined by analyzing the respective at least one generated subimage 34, 36 using a machine learning model 40 trained to map at least one subimage 34, 36 of an image 30, 32 generated by at least one primary beamlet 3 in a multi-beam charged particle imaging system 1 and / or characteristics thereof to one or more state parameters 42 of the at least one primary beamlet 3.

[0340] 10. The method of clause 9, the machine learning model 40 is trained to map a subimage 34, 36 of an image 30, 32 generated by a primary beamlet 3 in a multibeam charged particle imaging system 1 and / or characteristics thereof to one or more state parameters 42 of the primary beamlet 3.

[0341] 11. The method of any one of the preceding clauses, wherein the machine learning model 40 is a neural network.

[0342] 12. The method of clause 11, wherein the neural network is a residual neural network 54.

[0343] 13. The method of clause 11 or 12, wherein the neural network comprises at least one attention mechanism 900.

[0344] 14. The method of any one of the preceding clauses, wherein the machine learning model 40 generates a confidence score 80 estimating the accuracy of the predicted one or more state parameters 42 of the multi-beam charged particle imaging system 1.

[0345] 15. The method of clause 14, wherein the confidence score evaluates an ambiguity of the at least one subimage with respect to the predicted one or more state parameters of the multi-beam charged particle imaging system 1.16. The method of any one of the preceding clauses, wherein the primary beamlets 3 are grouped into at least two groups 82, 82’, 82”, wherein each group 82, 82’, 82” is associated with a group-specific machine learning model 40 trained for mapping at least one subimage 34, 36 of an image 30, 32 generated by at least one primary beamlet 3 of the group 82, 82’, 82” to one or more state parameters 42 of the multi-beam charged particle imaging system 1, and wherein the at least one subimage 34, 36 of the image 30, 32 generated by the at least one primary beamlet 3 is mapped to one or more state parameters 42 of the multi-beam charged particle imaging system 1 using the trained one or more group-specific machine learning models 40 associated with the one or more groups 82, 82’, 82” of the at least one primary beamlet 3.

[0346] 17. The method of clause 16, wherein the primary beamlets 3 are spatially grouped.

[0347] 18. The method of clause 16 or 17, wherein the groups correspond to different shells 162, 162’ of the plurality of primary beamlets 3.

[0348] 19. The method of any one of the preceding clauses, wherein each primary beamlet 3 is associated with a beamlet-specific machine learning model 40 trained for mapping the respective subimage 34, 36 of an image 30, 32 generated by the primary beamlet 3 to one or more state parameters 42 of the primary beamlet 3, and wherein the at least one subimage 34, 36 of the image 30, 32 generated by the at least one primary beamlet 3 is mapped to one or more state parameters 42 of the at least one primary beamlet 3 using the at least one corresponding trained beamlet-specific machine learning model 40.

[0349] 20. The method of any one of the preceding clauses, further comprising controlling the one or more determined state parameters 42 of the multi-beam charged particle imaging system 1 using electro-optical components of the multi-beam charged particle imaging system 1.

[0350] 21. The method of clause 20, wherein the steps of the method are iterated.

[0351] 22. A computer implemented method for training a machine learning model 40 for mapping at least one subimage 34, 36 of an image 30, 32 generated by at least one primary beamlet 3 in a multi-beam charged particle imaging system 1 to one or more state parameters 42 of the multi-beam charged particle imaging system 1 according to any one of the preceding clauses.A computer implemented method 84 for training a plurality of group-specific machine learning models 40 that each map at least one subimage 34, 36 of an image 30, 32 generated by at least one primary beamlet 3 from a group 82, 82’, 82” of primary beamlets 3 of a multi-beam charged particle imaging system 1 to one or more state parameters 42 of the multi-beam charged particle imaging system 1, the method comprising:

[0352] Providing a prototype machine learning model for mapping at least one subimage 34, 36 of an image 30, 32 generated by at least one primary beamlet 3 from a group 82, 82’, 82” of primary beamlets 3 of a multi-beam charged particle imaging system 1 to one or more state parameters 42 of the multibeam charged particle imaging system 1;

[0353] For each group 82, 82’, 82” of primary beamlets 3, providing group-specific training data 92’ comprising at least one subimage 34, 36 of one or more images 30, 32 obtained by at least one primary beamlet 3 of the group 82, 82’, 82” of primary beamlets 3 of the multi-beam charged particle imaging system 1 and corresponding one or more state parameters 42 of the multibeam charged particle imaging system 1, and training the corresponding group-specific machine learning model 40 by fine-tuning the prototype machine learning model using the group-specific training data 92’.

[0354] The method of clause 23, wherein each group 82, 82’, 82” of primary beamlets 3 of the multi-beam charged particle imaging system 1 comprises a single primary beamlet 3.

[0355] The method of clause 23 or 24, wherein the plurality of trained group-specific machine learning models 40 is used in a method for determining one or more state parameters 42 of a multi-beam charged particle imaging system 1 according to any one of clauses 16 to 19.

[0356] A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out a computer implemented method for training one or more machine learning models according to any one of clauses 22 to 25.A computer-readable medium, on which a computer program executable by a computing device is stored, the computer program comprising code for executing a computer implemented method for training one or more machine learning models according to any one of clauses 22 to 25.

[0357] A multi-beam charged particle imaging system 1 comprising:

[0358] means 305 for generating a plurality of primary beamlets 3 in a raster configuration for the parallel scanning of a sample 7;

[0359] a detector 600 arranged in a focal plane of a plurality of secondary beamlets 9 emitted from the sample 7;

[0360] a data processing unit 700 for determining state parameters 42 of the multi-beam charged particle imaging system 1 and a control unit 702 for controlling state parameters 42 of the multi-beam charged particle imaging system 1 using a method according to clause 20 or 21.

[0361] A method 180 for restoring an image 30 in a multi-beam charged particle imaging system 1, the system comprising means 305 for generating a plurality of primary beamlets 3, and a detector 600 arranged in or close to a focal plane of a plurality of secondary beamlets 9, the method comprising:

[0362] acquiring a plurality of aberrated subimages 34, wherein each aberrated subimage is generated from signals recorded by the detector 600 and corresponds to an area of a sample 7 scanned by a respective one of the plurality of primary beamlets 3;

[0363] obtaining, for each of the plurality of primary beamlets 3, beamlet-specific aberration information 184 comprising a point spread function measured by the detector 600 or parameters derived therefrom,

[0364] generating a plurality of restored subimages by applying, for each aberrated subimage 34, a restoration method 181 that uses the respective aberrated subimage 34 and the corresponding beamlet-specific aberration information 184 as inputs to generate a restored subimage 182, and generating a restored image from the plurality of restored subimages 182.

[0365] The method of clause 29, wherein the plurality of primary beamlets 3 is grouped into at least two groups, and wherein the restoration method 181 is applied using group-specific aberration information.31. The method of clause 30, wherein the groups correspond to different shells 162, 162’ of the plurality of primary beamlets 3.

[0366] 32. The method of clause 29, wherein the restoration method 181 is applied independently for each primary beamlet 3.

[0367] 33. The method of any one of clauses 29 to 32, further comprising controlling one or more electro-optical components 420 of the multi-beam charged particle imaging system 1 to partially compensate aberrations prior to restoring the image 30.

[0368] 34. The method of any one of clauses 29 to 33, wherein the detector 600 comprises a scintillator, and wherein the beamlet-specific aberration information 184 is obtained from a scintillator image acquired by the detector 600.

[0369] 35. The method of any one of clauses 29 to 34, wherein the restoration method 181 restores spatial frequency information beyond a resolution limit of the aberrated subimages 34 using known or constrained object information.

[0370] 36. The method of any one of clauses 29 to 35, wherein the restoration method 181 comprises a deconvolution algorithm 190 that uses the beamlet-specific point spread function 184 as a kernel for deconvolving the respective aberrated subimage 188.

[0371] 37. The method of any one of clauses 29 to 35, wherein the restoration method comprises a constrained deconvolution algorithm 192 configured to estimate a beamlet-specific point spread function while simultaneously restoring the subimage 188, and wherein the obtained beamlet-specific aberration information 184 is used to provide one or more constraints 194 for the estimation of the point spread function.

[0372] 38. The method of any one of clauses 29 to 37, wherein the restoration method 181 comprises a machine learning image restoration model 404 configured to map an aberrated subimage 188 and corresponding beamlet-specific aberration information 184 to a restored subimage 182.

[0373] 39. The method of clause 38, wherein the beamlet-specific aberration information 184 is provided to the machine learning image restoration model 404 as an additional input channel or as conditioning information.40. The method of clause 38 or 39, wherein the machine learning image restoration model 404 comprises a convolutional neural network, a residual neural network 54, or a neural network comprising at least one attention mechanism 900.

[0374] 41. The method of clause 40, wherein the convolutional neural network comprises an encoder-decoder architecture 406.

[0375] 42. The method of clause 38, wherein the restoration method 181 comprises a combination of a deconvolution algorithm 190, 192 and a machine learning image restoration model 404, the machine learning image restoration model 404 being configured to map an aberrated subimage 188 deconvolved using the beamlet- specific point spread function as a kernel to a restored subimage 182.

[0376] 43. A method 416 for training a machine learning image restoration model 404 for restoring images 30 in a multi-beam charged particle imaging system 1, the system comprising means 305 for generating a plurality of primary beamlets 3, and a detector 600 arranged in or close to a focal plane of a plurality of secondary beamlets 9, the method comprising:

[0377] acquiring, for each of a plurality of primary beamlets 3, training data 419 comprising at least one aberrated subimage 188 generated from signals recorded by the detector 600 and corresponding beamlet-specific aberration information 184 comprising a point spread function measured by the detector 600 or parameters derived therefrom as input data;

[0378] providing, for each aberrated subimage 188, a corresponding reference subimage 422 representing restored image information 182 as output data; and training the machine learning image restoration model 404 using the training data 419.

[0379] 44. The method of clause 43, wherein the reference images 422 are generated from images acquired using different imaging conditions of the multi-beam charged particle imaging system 1.

[0380] 45. The method of clause 43 or 44, wherein the beamlet-specific aberration information 184 is provided to the machine learning image restoration model 404 as an additional input channel or as conditioning information during training.46. The method of any one of clauses 43 to 45, wherein the plurality of primary beam- lets 3 is grouped into at least two groups, and wherein the machine learning image restoration model 404 is trained using group-specific training data.

[0381] 47. The method of clause 46, wherein the groups correspond to different shells 162, 162’ of the plurality of primary beamlets 3.

[0382] 48. The method of any one of clauses 43 to 47, wherein a single machine learning image restoration model 404 is trained for restoring images corresponding to a plurality of primary beamlets 3.

[0383] 49. The method of any one of clauses 43 to 47, wherein separate machine learning image restoration models 404 are trained for different primary beamlets 3 or different groups of primary beamlets.

[0384] 50. The method of any one of clauses 38 to 42, wherein the machine learning image restoration model 404 is trained using a training method 416 according to any of clauses 43 to 49.

[0385] 51. A method 418 for generating training data 419 for training a machine learning image restoration model 404 configured for image restoration in a multi-beam charged particle imaging system 1, comprising:

[0386] acquiring, for at least one primary beamlet 3, a reference subimage 422; introducing a predefined aberration in the multi-beam charged particle imaging system 1;

[0387] acquiring an aberrated subimage 188 and corresponding beamlet-specific aberration information 184; and

[0388] associating the aberrated subimage 188 with the reference subimage 422 and the beamlet-specific aberration information 184 as a training data triplet of the training data 419.

[0389] 52. The method of clause 51, wherein the generated training data 419 is used for training a machine learning image restoration model 404 according to any one of clauses 43 to 50.

[0390] 53. A computer program comprising instructions which, when executed by a data processing system, cause the data processing system to carry out the trainingmethod 416 or training data generation method 418 according to any one of clauses 43 to 52.

[0391] 54. A computer-readable medium storing instructions which, when executed by a data processing system, cause the data processing system to carry out the training method 416 or training data generation method 418 according to any one of clauses 43 to 52.

[0392] 55. A multi-beam charged particle imaging system 1 comprising:

[0393] means 305 for generating a plurality of primary beamlets 3 for scanning a sample 7;

[0394] a detector 600 arranged in or close to a focal plane of a plurality of secondary beamlets 9;

[0395] one or more processing devices 700;

[0396] one or more machine-readable hardware storage devices 76 comprising instructions that are executable by one or more processing devices 700 to perform operations comprising the image restoration method 181 according to any one of clauses 29 to 42.

[0397] 56. The method of any one of clauses 1 to 28, comprising controlling one or more electro-optical components 420 of the multi-beam charged particle imaging system 1 based on determined state parameters 42, and subsequently restoring the image 30 using the image restoration method 181 according to any one of clauses 29 to 42.

[0398] In summary, the invention relates to a method for determining one or more state parameters 42 of a multi-beam charged particle imaging system, the method comprising: generating, by irradiation of a sample with a plurality of primary beamlets, a plurality of secondary beamlets that are emitted from a sample; for each of at least one primary beamlet: generating a subimage 34, 36 of an image from a signal recorded by thedetector for a corresponding secondary beamlet generated in response to an irradiation of the sample with this primary beamlet; and determining one or more state parameters 42 of the multi-beam charged particle imaging system by analyzing the respective generated subimage 34, 36 using a machine learning model 40 trained to map at least one subimage 34, 36 of an image generated by at least one primary beamlet or characteristics thereof to one or more state parameters 42 of the multibeam charged particle imaging system 1.Reference number list

[0399] 1 Multi-beam charged particle imaging system 3 Primary beamlet

[0400] 5 Focus spots

[0401] 7 Sample

[0402] 9 Secondary beamlet

[0403] 13 Detection beam path

[0404] 15 Focus point

[0405] 21 Pupil plane

[0406] 21a Crossover plane

[0407] 21b Second pupil plane

[0408] 25 Surface

[0409] 27 Method

[0410] 30 Multi-beam charged particle imaging system image 32, 32’ Detector image

[0411] 34 Subimage

[0412] 36, 36’ Detector subimage

[0413] 38 Point spread function

[0414] 39 Characteristics

[0415] 40 Machine learning model

[0416] 42 State parameters

[0417] 44 Control step

[0418] 45 Iteration

[0419] 46 State parameter selection step

[0420] 48 Image acquisition step

[0421] 50 Training step

[0422] 52 Iteration

[0423] 54 Residual neural network

[0424] 56 3x3 convolution layer

[0425] 58 Max pooling layer

[0426] 60 Fully connected layer

[0427] 62 Skip connection

[0428] 64 Image acquisition step

[0429] 66 Analysis step

[0430] 68 Comparison stepOptimum reached

[0431] Exit step

[0432] Optimum not reached

[0433] Correction step

[0434] Repetition

[0435] Confidence score

[0436] , 82’, 82”, 82’” Group

[0437] Computer implemented method Pre-training step

[0438] Fine-tuning step

[0439] Training data

[0440] ’ Group-specific training data

[0441] Focus value

[0442] Astigmatism value in X direction Astigmatism value in Y direction

[0443] 0 Sample irradiation unit

[0444] 1 Sample plane

[0445] 2 Objective lens system

[0446] 3 Field lens

[0447] 8 Crossover point

[0448] 0 First scanning deflector

[0449] 2 Multi-pole element

[0450] 3 Scanning direction

[0451] 4 Beam shaper plate

[0452] 6 Field curvature corrector

[0453] 0 Focus offset

[0454] 2 Additional hardware

[0455] 4 Stigmation corrector

[0456] 6 Stigmated beam profiles at sample plane 8 Corrected beam profiles at sample plane 0 Multi-field-of-view

[0457] 2, 162’ Shell

[0458] 4 Amount of information dispersion 6 Shell number

[0459] 8 DefocusAstigmatism

[0460] Production cost I complexity Number of beams

[0461] Hardware-based correction Software-based correction

[0462] Method

[0463] Restoration method

[0464] Restored image

[0465] Beamlet-specific aberration information Noise

[0466] Aberrated image

[0467] Deconvolution method

[0468] Constrained deconvolution method Contraints

[0469] Secondary electron imaging system Projection lens

[0470] Filter module

[0471] Beam deflector

[0472] Detection plane

[0473] Monitoring system

[0474] CMOS sensor

[0475] Imaging lens

[0476] Beam divider mirror

[0477] Scanning line

[0478] Flyback

[0479] Image patch

[0480] Surface area segment

[0481] a, 284b Aperture stop

[0482] Beam generation apparatus

[0483] Particle emitter

[0484] Collimation lens

[0485] First multi-aperture plate

[0486] Multi-aperture arrangement

[0487] Further multi-aperture plate Illuminating charged particle beamSurface

[0488] First field lens

[0489] Second field lens

[0490] Beam divider

[0491] Machine learning image restoration model Encoder-decoder architecture

[0492] Encoder

[0493] Skip connection

[0494] Decoder

[0495] Input to different layers

[0496] Bottleneck

[0497] Training method

[0498] Training data generation method Training data samples

[0499] Electro-optical component

[0500] Reference subimage

[0501] Resolution in horizontal direction Resolution in vertical direction Resolution before restoration

[0502] Resolution after restoration

[0503] Combined control and restoration method Partially corrected image

[0504] Partially corrected detector image Restored subimage

[0505] Stage

[0506] Extraction voltage module

[0507] Detector

[0508] Electron to photon conversion unit Scintillator image acquisition unit Collection lens

[0509] Zoom lens

[0510] Photon detection element

[0511] Data processing unit

[0512] Control unit

[0513] Control unit810 Imaging control module

[0514] 820 Monitoring control unit

[0515] 830 Control module

[0516] 840 Control module

[0517] 850 Stage control module

[0518] 860 Scanning control unit

[0519] 880 Processor

[0520] 890 Memory

[0521] 900 Attention mechanism

[0522] 902 Token

[0523] 902’ Contextual information

[0524] 904 Multilayer perceptron

[0525] 906 Similarity function

[0526] 906’ Attention distribution

[0527] 908 Aggregation function

[0528] 910 Attention-based representation 912 Attention-weighted values 914 Query

[0529] 916 Key

[0530] 918 Value

[0531] 920 Transformer block

[0532] 922 Multihead attention

[0533] 924 Add & norm layer

[0534] 926 Feed forward neural network 928 Positional encoding

[0535] 2105 Optical axis

Claims

Claims1. A method (27) for determining one or more state parameters (42) of a multi-beam charged particle imaging system (1) comprising means (305) for generating a plurality of primary beamlets (3) in a raster configuration for the parallel scanning of a sample (7), and a detector (600) arranged in or close to a focal plane of a plurality of secondary beamlets (9) emitted from the sample (7), the method comprising:Generating, by irradiation of a sample (7) with a plurality of primary beamlets (3), a plurality of secondary beamlets (9) that are emitted from the sample (7);For each of at least one primary beamlet (3): generating a subimage (34, 36) of an image (30, 32) from a signal recorded by the detector (600) for a corresponding secondary beamlet (9) generated in response to an irradiation of the sample (7) with this primary beamlet (3); andDetermining one or more state parameters (42) of the multi-beam charged particle imaging system (1) by analyzing the generated at least one subimage (34, 36) using a machine learning model (40) trained to map at least one subimage (34, 36) of an image (30, 32) generated by at least one primary beamlet (3) in a multi-beam charged particle imaging system (1) and / or characteristics thereof to one or more state parameters (42) of the multi-beam charged particle imaging system (1).

2. The method of claim 1, wherein each subimage (36) of the image (32) comprises a point spread function (38) recorded by the detector (600) for a secondary beam- let (9) emitted from the sample (7) in response to an irradiation of the sample (7) with a primary beamlet (3).

3. The method of claim 2, wherein the image (32) is generated during a single irradiation of the sample (7) with the plurality of primary beamlets (3).

4. The method of claim 2, wherein a few images (32) are generated in response to a few irradiations of the sample (7) with the plurality of primary beamlets (3) using different imaging parameters, and wherein the one or more state parameters (42) of the multi-beam charged particle imaging system (1) are determined from the at least one subimage of the few generated images (36).

5. The method of claim 4, wherein the few generated images (32) are averaged, and wherein the one or more state parameters of the multi-beam charged particle imaging system (1) are determined from the at least one subimage of the average image.

6. The method of any one of claims 2 to 5, wherein the characteristics of the at least one subimage comprise characteristics of point spread functions (38) of one or more secondary beamlets (9) within the at least one subimage (36) of the image (32).

7. The method of claim 1, wherein the image is generated by scanning the plurality of primary beamlets (3) over the sample (7) in parallel, recording a plurality of timely resolved intensity signals provided by the detector (600) and combining the intensity signals from all scan positions of each primary beamlet (3) to form a different subimage (34) of the image (30).

8. The method of any one of the preceding claims, wherein the one or more state parameters (42) of the multi-beam charged particle imaging system (1) are from the group comprising focus parameters, astigmatism parameters, higher order aberration parameters, pitch parameters, beamlet current parameters, point spread function parameters, beamlet intensity parameters, illumination field curvature parameters, illumination field inclination parameters, stigmation parameters.

9. The method of any one of the preceding claims, wherein the one or more state parameters of the multi-beam charged particle imaging system (1) comprise one or more state parameters of each of the at least one primary beamlet (3), and wherein the one or more state parameters of each of the at least one primary beamlet (3) are determined by analyzing the respective at least one generated subimage (34, 36) using a machine learning model (40) trained to map at least one subimage (34, 36) of an image (30, 32) generated by at least one primary beamlet (3) in a multi-beam charged particle imaging system (1) and / or characteristics thereof to one or more state parameters (42) of the at least one primary beamlet (3).

10. The method of claim 9, the machine learning model (40) is trained to map a subimage (34, 36) of an image (30, 32) generated by a primary beamlet (3) in a multibeam charged particle imaging system (1) and / or characteristics thereof to one or more state parameters (42) of the primary beamlet (3).

11. The method of any one of the preceding claims, wherein the machine learning model (40) is a neural network.

12. The method of claim 11, wherein the neural network is a residual neural network (54).

13. The method of claim 11 or 12, wherein the neural network comprises at least one attention mechanism (900).

14. The method of any one of the preceding claims, wherein the machine learning model (40) generates a confidence score (80) estimating the accuracy of the predicted one or more state parameters (42) of the multi-beam charged particle imaging system (1).

15. The method of claim 14, wherein the confidence score evaluates an ambiguity of the at least one subimage with respect to the predicted one or more state parameters of the multi-beam charged particle imaging system (1).

16. The method of any one of the preceding claims, wherein the primary beamlets (3) are grouped into at least two groups (82, 82’, 82”), wherein each group (82, 82’, 82”) is associated with a group-specific machine learning model (40) trained for mapping at least one subimage (34, 36) of an image (30, 32) generated by at least one primary beamlet (3) of the group (82, 82’, 82”) to one or more state parameters (42) of the multi-beam charged particle imaging system (1), and wherein the at least one subimage (34, 36) of the image (30, 32) generated by the at least one primary beamlet (3) is mapped to one or more state parameters (42) of the multi-beam charged particle imaging system (1) using the trained one or more group-specific machine learning models (40) associated with the one or more groups (82, 82’, 82”) of the at least one primary beamlet (3).

17. The method of claim 16, wherein the primary beamlets (3) are spatially grouped.

18. The method of claim 16 or 17, wherein the groups correspond to different shells (162, 162’) of the plurality of primary beamlets (3).

19. The method of any one of the preceding claims, wherein each primary beamlet (3) is associated with a beamlet-specific machine learning model (40) trained for mapping the respective subimage (34, 36) of an image (30, 32) generated by the primary beamlet (3) to one or more state parameters (42) of the primary beamlet (3), and wherein the at least one subimage (34, 36) of the image (30, 32) generated by the at least one primary beamlet (3) is mapped to one or more state parameters (42) of the at least one primary beamlet (3) using the at least one corresponding trained beamlet-specific machine learning model (40).

20. The method of any one of the preceding claims, further comprising controlling the one or more determined state parameters (42) of the multi-beam charged particle imaging system (1) using electro-optical components of the multi-beam charged particle imaging system (1).

21. The method of claim 20, wherein the steps of the method are iterated.

22. A computer implemented method for training a machine learning model (40) for mapping at least one subimage (34, 36) of an image (30, 32) generated by at least one primary beamlet (3) in a multi-beam charged particle imaging system (1) to one or more state parameters (42) of the multi-beam charged particle imaging system (1) according to any one of the preceding claims.

23. A computer implemented method (84) for training a plurality of group-specific machine learning models (40) that each map at least one subimage (34, 36) of an image (30, 32) generated by at least one primary beamlet (3) from a group (82, 82’, 82”) of primary beamlets (3) of a multi-beam charged particle imaging system (1) to one or more state parameters (42) of the multi-beam charged particle imaging system (1), the method comprising:Providing a prototype machine learning model for mapping at least one subimage (34, 36) of an image (30, 32) generated by at least one primary beam- let (3) from a group (82, 82’, 82”) of primary beamlets (3) of a multi-beam charged particle imaging system (1) to one or more state parameters (42) of the multi-beam charged particle imaging system (1);For each group (82, 82’, 82”) of primary beamlets (3), providing group-specific training data (92’) comprising at least one subimage (34, 36) of one or more images (30, 32) obtained by at least one primary beamlet (3) of the group (82, 82’, 82”) of primary beamlets (3) of the multi-beam charged particle imaging system (1) and corresponding one or more state parameters (42) of the multi-beam charged particle imaging system (1), and training the corresponding group-specific machine learning model (40) by fine-tuning the prototype machine learning model using the group-specific training data (92’).

24. The method of claim 23, wherein each group (82, 82’, 82”) of primary beamlets (3) of the multi-beam charged particle imaging system (1) comprises a single primary beamlet (3).

25. The method of claim 23 or 24, wherein the plurality of trained group-specific machine learning models (40) is used in a method for determining one or more state parameters (42) of a multi-beam charged particle imaging system (1) according to any one of claims 16 to 19.

26. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out a computer implemented method for training one or more machine learning models according to any one of claims 22 to 25.

27. A computer-readable medium, on which a computer program executable by a computing device is stored, the computer program comprising code for executing a computer implemented method fortraining one or more machine learning models according to any one of claims 22 to 25.

28. A multi-beam charged particle imaging system (1) comprising:means (305) for generating a plurality of primary beamlets (3) in a raster configuration for the parallel scanning of a sample (7);a detector (600) arranged in a focal plane of a plurality of secondary beamlets (9) emitted from the sample (7);a data processing unit (700) for determining state parameters (42) of the multibeam charged particle imaging system (1) and a control unit (702) for controllingstate parameters (42) of the multi-beam charged particle imaging system (1) using a method according to claim 20 or 21.

29. A method (180) for restoring an image (30) in a multi-beam charged particle imaging system (1), the system comprising means (305) for generating a plurality of primary beamlets (3), and a detector (600) arranged in or close to a focal plane of a plurality of secondary beamlets (9), the method comprising:acquiring a plurality of aberrated subimages (34), wherein each aberrated subimage is generated from signals recorded by the detector (600) and corresponds to an area of a sample (7) scanned by a respective one of the plurality of primary beamlets (3);obtaining, for each of the plurality of primary beamlets (3), beamlet-specific aberration information (184) comprising a point spread function measured by the detector (600) or parameters derived therefrom,generating a plurality of restored subimages by applying, for each aberrated subimage (34), a restoration method (181) that uses the respective aberrated subimage (34) and the corresponding beamlet-specific aberration information (184) as inputs to generate a restored subimage (182), and generating a restored image from the plurality of restored subimages (182).

30. The method of claim 29, wherein the plurality of primary beamlets (3) is grouped into at least two groups, and wherein the restoration method (181) is applied using group-specific aberration information.

31. The method of claim 30, wherein the groups correspond to different shells (162, 162’) of the plurality of primary beamlets (3).

32. The method of claim 29, wherein the restoration method (181) is applied independently for each primary beamlet (3).

33. The method of any one of claims 29 to 32, further comprising controlling one or more electro-optical components (420) of the multi-beam charged particle imaging system (1) to partially compensate aberrations prior to restoring the image (30).

34. The method of any one of claims 29 to 33, wherein the detector (600) comprises a scintillator, and wherein the beamlet-specific aberration information (184) is obtained from a scintillator image acquired by the detector (600).

35. The method of any one of claims 29 to 34, wherein the restoration method (181) restores spatial frequency information beyond a resolution limit of the aberrated subimages (34) using known or constrained object information.

36. The method of any one of claims 29 to 35, wherein the restoration method (181) comprises a deconvolution algorithm (190) that uses the beamlet-specific point spread function (184) as a kernel for deconvolving the respective aberrated subimage (188).

37. The method of any one of claims 29 to 35, wherein the restoration method comprises a constrained deconvolution algorithm (192) configured to estimate a beamlet-specific point spread function while simultaneously restoring the subimage (188), and wherein the obtained beamlet-specific aberration information (184) is used to provide one or more constraints (194) for the estimation of the point spread function.

38. The method of any one of claims 29 to 37, wherein the restoration method (181) comprises a machine learning image restoration model (404) configured to map an aberrated subimage (188) and corresponding beamlet-specific aberration information (184) to a restored subimage (182).

39. The method of claim 38, wherein the beamlet-specific aberration information (184) is provided to the machine learning image restoration model (404) as an additional input channel or as conditioning information.

40. The method of claim 38 or 39, wherein the machine learning image restoration model (404) comprises a convolutional neural network, a residual neural network (54), or a neural network comprising at least one attention mechanism (900).

41. The method of claim 40, wherein the convolutional neural network comprises an encoder-decoder architecture (406).

42. The method of claim 38, wherein the restoration method (181) comprises a combination of a deconvolution algorithm (190, 192) and a machine learning imagerestoration model (404), the machine learning image restoration model (404) being configured to map an aberrated subimage (188) deconvolved using the beam- let-specific point spread function as a kernel to a restored subimage (182).

43. A computer implemented method (416) for training a machine learning image restoration model (404) for restoring images (30) in a multi-beam charged particle imaging system (1), the system comprising means (305) for generating a plurality of primary beamlets (3), and a detector (600) arranged in or close to a focal plane of a plurality of secondary beamlets (9), the method comprising:acquiring, for each of a plurality of primary beamlets (3), training data (419) comprising at least one aberrated subimage (188) generated from signals recorded by the detector (600) and corresponding beamlet-specific aberration information (184) comprising a point spread function measured by the detector (600) or parameters derived therefrom as input data; providing, for each aberrated subimage (188), a corresponding reference subimage (422) representing restored image information (182) as output data; andtraining the machine learning image restoration model (404) using the training data (419).

44. The method of claim 43, wherein the reference images (422) are generated from images acquired using different imaging conditions of the multi-beam charged particle imaging system (1).

45. The method of claim 43 or 44, wherein the beamlet-specific aberration information (184) is provided to the machine learning image restoration model (404) as an additional input channel or as conditioning information during training.

46. The method of any one of claims 43 to 45, wherein the plurality of primary beam- lets (3) is grouped into at least two groups, and wherein the machine learning image restoration model (404) is trained using group-specific training data.

47. The method of claim 46, wherein the groups correspond to different shells (162, 162’) of the plurality of primary beamlets (3).

48. The method of any one of claims 43 to 47, wherein a single machine learning image restoration model (404) is trained for restoring images corresponding to a plurality of primary beamlets (3).

49. The method of any one of claims 43 to 47, wherein separate machine learning image restoration models (404) are trained for different primary beamlets (3) or different groups of primary beamlets.

50. The method of any one of claims 38 to 42, wherein the machine learning image restoration model (404) is trained using a training method (416) according to any of claims 43 to 49.

51. A computer implemented method (418) for generating training data (419) for training a machine learning image restoration model (404) configured for image restoration in a multi-beam charged particle imaging system (1), comprising:acquiring, for at least one primary beamlet (3), a reference subimage (422); introducing a predefined aberration in the multi-beam charged particle imaging system (1);acquiring an aberrated subimage (188) and corresponding beam let-specific aberration information (184); andassociating the aberrated subimage (188) with the reference subimage (422) and the beamlet-specific aberration information (184) as a training data triplet of the training data (419).

52. The method of claim 51, wherein the generated training data (419) is used for training a machine learning image restoration model (404) according to any one of claims 43 to 50.

53. A computer program comprising instructions which, when executed by a data processing system, cause the data processing system to carry out the training method (416) or training data generation method (418) according to any one of claims 43 to 52.

54. A computer-readable medium storing instructions which, when executed by a data processing system, cause the data processing system to carry out the training method (416) or training data generation method (418) according to any one of claims 43 to 52.

55. A multi-beam charged particle imaging system (1) comprising:means (305) for generating a plurality of primary beamlets (3) for scanning a sample (7);a detector (600) arranged in or close to a focal plane of a plurality of secondary beamlets (9);one or more processing devices (700);one or more machine-readable hardware storage devices (76) comprising instructions that are executable by one or more processing devices (700) to perform operations comprising the image restoration method (181) according to any one of claims 29 to 42.

56. The method of any one of claims 1 to 28, comprising controlling one or more electro-optical components (420) of the multi-beam charged particle imaging system (1) based on determined state parameters (42), and subsequently restoring the image (30) using the image restoration method (181) according to any one of claims 29 to 42.

Images