Image distortion correction in charged particle inspection

Abstract

Improved systems and methods for correcting distortion in an inspection image are disclosed. The improved method for correcting distortion in an inspection image includes acquiring an inspection image, aligning a plurality of patches of the inspection image based on a reference image corresponding to the inspection image, evaluating an alignment between each patch of the plurality of patches and a corresponding patch of the reference image with a machine learning model, determining local alignment results for the plurality of patches of the inspection image based on the reference image corresponding to the inspection image, determining an alignment model based on the local alignment results, and correcting the distortion in the inspection image based on the alignment model.

Description

[Technical Field]

[0001] Cross-reference of related applications

[0001] This application claims priority to U.S. Patent Application No. 63 / 220,370, filed on 9 July 2021 and incorporated herein by reference in its entirety.

[0002]

[0002] Embodiments provided herein relate to image enhancement techniques, and more specifically, to distortion correction mechanisms for charged particle beam inspection images. [Background technology]

[0003]

[0003] In the manufacturing process of integrated circuits (ICs), unfinished or finished circuit components are inspected to ensure that they are manufactured according to design and free from defects. Inspection systems using charged particle (e.g., electron) beam microscopes or optical microscopes, such as scanning electron microscopes (SEMs), may be used. As the physical size of IC components continues to shrink, accuracy and yield in defect detection become more important. Inspection images, such as SEM images, can be used to identify or classify defects in the manufactured ICs. To improve the performance of defect detection, it is desirable to obtain accurate SEM images that are free from distortion and misalignment. [Overview of the project]

[0004]

[0004] Embodiments provided herein disclose particle beam inspection devices, more specifically inspection devices that use a plurality of charged particle beams.

[0005]

[0005] In some embodiments, methods are provided for correcting distortion in an inspection image. These methods include acquiring an inspection image, determining local alignment results for multiple patches of the inspection image based on a reference image corresponding to the inspection image, determining an alignment model for each subset of multiple subsets of local alignment results based on the subset of local alignment results, evaluating the alignment model based on a fit of the alignment model to the residual set of local alignment results, selecting one alignment model from the multiple alignment models based on the evaluation, and correcting distortion in the inspection image based on the selected alignment model.

[0006]

[0006] In some embodiments, a device is provided for correcting distortion in an inspection image. The device includes a memory for storing a set of commands and at least one processor configured to execute a set of commands causing the device to acquire an inspection image, determine local alignment results for multiple patches of the inspection image based on a reference image corresponding to the inspection image, determine an alignment model for each subset of multiple subsets of local alignment results based on the subset of local alignment results, evaluate the alignment model based on a fit of the alignment model to the residual set of local alignment results, select one alignment model from the multiple alignment models based on the evaluation, and correct distortion in the inspection image based on the selected alignment model.

[0007]

[0007] A non-temporary computer-readable medium is provided that stores a set of instructions executable on at least the processor of a computing device for causing the computing device to perform a method for correcting distortion of an inspection image. The method includes acquiring an inspection image, determining local alignment results for multiple patches of the inspection image based on a reference image corresponding to the inspection image, determining an alignment model for each subset of multiple subsets of local alignment results based on the subset of local alignment results, evaluating the alignment model based on the fit of the alignment model to the residual set of local alignment results, selecting one alignment model from the multiple alignment models based on the evaluation, and correcting distortion of the inspection image based on the selected alignment model.

[0008]

[0008] In some embodiments, methods are provided for correcting distortion in an inspection image. These methods include acquiring an inspection image, determining local alignment results for multiple patches of the inspection image based on a reference image corresponding to the inspection image, estimating a first alignment model based on a first subset of the local alignment results, estimating a second alignment model based on a second subset of the local alignment results, evaluating the first alignment model based on a fit of the first alignment model to a first residual set of local alignment results, evaluating the second alignment model based on a fit of the second alignment model to a second residual set of local alignment results, selecting one of the first and second alignment models based on the evaluation, and correcting distortion in the inspection image based on the selected alignment model.

[0009]

[0009] In some embodiments, a device is provided for correcting distortion in an inspection image. The device includes a memory for storing a set of commands and at least one processor configured to execute a set of commands causing the device to acquire an inspection image, determine local alignment results for multiple patches of the inspection image based on a reference image corresponding to the inspection image, estimate a first alignment model based on a first subset of the local alignment results, estimate a second alignment model based on a second subset of the local alignment results, evaluate a first alignment model based on a fit of the first alignment model to a first residual set of local alignment results, evaluate a second alignment model based on a fit of the second alignment model to a second residual set of local alignment results, select one of the first and second alignment models based on the evaluation, and correct distortion in the inspection image based on the selected alignment model.

[0010]

[0010] A non-temporary computer-readable medium is provided which stores a set of instructions that can be executed on at least the processor of a computing device in order to cause the computing device to perform a method for correcting distortion of an inspection image. The method includes acquiring an inspection image, determining local alignment results for multiple patches of the inspection image based on a reference image corresponding to the inspection image, estimating a first alignment model based on a first subset of the local alignment results, estimating a second alignment model based on a second subset of the local alignment results, evaluating a first alignment model based on a fit of the first alignment model to a first residual set of local alignment results, evaluating a second alignment model based on a fit of the second alignment model to a second residual set of local alignment results, selecting one of the first and second alignment models based on the evaluation, and correcting distortion of the inspection image based on the selected alignment model.

[0011]

[0011] In some embodiments, methods are provided for correcting distortion in inspection images. These methods include acquiring an inspection image, aligning multiple patches of the inspection image based on a reference image corresponding to the inspection image, evaluating the alignment between each patch of the multiple patches and the corresponding patch of the reference image using a machine learning model, determining the local alignment results of the multiple patches of the inspection image based on the reference image corresponding to the inspection image, determining an alignment model based on the local alignment results, and correcting distortion in the inspection image based on the alignment model.

[0012]

[0012] In some embodiments, a device is provided for correcting distortion in an inspection image. The device includes a memory for storing a set of commands and at least one processor configured to execute a set of commands causing the device to acquire an inspection image, align a plurality of patches of the inspection image based on a reference image corresponding to the inspection image, evaluate the alignment between each patch of the plurality of patches and the corresponding patch of the reference image using a machine learning model, determine the local alignment result of the plurality of patches of the inspection image based on the reference image corresponding to the inspection image, determine an alignment model based on the local alignment result, and correct distortion in the inspection image based on the alignment model.

[0013]

[0013] In some embodiments, a non-temporary computer-readable medium is provided that stores a set of instructions executable on at least the processor of a computing device for causing the computing device to perform a method for correcting distortion in an inspection image. The method includes acquiring an inspection image, determining local alignment results for a plurality of patches of the inspection image based on a reference image corresponding to the inspection image, estimating a first alignment model based on a first subset of the local alignment results, estimating a second alignment model based on a second subset of the local alignment results, evaluating a first alignment model based on a fit of the first alignment model to a first residual set of local alignment results, evaluating a second alignment model based on a fit of the second alignment model to a second residual set of local alignment results, selecting one of the first and second alignment models based on the evaluation, and correcting distortion in the inspection image based on the selected alignment model.

[0014]

[0014] In some embodiments, a method is provided for evaluating the alignment of an inspection image and a reference image. This method includes acquiring a plurality of patches of the inspection image and a plurality of reference patches of the reference image, wherein the plurality of patches correspond to the plurality of reference patches, and evaluating the alignment of the plurality of patches and the plurality of reference patches using a machine learning model.

[0015]

[0015] In some embodiments, a device is provided for evaluating the alignment of an inspection image and a reference image. The device includes a memory for storing a set of instructions and at least one processor configured to execute a set of instructions for causing the device to acquire a plurality of patches of an inspection image and a plurality of reference patches of a reference image, wherein the plurality of patches correspond to the plurality of reference patches, and to evaluate the alignment of the plurality of patches and the plurality of reference patches using a machine learning model.

[0016]

[0016] In some embodiments, a non - temporary computer - readable medium storing a set of instructions executable on at least a processor of a computing device to cause the computing device to evaluate an alignment of a test image and a reference image. The method includes obtaining a plurality of patches of the test image and a plurality of reference patches of the reference image, where the plurality of patches correspond to the plurality of reference patches, and evaluating the alignment of the plurality of patches and the plurality of reference patches by a machine - learning model.

[0017]

[0017] Other advantages of embodiments of the present disclosure will become apparent from the following description taken in conjunction with the accompanying drawings, which illustrate, by way of example, some embodiments of the invention.

[0018]

[0018] The above and other aspects of the present disclosure will become more apparent from the description of the exemplary embodiments interpreted in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0019] [Figure 1]

[0019] A schematic diagram showing an exemplary charged - particle beam inspection system in accordance with an embodiment of the present disclosure. ​​​​​​​​​​​​​​​​​​This is an illustrative graph showing local alignment results consistent with embodiments of the present disclosure. [Figure 5C]

[0025] This is an illustrative graph showing an alignment model consistent with the embodiments of this disclosure. [Figure 6A]

[0026] This section illustrates an exemplary process for correcting distortion in SEM images according to local alignment results. [Figure 6B]

[0027] An exemplary process for correcting distortion in SEM images according to an alignment model consistent with embodiments of this disclosure is shown. [Figure 6C]

[0028] A comparative example of distortion-corrected input and output images consistent with the embodiments of this disclosure is shown. [Figure 7A]

[0029] An exemplary examination image with a shifted repeating pattern is shown. [Figure 7B]

[0030] This is a block diagram of a training system for an alignment evaluation model, consistent with the embodiments of the present disclosure. [Figure 8A]

[0031] An exemplary training dataset for the training system shown in Figure 7B is consistent with the embodiments of this disclosure. [Figure 8B]

[0032] An exemplary configuration of an alignment evaluation model consistent with the embodiments of this disclosure is shown. [Figure 9]

[0033] This is a process flowchart illustrating an exemplary method for correcting distortion in an inspection image, consistent with the embodiments of this disclosure. [Figure 10]

[0034] This is a process flowchart illustrating an exemplary method for training an alignment evaluation model consistent with embodiments of the present disclosure. [Modes for carrying out the invention]

[0020]

[0035] Hereinafter, exemplary embodiments are described in detail. Examples of these embodiments are shown in the accompanying drawings. The following description refers to the accompanying drawings, and the same numbers in different drawings represent the same or similar elements unless otherwise noted. The embodiments described in the following description of exemplary embodiments are not representative of all embodiments. Rather, they are merely examples of apparatus and methods that correspond to aspects related to the disclosed embodiments enumerated in the accompanying claims. For example, some embodiments are described in the context of utilizing electron beams, but this disclosure is not limited in that way. Other types of charged particle beams can be applied similarly. Furthermore, other imaging systems such as optical imaging, photodetection, and X-ray detection can be used.

[0021]

[0036] Electronic devices consist of circuits formed on a piece of semiconductor material called a substrate. Semiconductor materials can include, for example, silicon, gallium arsenide, indium phosphide, or silicon germanium. Numerous circuits can be formed together on the same silicon piece and are called integrated circuits or ICs. The dimensions of these circuits have been dramatically reduced so that many more circuits can fit on a substrate. For example, an IC chip in a smartphone can be as small as a thumbnail, yet it can contain more than 2 billion transistors, and the dimensions of each transistor are smaller than 1 / 1000th the size of a human hair.

[0022]

[0037] Manufacturing ICs with these extremely small structures or components is a complex, time-consuming, and expensive process, often involving hundreds of individual steps. An error in just one step can result in a defect in the finished IC, rendering it unusable. Therefore, one of the goals of the manufacturing process is to avoid such defects and maximize the number of functional ICs produced in the process, i.e., to improve the overall yield of the process.

[0023]

[0038] One component that improves yield is monitoring the chip fabrication process to ensure that a sufficient number of functional integrated circuits are manufactured. One way to monitor the process is to inspect the chip circuit structures at various stages of their formation. Inspection can be done using a scanning charged particle microscope (SCPM). For example, an SCPM can be a scanning electron microscope (SEM). An SCPM can be used to take "photographs" of the wafer structure and image those extremely small structures. This image can be used to determine whether the structure has been properly formed in the correct location. If there are defects in the structure, the process can be adjusted to make the defects less likely to recur.

[0024]

[0039] As the physical size of IC components continues to shrink, the accuracy and effectiveness of defect detection become increasingly important. Inspection images, such as SEM images, can be used to identify or classify defects in manufactured ICs. To improve defect detection performance, it is desirable to obtain accurate SEM images free from distortion or misalignment. Various distortion correction techniques for SEM images have been introduced, but many of them rely on the local alignment of small patches in the SEM image. Although the amount of distortion within each patch is less than the amount of distortion in the overall SEM image, local alignment can be difficult for a variety of reasons, including, but not limited to, sparse or repeating patterns, insufficient imaging conditions, lack of pattern information, and residual distortion. With current methods, distortion correction of SEM images heavily depends on the performance of local alignment of small patches in the SEM image, so erroneous or incomplete local alignment can degrade the distortion correction performance of the SEM image.

[0025]

[0040] Embodiments of the present disclosure can provide distortion correction techniques for SEM images. According to some embodiments of the present disclosure, when correcting distortion in SEM images, potential defective or contaminating data in local alignment results can be recognized and their impact can be minimized. Embodiments of the present disclosure can provide a machine learning-based alignment evaluation algorithm that can reliably evaluate whether an SEM image is well aligned to a reference image. According to some embodiments of the present disclosure, a machine learning-based alignment evaluation algorithm can be trained using pairs of SEM image clips and reference image clips.

[0026]

[0041] The relative dimensions of components in the drawings may be exaggerated for ease of understanding. In the following description of the drawings, the same or similar reference numbers refer to the same or similar components or entities, and only the differences with respect to individual embodiments are described. As used herein, unless otherwise specified, the term “or” encompasses all possible combinations unless impractical. For example, if it is stated that a component may include A or B, then unless otherwise specified or impractical, the component may include A, or B, or A and B. As a second example, if it is stated that a component may include A, B, or C, then unless otherwise specified or impractical, the component may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0027]

[0042] Figure 1 shows an exemplary electron beam inspection (EBI) system 100 consistent with embodiments of the present disclosure. The EBI system 100 can be used for imaging. As shown in Figure 1, the EBI system 100 includes a main chamber 101, a loading / locking chamber 102, a beam tool 104, and an instrument front-end module (EFEM) 106. The beam tool 104 is located inside the main chamber 101. The EFEM 106 includes a first loading port 106a and a second loading port 106b. The EFEM 106 may include additional loading ports. The first loading port 106a and the second loading port 106b receive wafer FOUPs (front opening unified pods) containing wafers to be inspected (e.g., semiconductor wafers, or wafers made of other materials) or samples (wafers and samples can be used interchangeably). A “lot” is a group of wafers that can be loaded for processing as a batch.

[0028]

[0043] One or more robotic arms (not shown) within the EFEM106 can transport wafers to the loading / locking chamber 102. The loading / locking chamber 102 is connected to a loading / locking vacuum pump system (not shown), which removes gas molecules from within the loading / locking chamber 102 to reach a first pressure lower than atmospheric pressure. After reaching the first pressure, one or more robotic arms (not shown) can transport wafers from the loading / locking chamber 102 to the main chamber 101. The main chamber 101 is connected to a main chamber vacuum pump system (not shown), which removes gas molecules from within the main chamber 101 to reach a second pressure lower than the first pressure. After reaching the second pressure, the wafers are subjected to inspection by a beam tool 104. The beam tool 104 may be a single-beam system or a multi-beam system.

[0029]

[0044] The controller 109 is electronically connected to the beam tool 104. The controller 109 may be a computer configured to perform various controls of the EBI system 100. In Figure 1, the controller 109 is shown outside the structure, which includes the main chamber 101, the load / lock chamber 102, and the EFEM 106, but it will be understood that the controller 109 may be part of the structure.

[0030]

[0045] In some embodiments, the controller 109 may include one or more processors (not shown). A processor may be a general-purpose or specific electronic device capable of manipulating or processing information. For example, a processor may include any number of central processing units (i.e., "CPUs"), graphics processing units (i.e., "GPUs"), optical processors, programmable logic control units, microcontrollers, microprocessors, digital signal processors, IP (intellectual property) cores, programmable logic arrays (PLAs), programmable array logic (PALs), general-purpose array logic (GALs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), system-on-a-chip (SoCs), application-specific integrated circuits (ASICs), and any kind of circuitry capable of processing data. A processor may also be a virtual processor, including one or more processors distributed across multiple machines or devices connected via a network.

[0031]

[0046] In some embodiments, the controller 109 may further include one or more memories (not shown). The memories may be general-purpose or specific electronic devices capable of storing code and data accessible by the processor (e.g., via a bus). For example, the memories may include any number of random-access memories (RAM), read-only memories (ROM), optical disks, magnetic disks, hard drives, solid-state drives, flash drives, security digital (SD) cards, memory sticks, compact flash (CF) cards, or any combination of any type of storage device. The code and data may include an operating system (OS) and one or more application programs (i.e., "apps") for a particular task. The memories may also be virtual memories, including one or more memories distributed across multiple machines or devices connected via a network.

[0032]

[0047] Figure 2 shows a schematic diagram of an exemplary multibeam tool 104 (also referred to herein as apparatus 104) and an image processing system 290 that may be configured for use in an EBI system 100 (Figure 1) consistent with embodiments of the present disclosure.

[0033]

[0048] The beam tool 104 includes a charged particle source 202, a gun aperture 204, a focusing lens 206, a primary charged particle beam 210 emitted from the charged particle source 202, a radiation source conversion unit 212, multiple beamlets 214, 216, and 218 of the primary charged particle beam 210, a primary projection optical system 220, an electric wafer stage 280, a wafer holder 282, multiple secondary charged particle beams 236, 238, and 240, a secondary optical system 242, and a charged particle detection device 244. The primary projection optical system 220 may include a beam separator 222, a deflection scanning unit 226, and an objective lens 228. The charged particle detection device 244 may include detection sub-regions 246, 248, and 250.

[0034]

[0049] The charged particle source 202, gun aperture 204, focusing lens 206, radiation source conversion unit 212, beam separator 222, deflection scanning unit 226, and objective lens 228 can be aligned with the primary optical axis 260 of the device 104. The secondary optical system 242 and charged particle detection device 244 can be aligned with the secondary optical axis 252 of the device 104.

[0035]

[0050] The charged particle source 202 can emit one or more charged particles, such as electrons, protons, ions, muons, or any other charge-carrying particles. In some embodiments, the charged particle source 202 may be an electron emission source. For example, the charged particle source 202 may include a cathode, an extractor, or an anode, and primary electrons may be emitted from the cathode and extracted or accelerated to form a primary charged particle beam 210 (in this case, a primary electron beam) together with a (virtual or real) crossover 208. For the sake of ease of explanation without causing ambiguity, electrons are used as an example in some parts of this specification. However, it should be noted that any charged particle, not limited to electrons, may be used in any embodiment of this disclosure. The primary charged particle beam 210 can be visualized when it is being emitted from the crossover 208. The gun aperture 204 can block surrounding charged particles of the primary charged particle beam 210 to reduce the Coulomb effect. The Coulomb effect can cause an increase in the size of the probe spot.

[0036]

[0051] The radiation source conversion unit 212 may include an array of image forming elements and an array of beam limiting apertures. The array of image forming elements may include an array of micro-deflectors or microlenses. The array of image forming elements can form multiple (virtual or real) parallel images of the crossover 208 together with multiple beamlets 214, 216, and 218 of the primary charged particle beam 210. The array of beam limiting apertures can limit the multiple beamlets 214, 216, and 218. Three beamlets 214, 216, and 218 are shown in Figure 2, but embodiments of the present disclosure are not limited thereto. For example, in some embodiments, the apparatus 104 may be configured to generate a first number of beamlets. In some embodiments, the first number of beamlets may be in the range of 1 to 1000. In some embodiments, the first number of beamlets may be in the range of 200 to 500. In an exemplary embodiment, the apparatus 104 may generate 400 beamlets.

[0037]

[0052] The focusing lens 206 can focus the primary charged particle beam 210. The currents in the beamlets 214, 216, and 218 downstream of the radiation source conversion unit 212 can be changed by adjusting the focusing force of the focusing lens 206 or by changing the radius size of the corresponding beam limiting aperture in the array of beam limiting apertures. The objective lens 228 can focus the beamlets 214, 216, and 218 onto the imaging wafer 230, and can form a plurality of probe spots 270, 272, and 274 on the surface of the wafer 230.

[0038]

[0053] The beam separator 222 may be a Wien filter type beam separator that generates electrostatic and magnetic dipole fields. In some embodiments, when these are applied, the force exerted by the electrostatic dipole field on the charged particles (e.g., electrons) of the beamlets 214, 216, and 218 may be substantially equal in magnitude and opposite in direction to the force exerted by the magnetic dipole field on the charged particles. Thus, the beamlets 214, 216, and 218 can pass straight through the beam separator 222 with zero deflection angle. However, the total variance of the beamlets 214, 216, and 218 generated by the beam separator 222 may also be non-zero. The beam separator 222 can separate the secondary charged particle beams 236, 238, and 240 from the beamlets 214, 216, and 218 and guide the secondary charged particle beams 236, 238, and 240 into the secondary optical system 242.

[0039]

[0054] The deflection scanning unit 226 can deflect the beamlets 214, 216, and 218 to scan probe spots 270, 272, and 274 on the surface area of ​​wafer 230. In response to the incidence of beamlets 214, 216, and 218 at probe spots 270, 272, and 274, secondary charged particle beams 236, 238, and 240 may be emitted from wafer 230. The secondary charged particle beams 236, 238, and 240 may contain charged particles (e.g., electrons) having an energy distribution. For example, the secondary charged particle beams 236, 238, and 240 may be secondary electron beams containing secondary electrons (energy ≤ 50 eV) and backscattered electrons (energy between 50 eV and the landing energies of beamlets 214, 216, and 218). The secondary optical system 242 can focus the secondary charged particle beams 236, 238, and 240 onto the detection subregions 246, 248, and 250 of the charged particle detection device 244. The detection subregions 246, 248, and 250 may be configured to detect the corresponding secondary charged particle beams 236, 238, and 240 and generate corresponding signals (e.g., voltage, current, etc.) used to reconstruct an SCPM image of the structure on or beneath the surface area of ​​the wafer 230.

[0040]

[0055] The generated signals can represent the intensities of the secondary charged particle beams 236, 238, and 240 and can be provided to an image processing system 290 communicating with a charged particle detection device 244, a primary projection optical system 220, and an electric wafer stage 280. The movement speed of the electric wafer stage 280 can be synchronized and adjusted with the beam deflection controlled by a deflection scanning unit 226 so that the movement of the scanning probe spots (e.g., scanning probe spots 270, 272, and 274) neatly covers the area of ​​interest on the wafer 230. Such synchronization and adjustment parameters can be adjusted to suit various materials of the wafer 230. For example, different materials of the wafer 230 may have different resistive-capacitive characteristics that can result in different signal sensitivities to the movement of the scanning probe spots.

[0041]

[0056] The intensities of the secondary charged particle beams 236, 238, and 240 may vary depending on the external or internal structure of the wafer 230, and thus can indicate whether the wafer 230 contains defects. Furthermore, as described above, the beamlets 214, 216, and 218 can be projected onto different locations on the upper surface of the wafer 230 or onto different sides of the local structure of the wafer 230 to generate secondary charged particle beams 236, 238, and 240 which may have different intensities. Thus, by mapping the intensities of the secondary charged particle beams 236, 238, and 240 to areas of the wafer 230, the image processing system 290 can reconstruct an image that reflects the characteristics of the internal or external structure of the wafer 230.

[0042]

[0057] In some embodiments, the image processing system 290 may include an image acquirer 292, storage 294, and a controller 296. The image acquirer 292 may include one or more processors. For example, the image acquirer 292 may include a computer, server, mainframe host, terminal, personal computer, any kind of mobile computing device, or a combination thereof. The image acquirer 292 may be communicatively coupled to the charged particle detection device 244 of the beam tool 104 via a medium such as a conductor, fiber optic cable, portable storage medium, IR, Bluetooth, the Internet, wireless network, wireless radio, or a combination thereof. In some embodiments, the image acquirer 292 may receive signals from the charged particle detection device 244 and construct an image. In this way, the image acquirer 292 can acquire an SCPM image of the wafer 230. The image acquirer 292 may also perform various post-processing functions, such as generating contours and superimposing indicators on the acquired image. The image acquirer 292 may be configured to adjust the brightness and contrast of the acquired image. In some embodiments, the storage 294 may be a storage medium such as a hard disk, flash drive, cloud storage, random access memory (RAM), or other types of computer-readable memory. The storage 294 may be coupled to an image acquirer 292 and used to store scanned raw image data as original and post-processed images. The image acquirer 292 and the storage 294 may be connected to a controller 296. In some embodiments, the image acquirer 292, the storage 294, and the controller 296 may be integrated as a single control unit.

[0043]

[0058] In some embodiments, the image acquisition unit 292 may acquire one or more SCPM images of the wafer based on imaging signals received from the charged particle detection device 244. The imaging signals may correspond to scanning operations for imaging charged particles. The acquired image may be a single image containing multiple imaging areas. The single image may be stored in storage 294. The single image may be a source image that can be divided into multiple regions. Each of these regions may contain one imaging area containing features of the wafer 230. The acquired image may contain multiple images of a single imaging area of ​​the wafer 230 that are sampled multiple times over a time series. The multiple images may be stored in storage 294. In some embodiments, the image processing system 290 may be configured to perform image processing steps using multiple images of the same location on the wafer 230.

[0044]

[0059] In some embodiments, the image processing system 290 may include a measurement circuit (e.g., an analog-to-digital converter) for obtaining the distribution of detected secondary charged particles (e.g., secondary electrons). The distribution data of charged particles collected during the detection time window can be used in combination with the corresponding scan path data of beamlets 214, 216, and 218 incident on the wafer surface to reconstruct an image of the wafer structure under inspection. The reconstructed image can be used to reveal various features of the internal or external structure of the wafer 230, thereby revealing any defects that may be present in the wafer.

[0045]

[0060] In some embodiments, the charged particles may be electrons. When electrons from the primary charged particle beam 210 are projected onto the surface of the wafer 230 (e.g., probe spots 270, 272, and 274), the electrons from the primary charged particle beam 210 may interact with the particles of the wafer 230 and penetrate the surface of the wafer 230 to a certain depth. Some electrons from the primary charged particle beam 210 may interact elastically with the material of the wafer 230 (e.g., in the form of elastic scattering or collision) and be reflected or bounced back from the surface of the wafer 230. Elastic interactions conserve the total kinetic energy of the interacting objects (e.g., electrons from the primary charged particle beam 210), and the kinetic energy of the interacting objects is not converted into other forms of energy (e.g., heat, electromagnetic energy, etc.). Reflected electrons resulting from such elastic interactions are sometimes called backscattered electrons (BSEs). Some electrons from the primary charged particle beam 210 may interact inelastically with the material of the wafer 230 (e.g., in the form of inelastic scattering or collision). Inelastic interactions do not conserve the total kinetic energy of the interacting objects; rather, some or all of the kinetic energy of the interacting objects is converted into another form of energy. For example, inelastic interactions can cause the kinetic energy of some electrons in the primary charged particle beam 210 to cause electronic excitation and transitions of atoms in the material. Such inelastic interactions can also generate electrons that emanate from the surface of the wafer 230, sometimes called secondary electrons (SEs). The yield or emission rate of BSEs and SEs depends on several factors, including, for example, the irradiation energy of the electrons in the primary charged particle beam 210 irradiated onto the material under inspection and the surface of the material. The energy of the electrons in the primary charged particle beam 210 may be partially imparted by its accelerating voltage (for example, the accelerating voltage between the anode and cathode of the charged particle source 202 in Figure 2). The amount of BSEs and SEs may be greater than or less than (or even equal to) the number of electrons injected by the primary charged particle beam 210.

[0046]

[0061] Images generated by a scanning electron microscope (SEM) can be used for defect inspection. For example, a generated image capturing a test device area on a wafer can be compared to a reference image capturing the same test device area. The reference image may be predetermined (e.g., by simulation) and may not contain known defects. If the difference between the generated image and the reference image exceeds an acceptable level, a potential defect may be identified. As another example, the SEM can scan multiple areas of a wafer, each containing a similarly designed test device area, and generate multiple images capturing those test device areas during manufacturing. These multiple images can be compared to each other. If the difference between the multiple images exceeds an acceptable level, a potential defect may be identified.

[0047]

[0062] Figure 3 shows an SEM image 300 with exemplary patterns that can cause local misalignment. As shown in Figure 3, the SEM image 300 shows three exemplary sections 301 to 303 that have patterns that can cause local misalignment. Figure 3 shows enlarged views of the first section 301 to the third section 303 at the bottom. The first section 301 shows a sparse pattern, the second section 302 shows a debris pattern, and the third section 303 shows a cropping debris pattern. While three patterns are shown as features that cause local misalignment, it will be understood that various features can cause local misalignment in SEM images.

[0048]

[0063] Next, we refer to Figure 4, a block diagram of an exemplary strain correction system consistent with embodiments of the present disclosure. In some embodiments, the strain correction system 400 includes one or more processors and memory. In various embodiments, it will be understood that the strain correction system 400 may be part of a charged particle beam inspection system (e.g., the EBI system 100 in Figure 1), or a computational lithography system, or other photolithography system, or may be separate from them. In some embodiments, the strain correction system 400 may include one or more components (e.g., software modules) that can be implemented by a controller 109 or system 290 as discussed herein. As shown in Figure 4, the strain correction system 400 may include an inspection image acquirer 410, a reference image acquirer 420, an image aligner 430, an alignment model generator 440, and a strain corrector 450.

[0049]

[0064] According to some embodiments of this disclosure, the inspection image acquirer 410 can acquire an inspection image as an input image. In some embodiments, the inspection image is an SEM image of a sample or wafer. In some embodiments, the inspection image may be an inspection image generated by, for example, the EBI system 100 in Figure 1 or the electron beam tool 104 in Figure 2. In some embodiments, the inspection image acquirer 410 can obtain an inspection image from a storage device or system that stores inspection images. Figure 5A shows an exemplary inspection image 510, which will be described in detail with respect to some embodiments of this disclosure.

[0050]

[0065] Referring again to Figure 4, according to some embodiments, the reference image acquirer 420 can acquire a reference image corresponding to the inspection image acquired by the inspection image acquirer 410. In some embodiments, the reference image may be a layout file for a wafer design corresponding to the inspection image. This layout file may be in the Graphics Database System (GDS) format, Graphics Database System II (GDSII) format, Open Artwork System Interchange Standard (OASIS) format, Caltech Intermediate Format (CIF), etc. The wafer design may include patterns or structures to be included on the wafer. The patterns or structures may be mansk patterns used to transfer features from a photolithography mask or reticle to the wafer. In some embodiments, the layout in GDS or OASIS format, among other things, may include feature information stored in a binary file format representing planar geometric shapes, text, and other information related to the wafer design. In some embodiments, the reference image may be an image rendered from the layout file.

[0051]

[0066] According to some embodiments of this disclosure, the image aligner 430 can segment an inspection image into a number of smaller patches. Figure 5A shows that an inspection image 510 is segmented into several patches 511_1 through 511_n. While Figure 5A shows that the inspection image 510 is segmented in two dimensions, it will be understood that the inspection image 510 can be segmented in any dimension. Figure 5A shows that the first row of the inspection image 510 is segmented into n patches 511_1 through 511_n. While some embodiments describe patches arranged in one dimension (e.g., horizontally in Figure 5A), it will be understood that this disclosure can be applied to patches of various dimensions.

[0052]

[0067] After segmenting the inspection image 510 into a plurality of patches 511_1 to 511_n, the image aligner 430 is configured to align the plurality of patches 511_1 to 511_n with a reference image corresponding to the inspection image 510. In some embodiments, aligning the plurality of patches 511_1 to 511_n with the reference image may be performed based on feature matching between the patches 511_1 to 511_n and the reference image. In some embodiments, the image aligner 430 can determine a corresponding portion or patch in the reference image for each of the plurality of patches 511_1 to 511_n. In some embodiments of the present disclosure, while aligning the plurality of patches 511_1 to 511_n, a corresponding patch in the reference image can be determined for each patch of the inspection image.

[0053]

[0068] According to some embodiments of the present disclosure, the distortion correction system 400 may further include an alignment evaluator 460. Consistent with some embodiments of the present disclosure, the alignment evaluator 460 may be configured to evaluate whether multiple patches of an inspection image are well aligned with corresponding patches of a reference image. In some embodiments, the alignment evaluator 460 may provide an alignment index for each patch of an inspection image by evaluating whether the patches of the inspection image are well aligned with corresponding patches of a reference image. In some embodiments, the alignment index may represent the confidence that the patches of the inspection image are aligned with corresponding patches of a reference image. In some embodiments, the alignment evaluation may be performed based on the patches (e.g., patches 511_1 to 511_n) used when performing alignment by the image aligner 430. In some embodiments, the alignment evaluation may be performed based on patches different from patches 511_1 to 511_n used by the image aligner 430. For example, patches 511_1 to 511_n can be grouped based on alignment, and the alignment evaluator 460 can segment the grouped patches again into a set of different patches used for alignment evaluation. The re-segmented patches may have different sizes or shapes than those of patches 511_1 to 511_n. In some embodiments, the alignment evaluator 460 can evaluate whether the inspection image is well aligned to a reference image based on the alignment evaluation results for multiple patches of the inspection image. In some embodiments, the image aligner 430 can receive the alignment evaluation results from the alignment evaluator 460 and re-align multiple patches of the inspection image to the reference image according to the alignment evaluation results. In some embodiments, the alignment evaluator 460 may be a machine learning-based alignment algorithm. Referring to Figures 7A to 8B, training techniques for machine learning-based alignment evaluation algorithms are described.

[0054]

[0069] In some embodiments, local alignment results for a plurality of patches 511_1 to 511_n can be generated based on the alignment of a plurality of patches 511_1 to 511_n with respect to a reference image. Figure 5B is an exemplary graph showing local alignment results for a plurality of patches consistent with embodiments of the present disclosure. In Figure 5B, ten local alignment results LA are shown in a two-dimensional coordinate system, where the x-axis represents the position in the reference image, e.g., a GDS file, and the y-axis represents the position in the inspection image 510, e.g., an SEM image. In Figure 5A, the local alignment result LA may be associated with one of the plurality of patches 511_1 to 511_n. In some embodiments, the local alignment result LA for each patch may be the measured distance from a reference point in the inspection image, e.g., the reference point RP in the inspection image 510 in Figure 5A, to the center of each patch.

[0055]

[0070] In Figure 5B, the first local alignment result LA1 associated with the first patch 511_1 shows that the position of the first patch 511_1 in the examination image 510 is approximately 0.4 from the reference point RP, while the position of the corresponding patch in the reference image is 1 from the corresponding reference point in the reference image. The second local alignment result LA2 associated with the second patch 511_2 shows that the position of the second patch 511_2 in the examination image 510 is approximately 1.45 from the reference point RP, while the position of the corresponding patch in the reference image is 1.5 from the corresponding reference point in the reference image. Similarly, the tenth local alignment result LA10 associated with the tenth patch shows that the position of the tenth patch in the examination image 510 is approximately 5.6, while the position of the corresponding patch in the reference image is 5 from the corresponding reference point in the reference image. Please note that in this disclosure, the positive sign (+) indicating rightward movement is omitted in Figure 5A, and the negative sign (-) is used to indicate the opposite direction.

[0056]

[0071] Referring again to Figure 4, according to some embodiments of the present disclosure, the alignment model generator 440 is configured to generate an alignment model that can be used to correct distortion in an inspection image, for example, inspection image 510. According to some embodiments of the present disclosure, the alignment model generator 440 can generate an alignment model based on local alignment results. According to some embodiments of the present disclosure, the alignment model generator 440 can generate an alignment model that fits as many local alignment results as possible, for example, as shown in Figure 5C. As shown in Figure 5C, the alignment model 531 fits eight local alignment results (i.e., LA2 to LA9) out of ten alignment results (i.e., LA1 to LA10). In this disclosure, local alignment results that fit the alignment model may be called inliers, and local alignment results that do not fit the alignment model may be called outliers. In Figure 5C, eight alignment results, namely LA2 through LA9, are internal values ​​for the alignment model 531, while two alignment results, namely LA1 and LA10, are outliers. In some embodiments, the outliers can be assumed to be defective or contaminated data resulting from, for example, snapping or half-pitch shift. In some embodiments, these outliers are excluded when estimating the alignment model.

[0057]

[0072] According to some embodiments, an alignment model that minimizes the L0 norm, which counts the total number of non-zero distances between local alignment results and the alignment model, can be determined as the alignment model for the corresponding inspection image. In Figure 5C, local alignment results LA2 to LA9 that fit the alignment model 531 have a zero distance from the alignment model 531, while the two local alignment results LA1 and LA0 have a non-zero distance from the alignment model 531. According to some embodiments, potential defect data can be recognized and ignored when generating the alignment model, thereby minimizing the impact of defect data on distortion correction of the inspection image 510.

[0058]

[0073] According to certain embodiments of this disclosure, estimating an alignment model can be performed based on a random sample consensus (RANSAC) expressed as follows: For epoch in N: LA S =Sample(O,epoch,LA) F=Fit(LA S ) F * =Evaluate(F,P t ,LA) END FOR. Algorithm 1 Here, N represents the total number of iterations, epoch represents the current iteration number, O represents the regression order, and LA represents the local alignment results, e.g., LA1 to LA10 in Figure 5C.

[0059]

[0074] In the first step, a sample subset of local alignment results is randomly selected from a set of local alignment results according to the regression order O. The regression order O can represent any order of regression according to the embodiment. To uniquely define an alignment model, e.g., an alignment curve, several local alignment results for the sample subset can be determined based on the regression order O. For example, if the regression order O identifies the first order of regression, also called linear regression, a sample subset of two local alignment results can be randomly selected. This disclosure describes an embodiment in which two local alignment results are selected for the sample subset for illustrative purposes. The subset containing the selected alignment results is LA S This is shown as follows.

[0060]

[0075] In accordance with certain embodiments of this disclosure, in the second step, subset LA S A first alignment model F that fits the selected alignment results included can be determined. In some embodiments, the first alignment model F may be a linear or nonlinear model with regression order O. In some embodiments, subset LAS Parameters of a first alignment model F that fits the selected alignment results included in S and satisfies a defined regression order O can be calculated. For example, if the second local alignment result LA2 and the ninth local alignment result LA9 are selected in the first step and linear regression is defined by the regression order O, the alignment model 531, which is a linear equation, can be uniquely identified.

[0061]

[0076] In the third step, the first alignment model F obtained in the second step can be evaluated by examining how many local alignment results within the entire dataset fit the first alignment model F. According to some embodiments, the performance P of the first alignment model F is determined based on the number of local alignment results that fit the first alignment model F. t In some embodiments, local alignment results that fit the first alignment model F are regarded as inliers, and local alignment results that do not fit the first alignment model F are regarded as outliers. In some embodiments, if a local alignment result does not fit the first alignment model F but is close enough, that local alignment result can be regarded as an inlier. For example, a local alignment result whose distance from the first alignment model F is within a threshold can be regarded as fitting the first alignment model F. In some embodiments, the threshold can be determined based on whether the deviation from the first alignment model F is due to the influence of noise. In some embodiments, the performance P of the first alignment model F t can be determined based on the percentage of inliers in the entire dataset. For example, in FIG. 5C, 8 out of 10 local alignment results, namely LA2 to LA9, fit the fitting curve 531 as the first alignment model F. Therefore, the performance P of the first alignment model F t can be determined to be 80%. In some embodiments, the performance P of the first alignment model F tThis can be determined based on the percentage of internal values ​​in the residual set. For example, if the sample subset includes the second local alignment result LA2 and the ninth local alignment result LA9, then six of the eight local alignment results in the residual set are internal values, and therefore the performance P of the first alignment model F is... t This can be determined as 75%. In this step, the first alignment model F is the potential alignment model F * It is considered to be.

[0062]

[0077] As discussed above, steps 1 through 3 can be repeated for N iterations. After the completion of the first iteration, the second iteration is performed. In the first step, the composition of the subset for the second iteration may be randomly selected from the entire dataset. The local alignment results selected in the second iteration may differ from those selected in the first iteration. In the second step, a second alignment model F is estimated based on the selected local alignment results, and in the third step, the performance P of the second alignment model F is calculated. t Evaluate it in the same way as in the first iteration. In the third step, evaluate the performance P of the first alignment model F in the first iteration. t The performance of the second alignment model F is better than P t If one is superior, then the second alignment model F is used as the potential alignment model F. * Update as follows: Otherwise, the first alignment model F will be changed to the potential alignment model F. * It is maintained as follows. Therefore, when N iterations are completed, the highest-performing P is among the N alignment models F. t Alignment model F having the following characteristics is examined in image 510. * It can be selected as such.

[0063]

[0078] Referring again to Figure 4, which is consistent with some embodiments of this disclosure, the distortion corrector 450 is selected for alignment model F *It can be configured to correct distortion of the inspection image based on the selected alignment model F. * Based on this, all patches in the inspection image corresponding to all local alignment results, including internals and outliers, can be corrected. For example, in Figure 5C, the second patch corresponding to the second local alignment result LA2, which is an internal value with respect to alignment model 531, is corrected according to alignment model 531. To correct the distortion of the inspection image 510, the second patch 511_1 corresponding to the second local alignment result LA2 can be shifted by 0.05 according to alignment model 531 in Figure 5C. Similarly, in Figure 5C, the first patch corresponding to the first local alignment result LA1, which is an outlier with respect to alignment model 531, is also corrected according to alignment model 531 instead of the measured local alignment result LA1. To correct the inspection image 510, the first patch 511_1 corresponding to the first local alignment result LA1 can be shifted by (-)0.05 according to fitting model 531 instead of the 0.6 from the measured local alignment result LA1 in Figure 5C. While some embodiments of this disclosure illustrate linear alignment models, it will be understood that this disclosure is not limited to these but can be applied to any type of alignment model, including scaling, rotation, translation, and so on.

[0064]

[0079] Figure 6A illustrates an exemplary process for correcting distortion in an SEM image according to local alignment data. In Figure 6A, the SEM image 610 is programmed to contain 1% distortion out of 1000 pixels. In Figure 6A, the first arrow map 611 shows the local alignment results measured for each pixel in the SEM image 610. In the first arrow map 611, each arrow indicates in which direction and by how much the corresponding pixel should be moved to match the corresponding reference image, according to the corresponding measured local alignment result. In Figure 6A, the first corrected image 612 is the corrected image of the SEM image 610 according to the first arrow map 611. As shown in the corrected SEM image 612, correcting the inspection image according to the measured local alignment results may result in other types of misalignment or distortion, such as snapping.

[0065]

[0080] Figure 6B illustrates an exemplary process for correcting distortion in an SEM image according to an alignment model consistent with embodiments of the present disclosure. Here, the SEM image 610 from Figure 6A is also used as the input test image. In Figure 6B, the second arrow map 621 represents arrows 632 corresponding to internal values ​​and arrows 631 corresponding to outliers among the arrows in the first arrow map 611 for a selected alignment model, consistent with some embodiments of the present disclosure. The third arrow map 622 shows the fitting result of the SEM image 610 according to the selected alignment model. As shown in the third arrow map 622 in Figure 6B, arrows 631 corresponding to outliers are not considered, but the fitting result by the selected alignment model is considered when correcting distortion in the SEM image 610. In the third arrow map 622, each arrow represents, according to the selected alignment model, how much and in which direction the corresponding pixel should be moved to match the corresponding reference image. In Figure 6B, the second corrected image 630 is a corrected image of the SEM image 610 using the third arrow map 622. As shown as the second corrected SEM image 630, distortion of the inspection image can be effectively corrected according to an alignment model consistent with certain embodiments of the present disclosure. For example, in some cases, up to 99.98% of the distortion programmed in the SEM image 610 is corrected in the SEM image 630.

[0066]

[0081] Figure 6C shows a comparative example of a distortion-corrected input inspection image and an output correction image consistent with embodiments of the present disclosure. In Figure 6C, SEM image 610 and correction SEM image 630 are, respectively, SEM images before and after distortion correction, consistent with certain embodiments of the present disclosure. SEM image 610 includes portions 613, 614, and 615 that are not well aligned with the corresponding reference image. In the magnified images of portions 613, 614, and 615, pattern W of SEM image 610 is shown as a white hollow circle. Pattern B of the corresponding GDS image is also shown for illustrative purposes in the magnified images of portions 613, 614, and 615. Note that SEM image 610 is not well aligned with the corresponding GDS image, as shown in the magnified images of portions 613, 614, and 615. The corrected SEM image 630 also includes portions 631, 632, and 633 corresponding to portions 613, 614, and 615 of the input SEM image 610. Note that in the magnified images of portions 631, 632, and 633, the pattern W of the SEM image 630 is well aligned with the pattern B of the corresponding GDS image. As shown in Figure 6C, distortion of the input SEM image can be effectively corrected according to some embodiments of this disclosure.

[0067]

[0082] To improve the distortion correction performance of SEM images, it is crucial to first accurately align the SEM images to the design layout. As shown with respect to the alignment evaluator 460 in Figure 4, whether the SEM images are well aligned to the reference image can be performed before generating an alignment model based on local alignment results. There are many mechanisms that provide alignment confidence scores. However, alignment confidence scores are generally calculated by matching algorithms based on cross-correlation or mean squared error (MSE), and such algorithms do not yield robust matching scores when the field of view contains a relatively large portion of repeating patterns and lacks intrinsic features or sufficient underlying information derived from defects. Under these algorithms, the alignment confidence score may still be high even when the position of the SEM image with repeating patterns is misaligned.

[0068]

[0083] Figure 7A shows an exemplary inspection image with a shifted repeating pattern. Figure 7A shows that the SEM image is not aligned with the reference image, i.e., the SEM image is shifted by a distance T from the reference image. For example, pattern 712 in SEM image 711 is shifted from the corresponding pattern 713 in its reference image. However, the alignment confidence score by a matching algorithm based on cross-correlation or mean squared error (MSE) may still be high for SEM image 711 because the pattern in SEM image 711 is repeated as shown in Figure 7A. It should be noted that confidence scores based on cross-correlation or MSE may result in false positive results for alignment of SEM images containing repeating patterns. According to some embodiments of this disclosure, the alignment evaluator 460 may be a machine learning-based alignment algorithm, and the alignment evaluation model may be trained by a training system.

[0069]

[0084] Figure 7B is a block diagram of a training system 700 (also referred to as "apparatus 700") for an alignment evaluation model, consistent with embodiments of the present disclosure. In some embodiments, the training system 700 may include one or more processors and memory. In various embodiments, it will be understood that the training system 700 may be part of a charged particle beam inspection system (e.g., the EBI system 100 in Figure 1) or may be separate from the charged particle beam inspection system. It will also be understood that the training system 700 may include one or more components or modules that are separate from the charged particle beam inspection system and communicatively coupled to the charged particle beam inspection system. In some embodiments, the training system 700 may include one or more components (e.g., software modules) that can be implemented by the controller 109 or system 290 discussed herein. In some embodiments, the training system 700 and the strain correction system 400 are implemented on separate computing devices or on the same computing device. In some embodiments, the training system 700 may be part of the strain correction system 400 or the alignment evaluator 460 of the strain correction system 400 shown in Figure 4.

[0070]

[0085] As shown in Figure 7B, the training system 700 may include a training inspection image acquirer 710, a training reference image acquirer 720, and a model trainer 730. According to some embodiments of this disclosure, the training inspection image acquirer 710 can acquire inspection image patches. In some embodiments, the inspection image patch is a patch of SEM images of a sample or wafer. In some embodiments, the training inspection image may be one of several patches of SEM images. For example, the training inspection image may be one of patches 511_1 to 511_n shown in Figure 5A.

[0071]

[0086] According to some embodiments, the training reference image acquirer 720 can acquire a reference image to be compared with the training inspection image acquired by the training inspection image acquirer 710. In some embodiments, the reference image may be a layout file for wafer design. In some embodiments, the training reference image may or may not be well aligned with the training inspection image for training purposes.

[0072]

[0087] Figure 8A shows an exemplary training dataset for the training system of Figure 7B, consistent with embodiments of the present disclosure. Figure 8A shows exemplary pairs of training examination images and training reference images acquired by the training examination image acquirer 710 and the training reference image acquirer 720, respectively. In Figure 8A, the training examination image acquirer 710 can acquire training examination image patches 811_1 to 811_n, and the training reference image acquirer 720 can acquire training reference image patches 821_1 to 821_n. In some embodiments, each training examination image 811 is paired with a training reference image 821. For example, the first training examination image 811_1 is paired with the first training reference image 821_1 to constitute the first pair PA1. Similarly, the nth pair PAn is acquired from the second pair PA2.

[0073]

[0088] Referring again to Figure 7B, according to some embodiments of the present disclosure, the model trainer 730 is configured to train an alignment assessment model 731 to predict the alignment index of two images of PAn from each pair of PA1 provided as input. According to some embodiments of the present disclosure, the model trainer 730 is configured to train the alignment assessment model 731 under supervised learning. In some embodiments, the model trainer 730 is also provided with information on whether the training test image 811 and the training reference image 812 are aligned.

[0074]

[0089] Figure 8B shows an exemplary configuration of the alignment assessment model 731 consistent with embodiments of the present disclosure. As shown in Figure 8B, the alignment assessment model 731 receives a training test image 811 and a training reference image 821, processes the two images, and can predict how well aligned the two images are. In some embodiments, the alignment assessment model 731 may be configured to provide an alignment index representing the degree of alignment between the two images. In some embodiments, the alignment assessment model 731 may be a machine learning system or neural network, such as a Sham neural network. It will be understood that other types of machine learning systems may also be available.

[0075]

[0090] As shown in Figure 8B, consistent with some embodiments of the present disclosure, the alignment evaluation model 731 may be configured to include a first network 732 and a second network 733. In some embodiments, the training examination image 811 is provided to the first network 732, which is configured to extract features of the training examination image 811, and the training reference image 821 is provided to the second network 733, which is configured to extract features of the reference image 821. In some embodiments, the first network 732 and the second network 733 may be configured to have the same configuration. For example, the first network 732 and the second network 733 may be configured to have shared weights so that the two networks 732 and 733 can work together for two different inputs to compute equivalent outputs, for example, as output vectors. In some embodiments, the first network 732 and the second network 733 may be configured to have independent configurations. In this example, the first network 732 and the second network 733 do not have to share weights. In some embodiments, the first network 732 and the second network 733 may be implemented by various network architectures, including, but not limited to, Visual Geometry Group (VGG) neural networks, residual neural networks (ResNet), and Dense Convolutional Networks (DenseNet).

[0076]

[0091] As shown in Figure 8B, consistent with some embodiments of the present disclosure, the alignment evaluation model 731 may further include a processing layer 734. In some embodiments, features from a first network 732 and features from a second network 733 may be provided to the processing layer 734. As shown in Figure 8B, to output an alignment index between two images 811 and 821, the processing layer 734 may include multiple processing layers that process input features from the two networks 732 and 733. In some embodiments, the processing layer 734 may compute a convolution operation on the input features. In some embodiments, features from the first network 732 and features from the second network 733 may be combined in the input layer of the processing layer 734 and then further processed to output an alignment index.

[0077]

[0092] If the alignment index does not match the information regarding whether the training examination image 811 and the training reference image 812 are aligned, the model trainer 730 can adjust the parameters, weights, etc., of the alignment evaluation model 731. Since the alignment evaluation model 731 is further trained using PAn from PA1 of the training examination image 811 and the training reference image 812, the accuracy of the alignment index generated by the alignment evaluation model 731 can be improved.

[0078]

[0093] In accordance with some embodiments of this disclosure, the alignment evaluation model 731 can be used as an alignment evaluator 460 after the model trainer 730 has trained the alignment evaluation model 731 under supervised learning. In some embodiments, the trained alignment evaluation model 731 may be used to predict the alignment index between an input image and a reference image. In some embodiments, the trained alignment evaluation model 731 can be used independently of or in combination with distortion correction of the inspection image. In some embodiments, the trained alignment evaluation model 731 can be used in a print check process. In a print check process, a mask or reticle can be inspected by exposing the wafer together with the mask and inspecting the wafer to check whether any defects that may indicate a defect in the mask are repeated. In this application example, the trained alignment evaluation model 731 can be used to evaluate the alignment between an inspection image of the wafer and a corresponding layout image of the wafer.

[0079]

[0094] Figure 9 is a process flowchart representing an exemplary method for correcting distortion in an inspection image, consistent with embodiments of the present disclosure. The steps of Method 900 may be performed on a computing device, for example, the controller 109 in Figure 1, or by a system that uses its functions (for example, System 400 in Figure 4). It will be understood that the illustrated Method 900 can be modified to change the order of the steps and to include additional steps.

[0080]

[0095] In step S910, an inspection image and a reference image are acquired. Step S910 can be performed, for example, by an inspection image acquirer 410 or a reference image acquirer 420. In some embodiments, the inspection image is an SEM image of a sample or wafer. In some embodiments, the reference image may be a layout file for a wafer design corresponding to the inspection image. In some embodiments, the reference image may be an image rendered from the layout file.

[0081]

[0096] In step S920, the inspection image is aligned with a reference image. Step S920 can be performed, for example, by an image aligner 430, among other things. According to some embodiments of the present disclosure, the inspection image can be segmented into a plurality of smaller patches. Figure 5A shows that the inspection image 510 is segmented into a plurality of patches 511_1 to 511_n. After segmenting the inspection image 510 into a plurality of patches 511_1 to 511_n, the plurality of patches 511_1 to 511_n are aligned with the reference image corresponding to the inspection image 510. In some embodiments, aligning the plurality of patches 511_1 to 511_n with the reference image can be performed based on feature matching between the patches 511_1 to 511_n and the reference image. In some embodiments of the present disclosure, while aligning the plurality of patches 511_1 to 511_n, the corresponding patch of the reference image can be determined for each patch of the inspection image.

[0082]

[0097] In some embodiments of the present disclosure, method 900 may further perform step S921. In step S921, the alignment of the inspection image performed in step S920 can be evaluated. Step S921 may be performed, for example, by an alignment evaluator 460. In step S921, it can be evaluated whether multiple patches of the inspection image are well aligned with corresponding patches of the reference image. By evaluating whether the patches of the inspection image are well aligned with corresponding patches of the reference image, an alignment index can be generated for each patch of the inspection image. In some embodiments, the alignment index may represent the confidence that the patches of the inspection image are aligned with corresponding patches of the reference image. In some embodiments, it can be evaluated whether the inspection image is well aligned with the reference image based on the alignment evaluation results for multiple patches of the inspection image. In some embodiments, step S920 can be repeated to realign multiple patches of the inspection image with the reference image according to the alignment evaluation results. In some embodiments, step S921 may be a machine learning-based alignment algorithm.

[0083]

[0098] In step S930, an alignment model is generated. Step S930 can be performed, for example, by an alignment model generator 440. In some embodiments, local alignment results for a plurality of patches 511_1 to 511_n can be generated based on the alignment of a plurality of patches 511_1 to 511_n with respect to a reference image. According to some embodiments of the present disclosure, an alignment model that can be used to correct distortion of the inspection image can be generated based on the local alignment results. According to some embodiments of the present disclosure, an alignment model that fits as many local alignment results as possible can be generated. According to some embodiments, an alignment model that can minimize the L0 norm, which counts the total number of non-zero distances between the local alignment results and the alignment model, can be determined as the alignment model for the corresponding inspection image. In some embodiments, outliers can be assumed to be defective or contaminated data resulting from, for example, snapping, half-pitch shift, etc. In some embodiments, these outliers are excluded when estimating the alignment model.

[0084]

[0099] According to some embodiments of this disclosure, the estimation of the alignment model can be performed based on random sample consensus (RANSAC). According to some embodiments of this disclosure, multiple alignment algorithms can be generated for multiple randomly selected subsets of local alignment results. In some embodiments, one alignment algorithm that exhibits the best performance among the multiple alignment algorithms can be selected as the alignment algorithm for correcting the examination image. In some embodiments, the performance of the alignment algorithm can be determined based on the number of local alignment results that fit the alignment algorithm. This disclosure describes the process of generating the alignment model with respect to Algorithm 1, and therefore, for brevity, a detailed explanation is omitted here.

[0085]

[0100] In step S940, distortion of the inspection image is corrected based on the selected alignment model. Step S940 can be performed, for example, by a distortion corrector 450. In some embodiments, based on the selected alignment model F, all patches of the inspection image corresponding to all local alignment results, including internal and outlier values ​​of the inspection image, can be corrected.

[0086]

[0101] Figure 10 is a process flowchart representing an exemplary method for training an alignment evaluation model, consistent with embodiments of the present disclosure. The steps of Method 1000 may be performed on a computing device, for example, the controller 109 in Figure 1, or by a system that operates using its functions (for example, System 700 in Figure 7B). It will be understood that the illustrated Method 1000 can be modified to change the order of the steps and to include additional steps.

[0087]

[0102] In step S1010, a pair of training inspection images and training reference images is acquired. Step S1010 can be performed, for example, by a training inspection image acquirer 710 or a training reference image acquirer 720. The training inspection image may be an inspection image patch. In some embodiments, the inspection image patch is a patch of SEM images of a sample or wafer. According to some embodiments, the training reference image may be a reference image patch compared to the training inspection image. In some embodiments, the reference image may be a layout file for wafer design. In some embodiments, the training reference image may or may not correspond to the training inspection image for training purposes.

[0088]

[0103] In step S1020, the alignment assessment model is trained. Step S1020 may be performed, for example, by the model trainer 730. According to some embodiments of the present disclosure, the alignment assessment model is trained to predict the alignment index of two images provided as input. According to some embodiments of the present disclosure, the alignment assessment model is trained under supervised learning. In some embodiments, information may be provided as to whether the training test image and the training reference image are aligned. In some embodiments, the alignment assessment model may be configured to provide an alignment index representing the degree of alignment between the two images. If the alignment index does not match the information as to whether the training test image and the training reference image are aligned, the parameters, weights, etc., of the alignment assessment model can be adjusted. The accuracy of the alignment index generated by the alignment assessment model 731 may be improved as the alignment assessment model is trained using further pairs of training test images and training reference images. An exemplary configuration of the alignment assessment model is illustrated with reference to Figure 8B, and therefore, for brevity, a detailed description is omitted here.

[0089]

[0104] A non-temporary computer-readable medium may be provided in which the processor of the controller (e.g., controller 109 in Figure 1) stores instructions for, among other things, image inspection, image acquisition, stage positioning, beam focusing, electric field adjustment, beam bending, focusing lens adjustment, activation of charged particle sources, beam deflection, and for performing methods 900 and 1000. General forms of non-temporary media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tapes, or any other magnetic data storage media, compact disk read-only memory (CD-ROM), any other optical data storage media, any physical media having a pattern of holes, random access memory (RAM), programmable ROM (PROM), and erasable programmable ROM (EPROM), FLASH-EPROM or any other flash memory, non-volatile random access memory (NVRAM), caches, registers, any other memory chips or cartridges and their networked versions.

[0090]

[0105] The embodiments can be further described using the following clauses: 1. A method for correcting distortion in inspection images, To obtain examination images, Determining the local alignment results of multiple patches in an examination image based on a reference image corresponding to the examination image. For each subset of the multiple subsets of local alignment results, An alignment model is determined based on a subset of local alignment results. Evaluating the alignment model based on the fit of the alignment model to the residual set of local alignment results. Based on the evaluation, select one alignment model from among several alignment models, and Correcting distortion in the inspection image based on the selected alignment model. Methods that include... 2. Evaluating the alignment model is The percentage of local alignment results that fit the alignment model is determined within the residual set of local alignment results. The method described in Clause 1, including the method described in Clause 1. 3. The method according to clause 1 or 2, wherein multiple subsets are randomly selected. 4. Aligning multiple patches of the examination image based on a reference image, and Evaluating the alignment between the first patch of multiple patches and the corresponding patch in the reference image using a machine learning model. The method described in any one of the clauses 1 to 3, further including the method described in any one of the clauses 1 to 3. 5. Obtain training examination image patches and training reference image patches, and Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. The method described in Clause 4, further including the following. 6. A device for correcting distortion in inspection images, A memory that stores a set of instructions, To obtain examination images, Determining the local alignment results of multiple patches in an examination image based on a reference image corresponding to the examination image. For each subset of the multiple subsets of local alignment results, An alignment model is determined based on a subset of local alignment results. Evaluating the alignment model based on the fit of the alignment model to the residual set of local alignment results. Based on the evaluation, select one alignment model from among several alignment models, and Correcting distortion in the inspection image based on the selected alignment model. at least one processor configured to execute a set of instructions to cause the device to perform A device including a device. 7. When evaluating the alignment model, at least one processor The percentage of local alignment results that fit the alignment model is determined within the residual set of local alignment results. The apparatus described in Clause 6, configured to execute a set of instructions to cause the apparatus to perform further actions. 8. The apparatus described in Clause 6 or 7, wherein multiple subsets are randomly selected. 9. At least one processor Aligning multiple patches of the examination image based on a reference image, and Evaluating the alignment between the first patch of multiple patches and the corresponding patch in the reference image using a machine learning model. An apparatus as described in any one of clauses 6 to 8, configured to execute a set of instructions to cause the apparatus to perform further actions. 10. At least one processor To obtain training test image patches and training reference image patches, Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. The apparatus described in Clause 9, configured to execute a set of instructions to cause the apparatus to perform further actions. 11. A non-temporary computer-readable medium storing a set of instructions executable on at least the processor of a computing device for causing the computing device to perform a method for correcting distortion of an inspection image, the method being To obtain examination images, Determining the local alignment results of multiple patches in an examination image based on a reference image corresponding to the examination image. For each subset of the multiple subsets of local alignment results, An alignment model is determined based on a subset of local alignment results. Evaluating the alignment model based on the fit of the alignment model to the residual set of local alignment results. Based on the evaluation, select one alignment model from among several alignment models, and Correcting distortion in the inspection image based on the selected alignment model. Non-temporary computer-readable media, including [specific examples of such media]. 12. When evaluating the alignment model, a set of instructions that can be executed by at least one processor of the computing device is The percentage of local alignment results that fit the alignment model is determined within the residual set of local alignment results. A computer-readable medium as described in Clause 11, which allows a computing device to perform the same actions. 13. A computer-readable medium as described in Clause 11 or 12, in which multiple subsets are randomly selected. 14. A set of instructions that can be executed by at least one processor of a computing device Aligning multiple patches of the examination image based on a reference image, and Evaluating the alignment between the first patch of multiple patches and the corresponding patch in the reference image using a machine learning model. A computer-readable medium as described in any one of clauses 11-13, which allows a computing device to perform the same actions. 15. A set of instructions that can be executed by at least one processor of a computing device To obtain training test image patches and training reference image patches, Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. A computer-readable medium as described in Clause 14, which allows a computing device to perform the same actions. 16. A method for correcting distortion in an inspection image, To obtain examination images, Determining the local alignment results of multiple patches in an examination image based on a reference image corresponding to the examination image. To estimate a first alignment model based on a first subset of local alignment results, and to estimate a second alignment model based on a second subset of local alignment results, Evaluate the first alignment model based on the fit of the first alignment model to the first residual set of local alignment results, and evaluate the second alignment model based on the fit of the second alignment model to the second residual set of local alignment results. Based on the evaluation, select one of the first and second alignment models, and Correcting distortion in the inspection image based on the selected alignment model. Methods that include... 17. Evaluating the first alignment model is Determine the percentage of local alignment results that fit the first alignment model within the first residual set of local alignment results. The method described in Article 16, including the method described in Article 16. 18. The method according to clause 16 or 17, wherein the first subset and the second subset are randomly selected. 19. Aligning multiple patches of examination images based on a reference image. Evaluating the alignment between the first patch of multiple patches and the corresponding patch in the reference image using a machine learning model. The method described in any one of the clauses 16 to 18, further including the method described in any one of the clauses 16 to 18. 20. Obtain training examination image patches and training reference image patches. Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. The method described in Article 19, further including the method described in Article 19. 21. A device for correcting distortion in inspection images, A memory that stores a set of instructions, To obtain examination images, Determining the local alignment results of multiple patches in an examination image based on a reference image corresponding to the examination image. To estimate a first alignment model based on a first subset of local alignment results, and to estimate a second alignment model based on a second subset of local alignment results, Evaluate the first alignment model based on the fit of the first alignment model to the first residual set of local alignment results, and evaluate the second alignment model based on the fit of the second alignment model to the second residual set of local alignment results. Based on the evaluation, select one of the first and second alignment models, and Correcting distortion in the inspection image based on the selected alignment model. at least one processor configured to execute a set of instructions to cause the device to perform A device including a device. 22. When evaluating the first alignment model, at least one processor Determine the percentage of local alignment results that fit the first alignment model within the first residual set of local alignment results. The apparatus described in Clause 21, configured to execute a set of instructions to cause the apparatus to perform further actions. 23. The apparatus according to clause 21 or 22, wherein a first subset and a second subset are randomly selected. 24. At least one processor Aligning multiple patches of the examination image based on a reference image, and Evaluating the alignment between the first patch of multiple patches and the corresponding patch in the reference image using a machine learning model. An apparatus as described in any one of clauses 21 to 23, configured to execute a set of instructions to cause the apparatus to perform further actions. 25. At least one processor To obtain training test image patches and training reference image patches, Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. The apparatus described in Clause 24, configured to execute a set of instructions to cause the apparatus to perform further actions. 26. A non-temporary computer-readable medium storing a set of instructions executable on at least the processor of a computing device for causing the computing device to perform a method for correcting distortion of an inspection image, the method being To obtain examination images, Determining the local alignment results of multiple patches in an examination image based on a reference image corresponding to the examination image. To estimate a first alignment model based on a first subset of local alignment results, and to estimate a second alignment model based on a second subset of local alignment results, Evaluate the first alignment model based on the fit of the first alignment model to the first residual set of local alignment results, and evaluate the second alignment model based on the fit of the second alignment model to the second residual set of local alignment results. Based on the evaluation, select one of the first and second alignment models, and Correcting distortion in the inspection image based on the selected alignment model. Non-temporary computer-readable media, including [specific examples of such media]. 27. When evaluating the first alignment model, A set of instructions that can be executed by at least one processor of a computing device Determine the percentage of local alignment results that fit the first alignment model within the first residual set of local alignment results. Computer-readable media as described in Clause 26, which further enables a computing device to perform the same action. 28. A computer-readable medium as described in Clause 26 or 27, wherein the first subset and the second subset are randomly selected. 29. A set of instructions that can be executed by at least one processor of a computing device Aligning multiple patches of the examination image based on a reference image, and Evaluating the alignment between the first patch of multiple patches and the corresponding patch in the reference image using a machine learning model. A computer-readable medium as described in any one of clauses 26-28, which allows a computing device to perform the same actions. 30. A set of instructions that can be executed by at least one processor of a computing device To obtain training test image patches and training reference image patches, Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. A computer-readable medium as described in Clause 29, which allows a computing device to perform the same actions. 31. A method for correcting distortion in an inspection image, To obtain examination images, Aligning multiple patches of an examination image based on a reference image corresponding to the examination image. Evaluating the alignment between each patch in a set of multiple patches and the corresponding patch in a reference image using a machine learning model. Determining the local alignment results of multiple patches in an examination image based on a reference image corresponding to the examination image. Determining the alignment model based on local alignment results, and Correcting distortion in inspection images based on an alignment model. Methods that include... 32. Realigning multiple patches of the examination images based on the alignment evaluation. The method described in Clause 31, further including the method described in Clause 31. 33. The method according to clause 31 or 32, wherein multiple subsets are randomly selected. 34. Obtain training examination image patches and training reference image patches, and Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. The method described in any one of the clauses 31 to 33, further including the method described in any one of the clauses 31 to 33. 35. Determining the alignment model is For each subset of the multiple subsets of local alignment results, An alignment model is determined based on a subset of local alignment results. Evaluating the alignment model based on the fit of the alignment model to the residual set of local alignment results, and Selecting one alignment model from several based on an evaluation of the alignment models. The method described in any one of the clauses 31 to 34, including the method described in any one of the clauses 31 to 34. 36. A device for correcting distortion in inspection images, A memory that stores a set of instructions, To obtain examination images, Aligning multiple patches of an examination image based on a reference image corresponding to the examination image. Evaluating the alignment between each patch in a set of multiple patches and the corresponding patch in a reference image using a machine learning model. Determining the local alignment results of multiple patches in an examination image based on a reference image corresponding to the examination image. Determining the alignment model based on local alignment results, and Correcting distortion in inspection images based on an alignment model. at least one processor configured to execute a set of instructions to cause the device to perform A device including a device. 37. At least one processor Realigning multiple patches of the examination image based on the alignment evaluation. The apparatus described in Clause 36, configured to execute a set of instructions to cause the apparatus to perform further actions. 38. The apparatus according to clause 36 or 37, wherein multiple subsets are randomly selected. 39. At least one processor To obtain training test image patches and training reference image patches, Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. An apparatus as described in any one of clauses 36 to 38, configured to execute a set of instructions to cause the apparatus to perform further actions. 40. When determining the alignment model, at least one processor For each subset of the multiple subsets of local alignment results, An alignment model is determined based on a subset of local alignment results. Evaluating the alignment model based on the fit of the alignment model to the residual set of local alignment results, and Selecting one alignment model from several based on an evaluation of the alignment models. An apparatus as described in any one of clauses 36 to 39, configured to execute a set of instructions to cause the apparatus to perform further actions. 41. A non-temporary computer-readable medium storing a set of instructions executable on at least the processor of a computing device for causing the computing device to perform a method for correcting distortion of an inspection image, wherein the method To obtain examination images, Aligning multiple patches of an examination image based on a reference image corresponding to the examination image. Evaluating the alignment between each patch in a set of multiple patches and the corresponding patch in a reference image using a machine learning model. Determining the local alignment results of multiple patches in an examination image based on a reference image corresponding to the examination image. Determining the alignment model based on local alignment results, and Correcting distortion in inspection images based on an alignment model. Non-temporary computer-readable media, including [specific examples of such media]. 42. A set of instructions that can be executed by at least one processor of a computing device Realigning multiple patches of the examination image based on the alignment evaluation. A computer-readable medium as described in Clause 41, which allows a computing device to perform the same actions. 43. A computer-readable medium as described in Clause 41 or 42, in which multiple subsets are randomly selected. 44. A set of instructions that can be executed by at least one processor of a computing device To obtain training test image patches and training reference image patches, Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. A computer-readable medium as described in any one of clauses 41 to 43, which allows a computing device to perform the same actions. 45. When determining the alignment model, a set of instructions that can be executed by at least one processor of the computing device is For each subset of the multiple subsets of local alignment results, An alignment model is determined based on a subset of local alignment results. Evaluating the alignment model based on the fit of the alignment model to the residual set of local alignment results, and Selecting one alignment model from several based on an evaluation of the alignment models. A computer-readable medium as described in any one of clauses 41-44, which allows a computing device to perform the same actions. 46. ​​A method for evaluating the alignment of examination images and reference images, The acquisition of multiple patches of the examination image and multiple reference patches of the reference image, wherein the multiple patches correspond to multiple reference patches, and Evaluating the alignment of multiple patches and multiple reference patches using a machine learning model. Methods that include... 47. Evaluating alignment using machine learning models is Evaluating the alignment of the first patch of multiple patches and the first reference patch of multiple reference patches, and To generate an alignment index that represents the confidence that the first patch is aligned with the first reference patch. The method described in Article 46, including the method described in Article 46. 48. Evaluating alignment using machine learning models is Evaluating the alignment of each pair of multiple patches and multiple reference patches, and Generate an alignment index for each pair, representing the confidence level of alignment between the pairs. The method described in Article 46, including the method described in Article 46. 49. Obtain training examination image patches and training reference image patches, and Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. The method described in any one of the clauses 46 to 48, further including the method described in any one of the clauses 46 to 48. 50. Obtain pairs of training test image patches and training reference image patches. To obtain information on whether the training test image patch and the training reference image patch are aligned. The method described in Article 49, including the method described in Article 49. 51. A device for evaluating the alignment of inspection images and reference images, A memory that stores a set of instructions, The acquisition of multiple patches of the examination image and multiple reference patches of the reference image, wherein the multiple patches correspond to multiple reference patches, and Evaluating the alignment of multiple patches and multiple reference patches using a machine learning model. at least one processor configured to execute a set of instructions to cause the device to perform A device including a device. 52. When evaluating alignment using a machine learning model, at least one processor Evaluating the alignment of the first patch of multiple patches and the first reference patch of multiple reference patches, and To generate an alignment index that represents the confidence that the first patch is aligned with the first reference patch. The apparatus described in Clause 51, configured to execute a set of instructions to cause the apparatus to perform further actions. 53. When evaluating alignment using a machine learning model, at least one processor Evaluating the alignment of each pair of multiple patches and multiple reference patches, and Generate an alignment index for each pair, representing the confidence level of alignment between the pairs. The apparatus described in Clause 51, configured to execute a set of instructions to cause the apparatus to perform further actions. 54. At least one processor To obtain training test image patches and training reference image patches, Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. An apparatus as described in any one of clauses 51 to 53, configured to execute a set of instructions to cause the apparatus to perform further actions. 55. When acquiring a pair of training test image patches and training reference image patches, at least one processor To obtain information on whether the training test image patch and the training reference image patch are aligned. The apparatus described in Clause 54, configured to execute a set of instructions to cause the apparatus to perform further actions. 56. A non-temporary computer-readable medium storing a set of instructions executable on at least the processor of a computing device for causing the computing device to perform a method for evaluating the alignment of an examination image and a reference image, wherein the method The acquisition of multiple patches of the examination image and multiple reference patches of the reference image, wherein the multiple patches correspond to multiple reference patches, and Evaluating the alignment of multiple patches and multiple reference patches using a machine learning model. Non-temporary computer-readable media, including [specific examples of such media]. 57. When evaluating alignment using a machine learning model, a set of instructions that can be executed by at least one processor of the computing device Evaluating the alignment of the first patch of multiple patches and the first reference patch of multiple reference patches, and To generate an alignment index that represents the confidence that the first patch is aligned with the first reference patch. Computer-readable media as described in Clause 56, which further enables a computing device to perform the same action. 58. When evaluating alignment using a machine learning model, a set of instructions that can be executed by at least one processor of the computing device Evaluating the alignment of each pair of multiple patches and multiple reference patches, and Generate an alignment index for each pair, representing the confidence level of alignment between the pairs. Computer-readable media as described in Clause 56, which further enables a computing device to perform the same action. 59. A set of instructions that can be executed by at least one processor of a computing device To obtain training test image patches and training reference image patches, Training a machine learning model to predict the alignment index between a training examination image patch and a training reference image patch. A computer-readable medium as described in any one of clauses 56-58, which causes a computing device to perform the same action. 60. When acquiring a pair of training test image patches and training reference image patches, a set of instructions that can be executed by at least one processor of the computing device To obtain information on whether the training test image patch and the training reference image patch are aligned. Computer-readable media as described in Clause 59, which further enables a computing device to perform the same action.

[0091]

[0106] The block diagrams in the figures may illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer hardware or software products according to various exemplary embodiments of the present disclosure. In this regard, each block in the schematic diagram may represent a certain arithmetic or logical operation that can be implemented using hardware such as electronic circuits. A block may also represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should be understood that in some alternative implementations, the functions shown within a block may occur in a different order than shown in the drawings. For example, depending on the functions involved, two consecutively shown blocks may be executed or implemented almost simultaneously, or two blocks may be executed in reverse order. Some blocks may be omitted. It should also be understood that each block and combination of blocks in the block diagram may be implemented by a dedicated hardware-based system that performs a specified function or action, or by a combination of dedicated hardware and computer instructions.

[0092]

[0107] It should be understood that the embodiments of this disclosure are not limited to those described above and illustrated in the accompanying drawings, and that various modifications and changes can be made without departing from the scope of the invention. Although this disclosure has been described in relation to various embodiments, other embodiments of the invention will be obvious to those skilled in the art from the discussion herein and the practice of the invention disclosed herein. It is intended that this specification and examples are illustrative only, and that the true scope and spirit of the invention are indicated by the following claims.

Claims

1. A method for correcting distortion in inspection images, To obtain examination images, Determining the local alignment results of multiple patches in the examination image based on a reference image corresponding to the examination image, For each subset of the multiple subsets of the aforementioned local alignment results, An alignment model is determined based on the subset of the local alignment results. Evaluating the alignment model based on the fit of the alignment model to the residual set of the local alignment results, Based on the above evaluation, select one alignment model from among the multiple alignment models, and Correcting the distortion of the inspection image based on the selected alignment model. Methods that include...

2. Evaluating the aforementioned alignment model is The percentage of local alignment results that fit the alignment model is determined within the residual set of local alignment results. The method according to claim 1, including the method described in claim 1.

3. The method according to claim 1, wherein the plurality of subsets are randomly selected.

4. Aligning the multiple patches of the inspection image based on the reference image, and The alignment between the first patch of the plurality of patches and the corresponding patch of the reference image is evaluated using a machine learning model. The method according to claim 1, further comprising:

5. To obtain training test image patches and training reference image patches, Training the machine learning model to predict the alignment index between the training examination image patch and the training reference image patch. The method according to claim 4, further comprising:

6. A device for correcting distortion in inspection images, A memory that stores a set of instructions, To obtain examination images, Determining the local alignment results of multiple patches in the examination image based on a reference image corresponding to the examination image, For each subset of the multiple subsets of the aforementioned local alignment results, An alignment model is determined based on the subset of the local alignment results. Evaluating the alignment model based on the fit of the alignment model to the residual set of the local alignment results, Based on the above evaluation, select one alignment model from among the multiple alignment models, and Correcting the distortion of the inspection image based on the selected alignment model. at least one processor configured to execute the set of commands to cause the device to perform the following: A device including a device.

7. When evaluating the alignment model, the at least one processor The percentage of local alignment results that fit the alignment model is determined within the residual set of local alignment results. The apparatus according to claim 6, configured to execute the set of commands for causing the apparatus to perform the following further action.

8. The apparatus according to claim 6, wherein the plurality of subsets are randomly selected.

9. The at least one processor Aligning the multiple patches of the inspection image based on the reference image, and The alignment between the first patch of the plurality of patches and the corresponding patch of the reference image is evaluated using a machine learning model. The apparatus according to claim 6, configured to execute the set of commands for causing the apparatus to perform the following further action.

10. The at least one processor To obtain training test image patches and training reference image patches, Training the machine learning model to predict the alignment index between the training examination image patch and the training reference image patch. The apparatus according to claim 9, configured to execute the set of commands for causing the apparatus to perform the following further action.

11. A non-temporary computer-readable medium storing a set of instructions executable by at least one processor of a computing device for causing the computing device to perform a method for correcting distortion of an inspection image, wherein the method is: To obtain examination images, Determining the local alignment results of multiple patches in the examination image based on a reference image corresponding to the examination image, For each subset of the multiple subsets of the aforementioned local alignment results, An alignment model is determined based on the subset of the local alignment results. Evaluating the alignment model based on the fit of the alignment model to the residual set of the local alignment results, Based on the above evaluation, select one alignment model from among the multiple alignment models, and Correcting the distortion of the inspection image based on the selected alignment model. Non-temporary computer-readable media, including [specific examples of such media].

12. When evaluating the alignment model, the set of instructions that can be executed by at least one processor of the computing device The percentage of local alignment results that fit the alignment model is determined within the residual set of local alignment results. The computer-readable medium according to claim 11, wherein the computing device further performs the same action.

13. The computer-readable medium according to claim 11, wherein the plurality of subsets are randomly selected.

14. The set of instructions that can be executed by at least one processor of the computing device Aligning the multiple patches of the inspection image based on the reference image, and The alignment between the first patch of the plurality of patches and the corresponding patch of the reference image is evaluated using a machine learning model. The computer-readable medium according to claim 11, wherein the computing device further performs the same action.

15. The set of instructions that can be executed by at least one processor of the computing device To obtain training test image patches and training reference image patches, Training the machine learning model to predict the alignment index between the training examination image patch and the training reference image patch. The computer-readable medium according to claim 14, wherein the computing device further performs the same action.

Images