Defect detection using neural networks and defect intensity thresholding from depth maps derived from scanning electron microscopy

Abstract

Described herein are machine learning based systems, methods, and non-transitory computer readable media for determining three-dimensional (3D) information of structures of a patterned substrate. The 3D information can be determined using a neural network to convert a two-dimensional image to a 3D image. In a method, the neural network is trained by supplying varying parameters to simulated images of a sample to model a real manufactured IC structure. The trained network generates a depth map from a newly supplied image of the sample to predict defect locations. The depth map can be converted to a binary map to predict defect size and location, which can be used to guide inspection of the sample using an inspection tool.

Description

Cross-references to related applications

[0001] This application claims priority to US application 63 / 542,286, filed October 3, 2023, which is incorporated herein by reference in its entirety. Technical Field

[0002] This paper generally describes improved measurement systems and methods. More specifically, it relates to methods for determining and employing models configured to use trained neural networks to determine three-dimensional data of structures patterned on a substrate. Background Technology

[0003] Image inspection apparatuses (e.g., charged particle beam apparatuses or beam apparatuses) can generate two-dimensional (2D) images of a wafer substrate by detecting particles (e.g., photons, secondary electrons, backscattered electrons, mirror electrons, or other types of electrons) from the surface of the wafer substrate when struck by a beam (e.g., a charged particle beam or beam) generated by a source associated with the inspection apparatus. Various image inspection apparatuses are used on semiconductor wafers in the semiconductor industry for a variety of purposes, such as wafer processing (e.g., electron beam direct-write lithography systems), process monitoring (e.g., critical-size scanning electron microscopy (CD-SEM)), wafer inspection (e.g., electron beam inspection systems), or defect analysis (e.g., defect re-inspection SEM, or DR-SEM, and focused ion beam systems, or FIB).

[0004] To control the quality of structures fabricated on wafer substrates, 2D images of the wafer substrate can be analyzed to detect potential defects. In some applications, 2D images can be compared to simulated images. Simulated images can be generated using simulation techniques configured to mimic images measured by image inspection devices. In some applications, 2D geometric features (e.g., edges) or three-dimensional (3D) geometric features can be extracted from 2D images based on simulated images. The quality of the simulated images can be a significant factor in the performance and accuracy of these applications. Summary of the Invention

[0005] With advancements in photolithography and other patterning technologies, the size of functional components has been continuously miniaturized, while the number of functional components (such as transistors) in each device has steadily increased over the decades. Simultaneously, the requirements for accuracy in critical dimensions (CD), height, and other aspects have become increasingly stringent. Errors in the shape and size of the structure can lead to functional problems in the device, including malfunctions or one or more electrical issues in operating equipment. Therefore, it is desirable to measure the three-dimensional structure of functional components to characterize, reduce, or minimize one or more defects in the device. However, measuring the three-dimensional (3D) structure of functional components using existing metrology tools and methods is time-consuming and inaccurate, negatively impacting the productivity of patterning processes.

[0006] Some embodiments of this disclosure provide a method for defect inspection. The method includes: converting a two-dimensional image into a three-dimensional image via a trained convolutional neural network (CNN); calculating the defect intensity at a location among multiple locations in the three-dimensional image; generating a binary map by aggregating the calculated defect intensities from the multiple locations; filtering the binary map to remove noise; and using the binary map to guide the inspection of a sample using an inspection tool.

[0007] In some embodiments, a system is provided. The system includes: an image inspection apparatus configured to scan a sample and generate an image; and one or more processors configured to execute instructions to cause the system to perform operations. The operations include: converting a two-dimensional image into a three-dimensional image via a trained convolutional neural network (CNN); calculating defect intensities at locations among multiple locations in the three-dimensional image; aggregating the calculated defect intensities from the multiple locations to generate a binary map; filtering the binary map to remove noise; and using the binary map to guide the inspection of the sample using an inspection tool.

[0008] In some embodiments, a non-transitory computer-readable medium is provided, comprising a set of instructions executable by one or more processors of a computing device to cause the computing device to perform operations for defect inspection. The operations include: converting a two-dimensional image into a three-dimensional image via a trained convolutional neural network (CNN); calculating defect intensities at locations among multiple locations in the three-dimensional image; generating a binary map by aggregating the calculated defect intensities from the multiple locations; filtering the binary map to remove noise; and using the binary map to guide the inspection of a sample using an inspection tool.

[0009] Other advantages of this disclosure will become apparent from the following description taken in conjunction with the accompanying drawings, in which certain embodiments of the disclosure are illustrated and demonstrated by way of example. Attached Figure Description

[0010] The above and other aspects and features will become apparent to those skilled in the art from the following description of specific embodiments, taken in conjunction with the accompanying drawings, in which:

[0011] Figure 1 Block diagrams of various subsystems of a lithography system according to some embodiments are shown;

[0012] Figure 2A The illustration shows an example SEM image of a line with a depth of 10 nm in the z-direction, starting from the top of the substrate, according to some embodiments.

[0013] Figure 2B The illustration shows another example SEM image of a line with a depth of 100 nm in the z-direction, starting from the top of the substrate, according to some embodiments.

[0014] Figure 2C The illustration shows SEM images and corresponding depth maps of contact holes patterned on a substrate according to some embodiments.

[0015] Figure 2D The illustration shows an SEM image of a patterned set of lines on a substrate and a corresponding depth map, according to some embodiments.

[0016] Figure 2E The illustration shows an SEM image and a corresponding depth map of another set of lines patterned on a substrate, according to some embodiments.

[0017] Figure 3 This is a block diagram illustrating the training process of a model based on some embodiments;

[0018] Figure 4 The illustration shows the correlation between height and parallax data associated with a stereoscopic image of a patterned substrate, according to some embodiments;

[0019] Figure 5 This is a flowchart of a method for training a model for estimating depth information of the structure of a patterned substrate, according to some embodiments;

[0020] Figure 6A Based on some embodiments, use according to Figure 5 A block diagram illustrating an example of a trained model determining depth information of a structure formed on a substrate;

[0021] Figure 6BBased on some embodiments, use according to Figure 5 A flowchart of a method for using a trained model to determine depth information of a structure formed on a substrate;

[0022] Figure 7 This is a flowchart of a method for generating a model configured to estimate depth data of a patterned substrate structure, according to some embodiments.

[0023] Figure 8A The illustration shows an exemplary training of a model using simulated SEM data, according to some embodiments.

[0024] Figure 8B The illustration shows the use of measurement data to ( ) according to some embodiments. Figure 8A Exemplary fine-tuning performed on the model;

[0025] Figure 8C The illustration shows the effect of observed measurement data on the measurement of... according to some embodiments. Figure 8A The model generates an example of scaling the estimated depth data, and the scaled estimated data can be used to further fine-tune the model.

[0026] Figure 9A , Figure 9B and Figure 9C The figures illustrate SEM images with charging effect, predicted depth data with charging effect, and predicted depth data without charging effect, according to some embodiments.

[0027] Figure 10A According to some embodiments, it is for use by ( Figure 7 The flowchart shows a method for estimating depth data using a trained model and a single SEM image as input.

[0028] Figure 10B The illustration shows exemplary depth data estimated using a trained model according to some embodiments;

[0029] Figure 11 This is a flowchart of another method, according to some embodiments, for training a model for estimating depth information of the structure of a patterned substrate;

[0030] Figure 12 The illustration shows, according to some embodiments, based on Figure 11 The method is an example of training a CNN, in which training data generated by a deterministic process simulator is used;

[0031] Figure 13 This is a flowchart of yet another method for training a model for estimating depth information of the structure of a patterned substrate, according to some embodiments;

[0032] Figure 14 The illustration shows, according to some embodiments, based on Figure 13 The method is an example of training a CNN, in which training data generated by a deterministic process simulator is used;

[0033] Figure 15 According to some embodiments, it is for use by ( Figure 11 or Figure 13 A flowchart of a method for estimating depth data using a trained model and a single SEM image as input;

[0034] Figure 16 This is an example illustration of generating a depth map of a sample by training a convolutional neural network (CNN) according to some embodiments;

[0035] Figure 17 This is an example illustration of a depth map of a sample generated from an input image, according to some embodiments;

[0036] Figure 18A These are example illustrations of input images used for training the output depth map of a CNN, according to some embodiments.

[0037] Figures 18B to 18G This is an example illustration of a binary map generated by aggregating the defect intensities calculated at each location in the generated depth map, according to some embodiments.

[0038] Figure 19A These are example illustrations illustrating the use of machine learning models to quantify defect size according to some embodiments;

[0039] Figure 19B and Figure 19C This is an example illustration of a metric used to quantify machine learning models to determine defect size, based on some embodiments.

[0040] Figure 20 The illustration shows example correlation charts between predicted defect sizes and ground truth defect sizes with different parameter values ​​according to some embodiments;

[0041] Figure 21 This is a flowchart of an exemplary machine learning method for defect detection according to some embodiments;

[0042] Figure 22 An example of a scanning electron microscope (SEM) according to some embodiments is schematically depicted;

[0043] Figure 23An example of an electron beam inspection apparatus according to some embodiments is schematically depicted;

[0044] Figure 24 This is a block diagram of an example computer system according to some embodiments;

[0045] Figure 25 This is a schematic diagram of a photolithography projection apparatus according to some embodiments;

[0046] Figure 26 This is a schematic diagram of another photolithography projection apparatus according to some embodiments;

[0047] Figure 27 Based on some embodiments, Figure 25 A more detailed view of the device in the image; and

[0048] Figure 28 Based on some embodiments, Figure 26 and Figure 27 A more detailed view of the source collector module SO of the device. Detailed Implementation

[0049] Integrated circuit (IC) chips used in devices (e.g., telephones, laptops, computer memory, etc.) comprise complex circuit patterns. During the fabrication of such circuit patterns, images of the printed circuit patterns are captured to determine whether the desired circuit pattern has been accurately printed. The final performance of the manufactured device critically depends on the accuracy of the positioning and dimensional determination of various features of the product structure formed via photolithography and other processing steps. These features are three-dimensional (3D) structures with predetermined depths and shapes at the nanometer scale. A product structure produced by an imperfect photolithography process or other processing steps will result in a structure that differs slightly from the ideal or nominal desired structure.

[0050] To examine the dimensions of various features, three-dimensional information (e.g., feature height) can be highly beneficial in ensuring that a feature at one layer is connected to a feature at another layer. However, obtaining 3D information of nanoscale structures is not straightforward. In the prior art, 3D information can be obtained via tilt-beam scanning electron microscopy (SEM), which requires two or more images of the same location to infer appropriate depth information. However, using multiple images for 3D measurement has several limitations. For example, capturing paired stereo images reduces the throughput of patterning or measurement processes because the beam tilt angle must be switched. Capturing stereo images requires proper alignment between the images. Processing stereo images to determine depth information can be computationally expensive and susceptible to noise in the images and drift in the measurement hardware. Therefore, using existing techniques to extract 3D information may slow down chip manufacturing and measurement processes.

[0051] In some embodiments, machine learning may be employed in generating inspection images, reference images, or other images associated with devices 100, 104A, or 104B. For example, in some embodiments, the machine learning system may be used in conjunction with, for example... Figures 1 to 2B The controller 109 or 296, image processing system 199 or 290, image acquirer 120 or 292, or storage unit 130 or 294 operate in association. In some embodiments, machine learning can be implemented in a real-time recipe tuning method (e.g., as described below). Figure 3 Method 300 or Method 800 in Figure 8) and, for example Figure 3 The offline computing platform 301 is associatedly employed. In some embodiments, the machine learning system may include a discriminative model. In some embodiments, the machine learning system may include a generative model. For example, learning may have two types of mechanisms: discriminative learning, which can be used to create classification and detection algorithms; and generative learning, which can be used to actually create models, and in extreme cases, models that can render images. For example, as further described below, a generative model may be configured to generate images from design clips that resemble corresponding locations on the wafer in SEM images. This can be done by: 1) training the generative model using the design clips and associated actual SEM images from those locations on the wafer; and 2) using the model in inference mode to feed the model with the design clips at the locations of the desired simulated SEM images. Such simulated images may be used as reference images, for example, in die-to-database inspections.

[0052] If the model(s) include one or more discriminative models, then the discriminative models(s) can have any suitable architecture and / or configuration known in the art. Discriminative models, also known as conditional models, are a class of models in machine learning used to model the correlation between an unobserved variable “y” and an observed variable “x”. Within a probabilistic framework, this is done by modeling a conditional probability distribution P(y|x), which can be used to predict y based on x. Unlike generative models, discriminative models may not allow the generation of samples from the joint distribution of x and y. However, for tasks such as classification and regression that do not require a joint distribution, discriminative models can produce superior performance. On the other hand, generative models are generally more flexible than discriminative models in expressing correlations in complex learning tasks. Furthermore, most discriminative models are inherently supervised and may not be easily extended to unsupervised learning. The specific details of the application ultimately determine the suitability of choosing between discriminative and generative models.

[0053] Generative models can generally be defined as probabilistic models in nature. In other words, a "generative" model is not a model that performs forward simulation or a rule-based approach, and therefore, it may not be necessary to model the physical characteristics of the processes involved in generating the actual image or output (a simulated image or output of the actual image or output is generated). Instead, a generative model can be learned based on an appropriate training dataset (i.e., its parameters can be learned). Such generative models can have many advantages for the embodiments described herein. Furthermore, generative models can be configured with deep learning architectures, as generative models can include multiple layers that can perform several algorithms or transformations. The number of layers included in a generative model can depend on the specific use case. For practical purposes, a suitable range of layers is from 2 to dozens.

[0054] Deep learning is a type of machine learning. Machine learning can generally be defined as a type of artificial intelligence (AI) that provides computers with the ability to learn without being explicitly programmed. Machine learning focuses on the development of computer programs that can teach themselves to grow and change when exposed to new data. Machine learning explores the research and construction of algorithms that can learn from and predict data, overcoming the need to follow strict static program instructions by building models from sample inputs to make data-driven predictions or decisions.

[0055] The machine learning described herein can also be performed as described in “Introduction to Statistical Machine Learning,” by Sugiyama and Morgan Kaufmann, 2016, 534 pages; “Discriminative, Generative, and Imitative Learning,” Jebara, MIT Thesis, 2002, 212 pages; and “Principles of Data Mining (Adaptive Computation and Machine Learning),” Hand et al., MIT Press, 2001, 578 pages, which are incorporated herein by reference as if fully described herein. The embodiments described herein can also be configured as described in these references.

[0056] In some embodiments, a machine learning system may include a neural network. For example, the model may be a deep neural network having a set of weights that model the world based on data that has been fed into the network to train it. Neural networks can generally be defined as computational methods that loosely model how the biological brain solves problems using clusters of relatively large biological neurons connected by axons, based on a relatively large set of neural units. Each neural unit is connected to many other units, and the links can have a reinforcing or inhibitory effect on the activation state of the connected neural units. These systems are self-learning and trained, rather than explicitly programmed, and excel in areas where traditional computer programs struggle to express solutions or feature detection.

[0057] Neural networks typically consist of multiple layers, with signal paths traversing from front to back. The goal of neural networks is to solve problems in the same way as the human brain, but some neural networks are far more abstract. Modern neural network projects often deal with thousands to millions of neural units and millions of connections. Neural networks can have any suitable architecture and / or configuration known in the art.

[0058] In another embodiment, the model may include convolutional and deconvolutional neural networks. For example, the embodiments described herein can leverage learned concepts such as convolutional and deconvolutional neural networks to solve representation transformation problems (e.g., rendering) that are typically difficult to handle. The model may have any convolutional and deconvolutional neural network configuration or architecture known in the art.

[0059] In another embodiment, the model may include convolutional and deconvolutional neural networks. For example, the embodiments described herein can leverage learned concepts such as convolutional and deconvolutional neural networks to solve representation transformation problems (e.g., rendering) that are typically difficult to handle. The model may have any convolutional and deconvolutional neural network configuration or architecture known in the art.

[0060] While this document may specifically refer to the manufacture of ICs, it should be clearly understood that the description herein has many other possible applications. For example, it can be used in the manufacture of integrated optical systems, guide and detection patterns for magnetic domain memory, liquid crystal display panels, thin-film magnetic heads, and the like. Those skilled in the art will understand that, in the context of such alternative applications, any use of the terms “mask,” “wafer,” or “die” herein should be considered interchangeable with the more general terms “mask,” “substrate,” and “target portion,” respectively. The substrates referred to herein may be processed before or after exposure, in, for example, a track (a tool typically applied to the substrate and developed by the exposed resist) or a measurement or inspection tool. Where applicable, the disclosure herein can be applied to such and other substrate processing tools. Furthermore, a substrate may be processed more than once, for example, to create a multilayer IC, such that the term “substrate” as used herein may also refer to a substrate that already contains multiple processed layers.

[0061] The critical dimension (CD) of a device refers to the minimum width of a line or hole, or the minimum space between two lines or holes. Therefore, the CD determines the overall size and density of the designed device. Of course, one goal of device manufacturing is to faithfully reproduce the original design intent on the substrate (via a patterning device).

[0062] In this document, the terms “radiation” and “beam” can be used to cover all types of electromagnetic radiation, including ultraviolet radiation (e.g., having wavelengths of 365 nm, 248 nm, 193 nm, 157 nm, or 126 nm) and EUV (extreme ultraviolet radiation, e.g., having wavelengths in the range of about 5 nm to 100 nm).

[0063] As used herein, the terms "mask" or "patterning apparatus" can be broadly interpreted to refer to a general patterning apparatus that can be used to impart a patterned cross-section to an incoming radiation beam, corresponding to a pattern to be created in a target portion of the substrate; the term "optical valve" can also be used in this context. Examples of other such patterning apparatuses besides classic masks (transmissive or reflective; binary, phase-shifting, hybrid, etc.) include: Programmable mirror arrays. An example of such a device is a matrix-addressable surface with a viscoelastic control layer and a reflective surface. The basic principle behind such a device is that, for example, the addressed regions of the reflective surface reflect incident radiation as diffracted radiation, while the unaddressed regions reflect incident radiation as non-diffracted radiation. Using appropriate filters, the non-diffracted radiation can be filtered out from the reflected beam, leaving only the diffracted radiation; in this way, the beam is patterned according to the addressing pattern of the matrix-addressable surface. The desired matrix addressing can be performed using suitable electronic methods. Programmable LCD array. An example of such a construction is given in U.S. Patent No. 5,229,872, which is incorporated herein by reference.

[0064] As a brief introduction, Figure 1 An exemplary photolithography projection apparatus 10A is illustrated. Key components include a radiation source 12A, which may be a deep ultraviolet excimer laser source or other types of sources, including extreme ultraviolet (EUV) sources (as mentioned above, the photolithography projection apparatus itself does not need to have a radiation source); irradiation optics, which may define partial coherence (denoted as sigma) and may include optics 14A, 16Aa, and 16Ab for shaping the radiation from source 12A; a patterning apparatus 18A; and a transmission optics 16Ac, which projects an image of a pattern from the patterning apparatus onto a substrate plane 22A. An adjustable filter or aperture 20A at the pupil plane of the projection optics can limit the range of beam angles striking the substrate plane 22A, wherein the maximum possible angle defines the numerical aperture NA = n sin(Θ) of the projection optics. max ), where n is the refractive index of the medium between the substrate and the final element of the projection optics, and Θ max It is the maximum angle of the beam emitted from the projection optics that can still strike the substrate plane 22A.

[0065] In a photolithography projection apparatus, a source provides irradiation (i.e., radiation) to a patterning apparatus, and projection optics guide and shape the irradiation onto a substrate via the patterning apparatus. The projection optics may include at least some of components 14A, 16Aa, 16Ab, and 16Ac. A spatial image (AI) is the distribution of radiation intensity at the substrate level. A resist layer on the substrate is exposed, and the spatial image, as a potential “resist image” (RI), is transferred to the resist layer. The resist image (RI) can be defined as the spatial distribution of the solubility of the resist in the resist layer. A resist model can be used to calculate the resist image based on the spatial image; examples of resist models can be found in U.S. Patent Application Publication No. US2009-0157360, the disclosure of which is incorporated herein by reference in its entirety. The resist model relates only to the properties of the resist layer (e.g., the effects of chemical processes occurring during exposure, PEB, and development). The optical characteristics of a photolithography projection apparatus (e.g., the characteristics of the source, patterning equipment, and projection optics) determine the spatial image. Since the patterning equipment used in a photolithography projection apparatus can be replaced, it may be desirable to separate the optical characteristics of the patterning equipment from the optical characteristics of the rest of the photolithography projection apparatus, which includes at least the source and projection optics.

[0066] Once a semiconductor chip is manufactured, measurements can be performed to determine the dimensions of the structures fabricated on the substrate. For example, inspection of the fabricated structure can be based on critical dimension (CD) measurements obtained using metrological tools such as SEM, AFM, optical tools, etc. In some embodiments, the lateral dimensions of the structure (e.g., in the x, y plane) can be extracted from the CD measurements. In addition to the lateral dimensions of the feature, measurements (e.g., CD-SEM) may also include information about the feature's 3D dimensions (e.g., height, CD values ​​at different heights, height profile, etc.). In some embodiments, the height of the feature refers to its depth in the z-direction perpendicular to the xy plane.

[0067] Figure 2A and Figure 2B The illustration shows two different SEM images of lines with the same CD but different heights. Figure 2A The diagram illustrates the process starting from the top of the substrate in the z-direction (perpendicular to) Figure 2A and Figure 2B SEM image I1 of a line with a depth of 10 nm on a plane, and Figure 2BAnother SEM image I2 illustrates a line with a depth of 100 nm in the z-direction starting from the top of the substrate. These figures demonstrate the significant impact of depth on the captured SEM images. However, although 3D information (e.g., height information) is meaningful, it is rarely extracted. In the prior art, only lateral information such as CD can be extracted, while height cannot.

[0068] For many problems related to semiconductor manufacturing, there may be correlations between inputs and outputs, but these correlations can be too complex for humans to model or identify. According to this disclosure, the model is trained to extract depth information from a single image of the substrate (e.g., a SEM image). For example, deep learning algorithms can be applied using the training data and methods described herein to train machine learning models such as convolutional neural networks (e.g., CNNs). In some embodiments, the methods described herein enable the extraction of height features from a single image of the substrate. For example, Figures 2C to 2E Different SEM images I3, I4, and I5 are illustrated, and for each SEM image, a depth map can be extracted from the SEM image according to this disclosure. Figures 2C to 2E In the diagram, input SEM images I3, I4, and I5 are depicted in the top row, along with their corresponding depth maps. Figure 3 D3, 3D4, and 3D5 are depicted in the bottom row.

[0069] In some embodiments, the beam tilting capability of a SEM tool can be used to retrieve depth information from a single CD-SEM image. Unsupervised training of the model can be performed using stereo image pairs (e.g., obtained via beam-tilted SEM images) as the training dataset. When semiconductor topology is measured using CD-SEM, the electron beam is typically incident normally relative to the substrate, producing a top-view image. This measurement method can resolve substrate measurements with a lateral resolution close to 1 nm, while axial resolution is often neglected due to the ill-posed problem of how to convert the SEM signal into topological depth. To address this ill-posed problem, beam-tilted SEM can be used to generate information about the depth of the structure. For example, “C. Valade, et al., Tilted beam SEM, 3D metrology for industry The document “Proc. SPIE10959, Metrology, Inspection, and Process Control for Microlithography XXXIII, 109590Y (26 March 2019)” describes an example tilt-beam SEM for performing 3D measurements. In such a system, stereo image pairs created by tilted electron beams from different directions are typically used.

[0070] For example, SEM tools can guide the electron beam at desired angles (e.g., up to 12° perpendicular to the substrate) to capture images at different angles, thereby obtaining more information. Based on the analysis of such images, the pattern height and sidewall angles can be determined accordingly using geometric considerations.

[0071] However, in tilt-beam SEM, two or more images of the same location are required to infer appropriate depth information for the structure. For example, two images of the same structure on a substrate are obtained, the first with the beam incident in the normal direction, and the second with the beam incident at a tilt angle. Using multiple images for 3D measurement has several limitations. For instance, capturing paired stereo images reduces the productivity of patterning or measurement processes because the beam tilt angle must be switched. Capturing stereo images requires proper alignment between the images. Processing stereo images to determine depth information can be computationally expensive and susceptible to noise in the images and drift in the measurement hardware. Capturing stereo images requires measurement hardware to facilitate ease of use when changing the beam tilt direction. Capturing images at a tilt angle reduces the effective field of view because only areas present in all images can be used for depth inference.

[0072] In some embodiments, this disclosure describes a mechanism for estimating the depth of a structure using a single image (e.g., a SEM image) captured at a first tilt angle (e.g., perpendicular to the substrate). This potentially mitigates most of the disadvantages associated with stereo images described above. For example, once the model is trained according to this disclosure, depth information can be determined using a single SEM image, thereby increasing productivity, reducing measurement time, and reducing computational resources compared to methods using stereo images to determine depth information.

[0073] In some embodiments, this disclosure describes unsupervised training methods for training machine learning models (e.g., CNNs). In some embodiments, the term "unsupervised" refers to a training method that does not use benchmark ground truth depth data when training the model, or that does not involve human intervention during the training process. The methods described herein are inherently capable of learning how to perform depth estimation. In some embodiments, convolutional neural networks (CNNs) with a U-net-like architecture (e.g., encoder, decoder, and skip connections between them) can be employed to train the CNN.

[0074] Figure 3 This is a block diagram illustrating the training process of model M0 according to some embodiments of this disclosure. After training, the trained model can be referred to as model M1. Figure 3As depicted, model M0 (e.g., a CNN) receives an image 301 of the substrate as input and outputs parallax data 311 associated with the input image. The input image 301 may be a normal SEM image or an oblique SEM image of a patterned substrate. In this embodiment, the term "normal image" or "normal SEM image" refers to a top-view image of the substrate capturing a first topology of the structure when viewed from above. In some embodiments, the normal image may be obtained via a measurement tool configured to guide an electron beam perpendicular to the substrate. The term "oblique image" or "oblique SEM image" refers to an angled view image of the substrate, capturing a second topology of the structure when viewed at an angle relative to the perpendicular to the substrate. In some embodiments, the oblique image may be obtained via a measurement tool configured to guide an electron beam at an angle relative to the substrate.

[0075] exist Figure 3 In this model, M0 is configured to output data representing the difference between two images of the same structure. Based on this difference data, the depth information of the structure can be determined. In some embodiments, model M0 generates disparity data 311, which can be converted into depth information 315 of the structure on the substrate. For example, disparity data 311 can be converted into depth information 315 by multiplying the magnitude of disparity data 311 by a transformation function (such as a scaling factor k). In some embodiments, the transformation function can be a linear function, a non-linear function, or a constant determined by comparing the result of applying the transformation function to one of the two images. Figure 3 In this disclosure, as an example, disparity data 311 is visually represented as a disparity map or a disparity image. In this disclosure, disparity data 311 refers to the difference in coordinates of similar features within two stereo images (e.g., two SEM images of the same structure). In some embodiments, the magnitude of the displacement vector in the disparity map is directly proportional to the depth of the structure.

[0076] To train model M0, disparity data 311 can be transformed into another image 321 corresponding to the input image 301 of model M0 via a transformation operation T1. Transformation operation T1 represents any transformation function that uses disparity data 311 to modify the tilted image 302. For example, the transformation function can be a combination operation or a convolution operation between disparity data 311 and tilted image 302, thereby producing another image that should correspond to the normal image.

[0077] In this example, disparity data 311 is combined with a tilted image 302 measured with a specified beam tilt (non-direction) to map the pixels of the tilted image 302 to a normal SEM image. In some embodiments, the tilted image 302 may be represented as a function mFurthermore, disparity data 311 (e.g., a mapping) can be represented as another function. φ A reconstructed image can be obtained by combining the parallax data 311 with the tilted image 302. For example, the reconstructed image is obtained from the function... m and φ The reconstructed image is represented by a function obtained from the combination of these functions. In some embodiments, the reconstructed image can be represented as... m φ , where the symbol This represents the combination of functions, for example m ( x ) φ ( x ) = m ( φx ), x This is a vector of the x and y coordinates of the image. If the estimated disparity data is accurate, the reconstructed image is expected to be highly similar to the input SEM image. For example, the reconstructed image is more than 95% similar to the normal SEM image input to model M0. If the reconstructed images are not similar, model M0 is modified or trained to make the reconstructed image similar to the input SEM image. Modification of model M0 may involve adjusting one or more parameters of model M0 (e.g., weights and biases) until a satisfactory reconstructed image is generated by the model. In some embodiments, the adjustment of one or more parameters is based on the difference between the reconstructed image and the input SEM image. In some embodiments, a performance function, which is the difference between the reconstructed image and the input SEM image, may be used to guide the adjustment of one or more parameters of model M0.

[0078] In some embodiments, training model M0 involves adjusting model parameters to minimize a performance function. At the end of the training process, model M0 is referred to as model M1 or the trained model M1. In some embodiments, the adjustment includes determining a gradient map of the performance function by differentiating the performance function with respect to one or more model parameters. The gradient map guides the adjustment of one or more model parameters in a direction that minimizes the performance function value or keeps the performance function value within a specified threshold. For example, the specified threshold could be that the difference between the performance function values ​​of the current iteration and subsequent iterations is less than 1%.

[0079] In some embodiments, the performance function includes similarity loss. Similarity loss The reconstructed image is compared to the input SEM image (e.g., represented as a function). fThe similarity loss function is calculated between multiple images of the structure, each obtained from a different perspective. For example, the performance function can be modified to include a loss function calculated as the sum of the similarities between the images in the multiple images and their corresponding reconstructed images.

[0080] In addition, the performance function may include another loss function. This other loss function The disparity is determined based on prior information relating to the disparity characteristics of paired stereo SEM images. Prior information includes, but is not limited to: disparity characterized as a piecewise smooth function, disparity characterized as a piecewise constant function, or disparity characterized as a function that allows abrupt changes at the edges of features in the normal image. For example, edges of features can be detected by applying a gradient operation to the image (e.g., the normal image). The gradient operation identifies abrupt changes in the slope of the intensity profile at one or more locations in the image (e.g., the normal image), such locations can be characterized as edges where disparity can abruptly change from one function type to another.

[0081] As previously described, a disparity function can be determined based on previous stereo images of one or more previously patterned substrates. For example, disparity can be a piecewise smooth function, where the derivative of the disparity is piecewise continuous. For instance, the disparity associated with a structure having non-vertical walls would be piecewise continuous. In another example, disparity can be a piecewise constant. For instance, the disparity associated with a structure having vertical walls would be piecewise constant. In yet another example, disparity can be a function with jumps at the edges of the structure within the image, the edges being detected based on the gradient of the intensity profile within the image. For instance, in the disparity associated with both structures having vertical and non-vertical walls, the jump locations are predetermined from the SEM image. Therefore, this method enables the incorporation of disparity data from previously patterned substrates in the form of a disparity function to train a machine learning model. The use of such a disparity function enables faster convergence or training of model M0, and more accurate results.

[0082] In some embodiments, the model M0 (e.g., a CNN) can be trained based on a performance function, which is expressed as the sum of a similarity loss function and another loss function associated with the previously patterned substrate (see equation below). In one example, the performance function... L The performance function is minimized during training by modifying one or more model parameters of the CNN. L As shown below:

[0083] According to this disclosure, reference Figure 4The inventors identified that, based on experiments, depth information of a patterned structure on a substrate can be inferred from the parallax between two or more SEM images captured at different tilt angles. For example, the experiments involved generating a depth map and feeding it into a Monte Carlo SEM simulator to compute denoised pairs of stereo SEM images: one stereo SEM image at 0 degrees to the normal (top view image), and another stereo SEM image at 5.7 degrees to the normal (tilted image). An example Monte Carlo simulator configured to generate SEM images is described in "L. van Kessel, and CW Hagen, Nebula: Monte Carlo simulator of electron–matter interaction The concept is discussed in "SoftwareX, Volume 12, 2020, 100605, ISSN 2352-7110". In the experiments, point sets were identified in two SEM images, and the associated disparity data was analyzed. For example, the point set corresponds to a step transition in the height profile of the structure, which results in a peak in the SEM signal. For the point set, the disparity data (e.g., displacement in the x-coordinate) for each point between the two SEM images was calculated. The height or depth of each point was plotted against the disparity, as shown below. Figure 4 As shown. Based on line 401 fitted between the disparity data and the height at the point set, it can be concluded that there is a strong correlation between disparity and the height of the structure between two SEM images.

[0084] In some embodiments, once the model M1 is trained, depth information prediction can be performed based on a single image (e.g., a SEM image) of any patterned substrate. Figure 6A The illustration shows how to estimate height or depth information associated with a structure using a single SEM image 351. Figure 6A In the model M1 (for example, such as...) Figure 3 (For training) The disparity data 355 is estimated using the normal SEM image 351 of the structure as input. The disparity data 355 is further converted into height data 360 of the structure patterned on the substrate by applying a transformation function k to the disparity data 355. For example, the magnitude of the estimated disparity data can be multiplied by a constant to generate depth information. Thus, tilting the image is not required to infer the height information of the structure.

[0085] Figure 5 This is an exemplary flowchart of a method 500 for determining a model that estimates depth information of the structure of a patterned substrate. In an example implementation, method 500 includes the following procedures P502, P504, P506, and P508, which will be discussed in detail below.

[0086] Process P502 involves obtaining a pair of images IM1 and IM2 of the structure of a patterned substrate. The pair of images includes a first image IM1 captured at a first angle relative to the patterned substrate, and a second image IM2 captured at a second angle different from the first angle. As an example, the first image IM1 captures top-view details of the structure, and the second image IM2 captures corner-view details of the same structure from another angle (e.g., an angle between 1° and 15° relative to the perpendicular to the substrate). An example of the first image IM1 could be normal image 301 (see...). Figure 3 ), and an example of the second image IM2 could be a tilted image 302 (see Figure 3 In some embodiments, the first image IM1 is a normal image associated with an electron beam guided perpendicular to the patterned substrate, and the second image IM2 is associated with an electron beam guided at an angle greater than 90° or less than 90° relative to the patterned substrate. In some embodiments, multiple pairs of images can be obtained and used as a training dataset. In some embodiments, the pairs of images obtained via a metrology tool include multiple pairs of SEM images of the patterned substrate. Each pair includes a first SEM image associated with a first electron beam tilt setting of the metrology tool and a second SEM image associated with a second electron beam tilt setting of the metrology tool. In some embodiments, each image in the pair is captured from a different SEM tool. For example, the first SEM image is captured by a first SEM tool configured to capture a normal image of the structure, and the second SEM image is captured by a second SEM tool configured to capture an image of the same structure at a second electron beam tilt setting.

[0087] Process P504 involves generating disparity data DD1 between the first image IM1 and the second image IM2 via a model M0 using the first image IM1 as input. Disparity data DD1 indicates depth information associated with the first image IM1. In some embodiments, disparity data DD1 includes the coordinate differences of similar features within the first image IM1 and the second image IM2. In some embodiments, after a training process, model M1 is considered a trained model of M0. In some embodiments, model M0 or M1 can be a machine learning model configured to predict disparity data using a single image of a substrate. For example, model M0 or M1 can be a convolutional neural network (CNN), a deep CNN, or other machine learning model.

[0088] Process P506 involves applying disparity data DD1 to a second image IM2 to generate a reconstructed image corresponding to the first image IM1. In some embodiments, the reconstructed image is generated by performing a combination operation between the disparity data DD1 and the second image IM2. For example, Figure 3 The figure illustrates a reconstructed image 321 generated based on disparity data (represented as a disparity map 311) and a tilted image 302.

[0089] Procedure P508 involves adjusting one or more parameters of model M1 based on a performance function such that the performance function is within a specified performance threshold. The performance function can be a function of the disparity data DD1, the reconstructed image, and the first image IM1. Model M1 is configured to generate data that can be converted into depth information that can be transformed into a structure of a patterned substrate.

[0090] In some embodiments, the performance function further includes a loss function computed based on disparity properties associated with paired stereo images of one or more previously patterned substrates and disparity data DD1 predicted by model M1. In one example, the disparity property may include disparity characterized as a piecewise smooth function, where the derivative of the disparity is piecewise continuous. In another example, the disparity property may include disparity as a piecewise constant. In yet another example, the disparity property may include disparity characterized as a function having abrupt changes at edges of structures within the image, the edges being detected based on gradients of intensity contours within the image. The above is relative to... Figure 3 Example performance functions were discussed. L .

[0091] In some embodiments, training is performed on multiple images of structures obtained at different angles. Accordingly, the performance function may include a loss function relating to the sum of similarities between the images in the multiple images and their corresponding reconstructed images.

[0092] In one example, the performance function is minimized during training by modifying one or more model parameters of the CNN. For example, the performance function L is calculated as the sum of the loss function related to the reconstructed image and the first image, and the disparity characteristics of the previous stereo image of the previously patterned substrate. For example, as previously stated, the performance function can be calculated as... .

[0093] In some embodiments, adjusting one or more parameters of model M0 is an iterative process, each iteration including determining a performance function based on disparity data DD1 and the reconstructed image. The iteration also includes determining whether the performance function is within a specified performance threshold; and in response to the performance function not being within the specified disparity threshold, adjusting one or more parameters of model M0 such that the performance function is within the specified performance threshold. The adjustment may be based on the gradient of the performance function with respect to one or more parameters. Once model M0 is trained, model M1 can be applied to determine depth information of any structure based on a single image (e.g., a normal SEM image) of any patterned substrate.

[0094] In some embodiments, method 500 may further include a step of applying a trained model M1 during measurement or inspection of the patterned substrate. In some embodiments, method 500 may include a process of obtaining a SEM image of the patterned substrate via a measurement tool at a first electron beam tilt setting of the measurement tool. The SEM image may be a normal image obtained by guiding an electron beam approximately perpendicular to the patterned substrate. The method may further include using the SEM image as input to execute model M1 to generate disparity data associated with the SEM image; and applying a transformation function (e.g., a linear function, a constant transformation factor, or a nonlinear function) to the disparity data to generate depth information of the structure in the SEM image. Method 500 includes determining physical properties of the structure of the patterned substrate based on the depth information. In some embodiments, physical properties may include the shape, size, or relative positioning of polygonal shapes at one or more characteristic depths of the structure.

[0095] In some embodiments, the processes described herein may be stored as instructions on a non-transitory computer-readable medium, which, when executed by one or more processors, cause the steps of the method to be performed. For example, the medium includes operations that include receiving a normal SEM image of a structure on a patterned substrate via a metrology tool (e.g., a SEM tool). The normal SEM image is associated with an electron beam directed perpendicular to the patterned substrate. Furthermore, the medium includes operations that use the normal SEM image to perform a model to determine depth information of a structure on the patterned substrate. As discussed herein, the model may be trained to estimate depth information based solely on a single SEM image and stored on the medium. For example, a model M1 (e.g., a CNN) may be used... Figure 5 The method is to train.

[0096] In some embodiments, the medium is further configured to determine the physical properties of the structure of the patterned substrate based on depth information. For example, physical properties include, but are not limited to, the shape, size, or relative positioning of polygonal shapes at one or more depths of the structure.

[0097] In some embodiments, the medium is also configured to identify defects in the structure based on physical properties, indicating that the structure does not meet design specifications. Based on the defects, parameters of one or more patterning processes (e.g., parameters related to resist or etching) can be adjusted to eliminate the defects in subsequent processes of the patterning process.

[0098] Figure 6B According to some embodiments, it is for use according to Figure 5A flowchart of method 600 for determining depth information of a structure formed on a substrate using a trained model. The depth information is determined based on scanning electron microscope (“SEM”) images of the structure. The depth information is determined using a convolutional neural network (CNN), which is trained to mimic the parallax effect present in paired stereo SEM images. Method 600 includes procedures P602, P604, and P606. Procedure P602 involves receiving an SEM image SEM1 of a structure patterned on a substrate via an SEM tool.

[0099] Process P604 involves inputting the SEM image SEM1 into CNN1 (an example of model M1 trained according to method 500) to predict disparity data 605 associated with the SEM image SEM1. In some embodiments, CNN1 can be based on... Figure 5 For example, CNN1 can be trained by: obtaining a stereo pair of SEM images of a patterned substrate via an SEM tool, the stereo pair including a first SEM image obtained under a first electron beam tilt setting of the SEM tool and a second SEM image obtained under a second electron beam tilt setting of the SEM tool; generating disparity data between the first SEM image and the second SEM image; combining the disparity data with the second SEM image to generate a reconstructed image of the first SEM image; and comparing the reconstructed image with the first SEM image.

[0100] Process P606 involves generating depth information 610 associated with the structure patterned on the substrate based on the predicted parallax data 605.

[0101] In some embodiments, depth information can also be used to identify structural defects by comparing the CDs of features at different depths and determining whether the CDs meet design specifications. In some embodiments, depth information can be used to determine one or more parameters of a patterning process (such as a resist process, an etching process, or other patterning-related processes) such that the 3D structure of the feature is within the desired design specifications.

[0102] In some embodiments, this document describes a method and system for semi-supervised training of a depth estimation model for a structure formed on a substrate and for using the trained model to predict depth data from a single SEM image. In some systems, to obtain depth information related to the structure, two images of the structure are obtained, the first image at the normal beam incidence and the second image using a tilted beam. However, adjusting the alignment is not only difficult and inaccurate but also time-consuming, negatively impacting manufacturing productivity.

[0103] This disclosure overcomes these problems by using machine learning models such as convolutional neural networks (“CNNs”), which take SEM images as input to predict depth data associated with the SEM images. However, the use of machine learning models presents additional technical problems, namely, the training data may be sparse and lack adequate diversity, leading to poor predictions. To overcome this problem, the machine learning model described herein is trained to predict the depth data of an input SEM image based on multiple simulated SEM images labeled with simulated depth data. For example, by using simulated depth data, the system can have complete control over quantity and diversity.

[0104] While using simulated SEM images overcomes the technical problem of sparse training data and provides reasonably high-quality depth information as model input, the depth information often still does not match the observed depth information. To overcome this second technical problem, the machine learning model described herein includes a second supervised calibration step. Specifically, the machine learning model is calibrated to scale the predicted depth data to correspond to the observed depth data of the actual SEM image, as determined using SEM tools. For example, to correct this problem, the system scales the predicted depth information to the depth information observed using SEM tools or other measurement tools, and uses the scaled predicted depth information as a new ground truth for subsequent training processes.

[0105] Figure 7 This is a flowchart of a method for generating a model 710 according to some embodiments, the model 710 being configured to estimate depth data of a structure on a patterned substrate. In some embodiments, method 700 is semi-supervised training of the depth estimation model 710 of the structure formed on the substrate based on SEM images of the structure. For example, in a semi-supervised training method, initially, the depth estimation model 710 is trained using simulation data, and additionally observed structural depth data is used to modify the depth data predicted by the model, and the modified depth data is used as a baseline to further fine-tune the depth estimation model 710. Method 700 will be discussed in further detail below with respect to processes P702, P704, P706, and P710.

[0106] Process P702 includes obtaining multiple simulated measurement images 703 of the structure via simulator 701. Each of the multiple simulated measurement images 703 is associated with depth data used by simulator 701. For example, the depth data used in simulator 701 (also referred to as simulator depth data) may be shape, size, sidewall angles, relative positioning of polygonal shapes, material, or depth-related parameters associated with each layer of the substrate. Simulator 701 is configured to perturb the simulator depth data to generate simulated measurement images. For example, the simulator depth data that can be adjusted includes, but is not limited to, the shape of the resist profile of the structure in the resist layer, and the sidewall angles of the resist profile. For each adjusted simulator depth data, a corresponding measurement image (e.g., an SEM image) can be generated.

[0107] In some embodiments, simulator 701 is a Monte Carlo simulator configured to generate a simulated measurement image 703 of the structure by varying depth-related parameters defined in the Monte Carlo simulator. An example Monte Carlo simulator configured to generate SEM images is described in "L. van Kessel, and CW Hagen, Nebula: Monte Carlo simulator of electron–matter interaction The Monte Carlo simulator is discussed in SoftwareX, Volume 12, 2020, 100605, ISSN 2352-7110. The Monte Carlo simulator is provided as an example, but this disclosure is not limited to a particular simulator.

[0108] Process P704 includes generating a model M70 based on multiple simulated measurement images 703 and corresponding simulated depth data. The model M70 is configured to predict depth data based on the input images.

[0109] In some embodiments, the generative model M70 is an iterative process. Each iteration includes predicting depth data, comparing the predicted data with simulator data, and generating a model based on the comparison. For example, process P704 includes inputting multiple simulated measurement images 703 into model M70 to predict depth data associated with each of the multiple simulated measurement images 703. The predicted depth data is compared with simulator depth data. Based on the comparison, the model parameters of model M70 are adjusted such that the predicted depth data is within a specified matching threshold of the simulator depth data. For example, the predicted depth data has a match of more than 90% with the simulator depth data. For example, the height profile of the predicted depth data is more than 90% similar to the height profile of the simulator depth data. Model M70 is also fine-tuned based on observation data, which is part of semi-supervised training.

[0110] Process P706 includes acquiring a captured image 707 of a patterned structure on a substrate and observation depth data 708. In some embodiments, the captured image 707 is acquired via an image capture tool, and the observation depth data 708 is acquired from one or more measurement tools. In some embodiments, the image capture tool is a SEM tool, and the captured image is an SEM image. In some embodiments, the observation depth data is acquired from a measurement tool. The measurement tool is one or more of the following: an optical measurement tool configured to measure the structure of the patterned substrate and extract depth information based on diffraction-based measurements of the patterned substrate; or an atomic force microscope (AFM). For example, the observation depth data 708 includes the height profile of the structure captured by an atomic force microscope (AFM) tool, or shape parameter data captured by an optical scattering measurement tool (e.g., Yieldstar). In some embodiments, the measurement tool is an optical measurement tool configured to measure the structure of the patterned substrate and extract depth information based on diffraction-based measurements of the patterned substrate.

[0111] In some embodiments, the observation depth data includes one-dimensional height data of the structure traced from the captured image 707. Similarly, simulator depth data may include one-dimensional height data determined from a simulated image of the structure on a patterned substrate. In some embodiments, the observation depth data includes two-dimensional height data of the structure traced from the captured image 707. Similarly, simulator depth data may include two-dimensional height data extracted from a simulated image of the structure on a patterned substrate. In another embodiment, the observation depth data includes shape parameters obtained from an optical metrology tool used to measure the structure of the patterned substrate. Similarly, shape parameters may be extracted from simulator depth data. As an example, one-dimensional height data includes the height profile of the structure along a cut line of the captured image 707 (e.g., Figure 8B The height profile 3DD1 is shown. In some embodiments, the two-dimensional height data includes the height of the structure along a first direction and a second direction on the captured image 707. In some embodiments, the shape parameters include one or more of the following: the top CD measured at the top of the structure, the bottom CD measured at the bottom of the structure, or the sidewall angles of the structure. For example, Figure 8B The figure shows the shape parameter data 3DD2.

[0112] Process P710 includes calibrating model M70 based on captured image 707 and observed depth data 708, such that the predicted depth data is within a specified matching threshold of observed depth data 708. After calibration, model M70 may be referred to as model 710, trained model 710, or calibrated model 710.

[0113] In some embodiments, calibrating model M70 is an iterative process. Each iteration includes adjusting the predicted depth data based on observed data to generate modified predicted data for training. For example, each iteration includes inputting captured image 707 into model M70 to predict depth data. The predicted depth data is adjusted by comparing the predicted depth data with observed depth data 708. Based on the adjusted predicted depth data, the model parameters of model M70 can be adjusted so that model M70 generates depth data within a matching threshold of observed depth data 708.

[0114] In some embodiments, adjusting the predicted depth data includes: extracting a one-dimensional height profile of the structure along a given direction from the predicted depth data; comparing the predicted height profile with a one-dimensional height profile of the observed depth data 708 of the structure along the given direction; and modifying the predicted height profile to match the height profile of the observed depth data 708 of the structure.

[0115] In some embodiments, adjusting the predicted depth data includes: extracting predicted shape parameters of the structure from the predicted depth data and extracting true shape parameters from the observed depth data 708; comparing the predicted shape parameters with the true shape parameters of the structure; and modifying the predicted shape parameters to match the true shape parameters.

[0116] In some embodiments, adjusting the predicted depth data includes: deriving the predicted average height of the structure from the predicted depth data of the structure, and deriving the true average height of the structure from the observed depth data 708; and scaling the predicted average height to match the true average height. For example, a scaling factor (e.g., Figure 8C The factor SF1 is calculated as the ratio of the average height calculated from the predicted depth data to the average height obtained from an optical scattering measurement tool (e.g., Yieldstar).

[0117] Figure 8A The illustration shows an example of generating an initial model M70 using simulated measurement images and simulated depth data (e.g., generated via a Monte Carlo SEM simulator). Figure 8A In this process, the convolutional neural network M70 (e.g., with encoder-decoder, U-net, or Res-net architecture) is trained on a sufficiently large dataset generated from a Monte Carlo SEM model. The advantage of using simulation data at this step is that it provides complete control over the quantity and diversity of training examples (e.g., SEM settings, feature geometry, and feature materials).

[0118] As an example, a synthetic or simulated SEM image SSEM1 generated by a SEM model and the corresponding simulator data SDEP can be used to train an initial CNN M70. In some embodiments, the CNN M70 predicts depth data, which is compared with the simulated data SDEP. Based on the comparison, one or more weights and biases of one or more layers of the CNN are adjusted so that the predicted data matches the simulated data SDEP. In some embodiments, the difference between the predicted depth data and the depth data extracted from the simulated data SDEP can be calculated. During the training of the CNN M70, in each iteration, the weights and biases are modified to reduce the difference between the predicted data and the simulated data. In some embodiments, the training process terminates when the difference cannot be further reduced in subsequent iterations. At the end of training, the model M70 is sufficiently trained to predict depth data based on the input SEM image. However, as mentioned above, the data predicted by the CNN M70 may not accurately correspond to the observed data. Therefore, as per [the previous point]... Figure 8B and Figure 8C The CNN M70 discussed was further fine-tuned.

[0119] Figure 8B The illustration shows examples of observed depth data obtained from different measurement tools (e.g., AFM, SEM, or Yieldstar), which can be used to fine-tune the CNN M70 model. In this example, a subset of paired experimental SEM images SEM80 and measurement depth data 3DD1 or 3DD2 (e.g., from the measurement tools Yieldstar or AFM) are generated. The experimental SEM image SEM80 is used as input to the CNN M70 to provide qualitatively plausible depth data 3DM, such as a height map. However, in some embodiments, the height map is predicted. Figure 3 DM may be highly correlated with observation. Figure 3 DD1 or 3DD2 do not match. For example, height mapping. Figure 3 DD1 is a 1D height map of the cut lines across the AFM data, and the height map... Figure 3 DD2 is a 2D height map obtained from an optical scattering measurement tool (e.g., Yieldstar). To correct for this, the predicted height map can be scaled to the observed depth data 3DD1 or 3DD2. The scaled predicted height map serves as a new ground truth for subsequent training procedures.

[0120] In some embodiments, the CNN M70 may be a layer, which may include, for example, convolutions using kernels modeled with the limitations of the AFM tool tip. In some embodiments, additional layers corresponding to the measurement tool may be augmented in the CNN M70. Thus, during fine-tuning, the model parameters associated with such a specific layer augmented in the CNN M70 can be adjusted. In some embodiments, the experimental dataset will typically be limited by the measurement tool (e.g., AFM or YS), so only a subset of the degrees of freedom (e.g., layers) of the CNN M70 is optimized, while the rest of the layers remain unchanged. In one embodiment, a correction bias in the latent space variables may be employed when training the CNN M70. For example, latent space variables z It can be calculated as follows: ,

[0121] In the above equation, Is with z Vectors of the same dimension. (Or its norm) can be used as a measure of the difference between a synthetically trained model (e.g., CNN M70) and a calibrated model (e.g., 710). Ideally, this distance will be relatively small. In another embodiment, the weights and biases of the final encoder layer are used as degrees of freedom.

[0122] Figure 8C The illustration shows an example adjustment to predicted depth data based on observational data from an optical measurement tool (e.g., Yieldstar), configured to measure a patterned substrate. In this example, the experimental SEM image SSEM2 is input into a CNN M70 to generate a qualitatively reasonable height map PDD2, but the average height does not match the observation height YSh provided by the optical tool (e.g., Yieldstar). To correct this, the predicted height map PDD2 can be scaled to the observation height YSh using a scaling factor SF1. For example, the scaling factor can be calculated as the ratio of the average height of the observation height YSh to the average height of the structure calculated from the predicted height map PDD2. Multiplying the predicted height map PDD2 by the scaling factor SF1 generates a scaled predicted height map SPD2, which is used as a new ground truth for subsequent training processes to generate a fully trained CNN 710 (e.g., ...). Figure 8B middle).

[0123] In some embodiments, the acquisition / setting experimental results demonstrate that the depth data predicted using the calibrated model 710 closely follows the programmed depth data used in the experiment. For example, the depth of an individual contact hole predicted by model 710 (acquisition depth) closely follows the depth programmed in the geometry used to generate the simulated SEM image (setting depth). Similarly, the sidewall angles of edges extracted from the simulated data closely follow the simulated SEM image. In some embodiments, the brightness and contrast settings in the simulated image can be varied during the training process of model 710, such that the predictions from the deep learning network are independent of the brightness and contrast settings.

[0124] The calibrated model 710 is capable of handling various physical effects that occur during SEM image capture or patterning processes. For example, such as Figures 9A to 9C As shown, even when the charging effect exists in the SEM image, model 710 is able to make good predictions.

[0125] Figure 9A This is an exemplary SEM image 902 that includes charging artifacts. For example, in SEM image 902 of linear spatial features, SEM charging artifacts are seen on the right side of the trench, where the SEM signal is reduced compared to the edge of the trench. Figure 9B The predicted height mapping diagram 904 is shown in the figure (see right side of the trench). When such a SEM image 902 with a charging effect is input into the initial model M70 (e.g., Figure 7 When the SEM intensity is reduced (in the middle), the predicted height map is affected by this reduction. To correct for this, as previously mentioned, fine-tuning or other calibration steps are employed during the training of model M70. In such a calibration step, the predicted data is adjusted to assume that the lines in the new training set have flat surfaces. When such adjusted predicted data is used to generate a calibrated model 710 (e.g., Figure 7 In the middle (of the process), the predicted height map (using model 701) is no longer affected by charging, but rather... Figure 9C As shown in the predicted height mapping figure 906, there is indeed a flat surface on top of the line. This example serves as a proof of principle that, in some embodiments, deep learning networks (e.g., CNNs) are taught to identify artifacts not present in the simulated dataset and to perform nonlinear corrections based on the observation data.

[0126] Figure 10A It is achieved by using a trained model 710 ( Figure 7The flowchart describes a method for estimating depth data using a single SEM image as input. Method 800 includes processes P802 and P804. In some embodiments, process P802 includes receiving an SEM image SEM1 of a structure patterned on a substrate. In some embodiments, process P804 includes predicting depth data 810 associated with the SEM image via a CNN (e.g., model 710) using the SEM image as input. As previously described, the CNN (e.g., model 710) can be trained to predict the depth data of the input SEM image based on multiple simulated SEM images labeled with simulated depth data; and further calibrated using scaled predicted depth data, wherein the scaled predicted data corresponds to observed depth data, as determined using SEM tools, of the captured SEM image of the structure patterned on the substrate.

[0127] For example, training a CNN 710 includes: receiving captured SEM images and observation depth data via an SEM tool, the observation depth data being associated with captured SEM images of a structure patterned on a substrate; inputting the captured SEM images into the model to predict the depth data; adjusting the predicted depth data by comparing the predicted depth data with the observation depth data; and tuning the model based on the adjusted predicted depth data so that the predicted depth data is within a specified matching threshold of the observation depth data.

[0128] In some embodiments, the estimated depth data includes physical characteristics of the structure of the patterned substrate. For example, physical characteristics include shape (e.g., resist profile), size, sidewall angles (e.g., angles of the resist profile), or the relative positioning of polygonal shapes relative to each other at one or more depths of features of the structure.

[0129] Figure 10B The illustration shows an example of using a trained model 710 and taking a SEM image 821 as input to predict depth data 825 (e.g., a height map). As shown, model 710 predicts 3D information based on 2D data in the SEM image.

[0130] In some embodiments, instead of using a SEM simulator to train the depth estimation model using paired data, a calibrated deterministic or stochastic process model (e.g., a resist model) can be used to generate training data. Where the accuracy of the process model may be insufficient, the process model can be fine-tuned using a limited set of experimental depth data (e.g., from AFM or Yieldstar). In some embodiments, depending on the type of process model (deterministic or stochastic), two separate approaches are described herein.

[0131] In some embodiments, using a resist model instead of a SEM model to generate training data (e.g., depth maps and corresponding SEM images) has several advantages. For example, the SEM image corresponding to the simulated resist profile is an experimental image, while the SEM image generated by the SEM model is synthetic. Regardless of whether an SEM simulator is included, the process model (e.g., the resist model) can be well calibrated for wafer data.

[0132] Figure 11 This is a flowchart of a method 1100 for generating a model configured to estimate depth data of a structure on a patterned substrate. For example, the depth data of the structure includes one or more features, such as lines, stripes, contact holes, other features, or combinations of features. In some embodiments, the model is trained using training data generated as pairs of SEM images and corresponding simulated 3D contours that may be formed on the substrate. The simulated contours may be generated via a calibrated process model (e.g., a calibrated resist model). In some embodiments, calibration is performed to satisfy critical size criteria associated with the structure. In some embodiments, process model calibration may involve comparing the height of the 3D simulated geometry with heights extracted from atomic force microscopy (AFM) data or optical measurement data. In some embodiments, the model is a CNN trained using a generative adversarial network (GAN), an encoder-decoder network, or other machine learning-related training methods. For example, the model includes a first model (e.g., a generator model or an encoder model) and a second model (e.g., a discriminator model or a decoder model) trained together. In some embodiments, both the first and second models may be CNN models. An example implementation of method 1100 includes a process P1102 for obtaining training data and a process P1104 for generating a model based on the training data. Processes P1102 and P1104 are discussed in detail below.

[0133] Process P1102 includes obtaining (i) multiple SEM images 1101 of structure associated with programming variation PV1 in the mask pattern; and (ii) a simulation profile 1103 of structure based on programming variation PV1, wherein each SEM image in the multiple SEM images 1101 is paired with a simulation profile corresponding to the programming variation in the mask pattern.

[0134] In some embodiments, the programming variation PV1 includes variation among one or more of the following: auxiliary feature variation associated with the mask pattern, main feature variation associated with the mask pattern, or resist coating thickness variation. In some embodiments, the auxiliary feature variation includes modifying the size of an auxiliary feature of the mask pattern, the distance of the auxiliary feature to the main feature of the mask pattern, or both. In one example, the first programming variation may be a change in the size of an auxiliary feature, and a mask including the first programming variation may be fabricated. Using a mask with the first programming variation, a substrate may be patterned (e.g., via a photolithography apparatus). Furthermore, an SEM image of the patterned substrate may be captured via an SEM tool. Thus, the captured SEM image includes features associated with the first programming variation in the mask. In another example, the second programming variation in the mask pattern may be a modification of the shape of a main feature of the mask pattern. The substrate may be patterned using a mask with the second programming variation, and another SEM image of the patterned substrate may be captured such that the SEM image includes variation associated with the second programming variation. Such SEM images, corresponding to the first, second, and third programming variations PV1, can be obtained and included in the training data.

[0135] In some embodiments, the training data may further include a simulated profile 1103 of the structure. In some embodiments, the simulated profile 1103 may be a 3D profile (e.g., a resist profile) that may be formed on a substrate. Such a 3D profile may be generated by a process model (e.g., a resist model). In some embodiments, the simulated profile 1103 is generated by a calibrated deterministic process model (DPM) associated with a patterning process using a programming variation PV1 in a mask pattern. For example, a first programming variation (e.g., a change in auxiliary feature size), a second programming variation (e.g., a change in the shape of a main feature), or other programming variations may be input into the calibrated process model to generate the simulated profile 1103. Each of these simulated profiles 1103 is a 3D profile and may include 3D profile variations caused by a corresponding programming variation in the mask pattern. Thus, the SEM image 1101 and the simulated profile 1103 can be paired according to the programming variation PV1.

[0136] In some embodiments, the calibrated deterministic process model (DPM) is a process model calibrated using inspection data (also known as wafer data) of a patterned substrate, such that the simulated process model generates a simulated 3D profile of the structure within a desired range of the measured critical dimensions (CD) of the structure on the patterned substrate. In some embodiments, the deterministic process model (DPM) may not be calibrated to satisfy local CD uniformity, line edge roughness (LER), linewidth roughness (LWR) related to line space features, contact edge roughness related to contact hole features, or random variations associated with the structure. In other words, the calibrated deterministic process model (DPM) can simulate a profile of the structure that meets CD requirements. In another embodiment, the process model may be calibrated to satisfy LCDU, LER, LWR, or other random variations associated with the physical properties of the structure.

[0137] Process P1104 includes generating a model M110 based on multiple SEM images 1101 paired with the corresponding simulated contour 1103 to estimate the depth data of the structure, such that the estimated depth data is within an acceptable threshold of the depth data associated with the simulated contour 1103. For example, model M110 can be trained such that the estimated height map is more than 90% similar to the height map extracted from the simulated contour 1103.

[0138] In some embodiments, the generated depth data may include height data of the structure, shape parameters of the structure, voxel mapping of overhang features, or other depth characteristics of the structure. In some embodiments, shape parameters include one or more of the following: top CD measured at the top of the structure, bottom CD measured at the bottom of the structure, or sidewall angles of the structure.

[0139] In some embodiments, the generation of model M110 can be based on a generative adversarial network (GAN) architecture, an encoder-decoder network, ResNet, or other machine learning architectures. For example, a conditional GAN ​​learns a mapping from observed image x (e.g., SEM image 1101) and random noise vector z to output y (e.g., simulated contour 1103), denoted as G:{x, z}→y. In some embodiments, the generator G is trained to produce outputs that cannot be distinguished from reference images (e.g., “real” images) by an adversarially trained discriminator D, which is trained to perform the classification of generated images as “fakes” as well as possible. For example, in GAN training, G attempts to minimize the objective against the adversarial D, which attempts to maximize the objective. The generator can be an encoder-decoder network or a U-net.

[0140] In some embodiments, generating model M110 includes: training a first model (e.g., CNN1) in conjunction with a second model (e.g., CNN2), such that the first model generates depth data using SEM images from multiple images as input, and the second model classifies the generated depth data into a first category (e.g., real) or a second category (e.g., fake) based on pairs of simulated contours. In some embodiments, the first and second models are convolutional neural networks (CNNs) or deep CNNs. After the training process, the first model (e.g., CNN1) can be used as model M110 to generate depth data for any input SEM image.

[0141] In some embodiments, generating model M110 includes: inputting SEM images from a plurality of SEM images 1101 into a first model; estimating depth data for the input SEM images by executing the first model; classifying the estimated depth data into a first category (e.g., real) or a second category (e.g., fake) via a second model using a simulated contour 1103 as a reference, the first category indicating that the estimated depth data corresponds to the reference, and the second category indicating that the estimated depth data does not correspond to the reference; and updating the model parameters of both the first and second models such that the first model estimates the depth data and the second model classifies the estimated depth into the first category. In some embodiments, the model parameters are the weights and biases of one or more layers of a CNN or DCNN.

[0142] Figure 12 The illustration shows an exemplary training of model 1205 using an encoder-decoder architecture and training data, including paired SEM images 1202 and simulated contour data 1210. The simulated contour 1210 here is merely an example and may not correspond to the exact same mask pattern as the SEM images; the simulated contour 1210 is also uncalibrated. In some embodiments, model 1205 may be trained using a GAN architecture. In some embodiments, the simulated contour data 1210 represents an average 3D resist contour 1210, which describes the average 3D behavior of the examined features of the patterned structure (e.g., the features could be an array of contact holes). In the present case, a deterministic resist model (an example of a calibrated deterministic process model DPM) can be used to generate the simulated 3D resist contour 1210 of the structure.

[0143] To introduce local contour variations for training model 1205, programming variations (e.g., sub-resolution auxiliary feature (SRAF) variations) in the mask pattern within the field of view can be performed to induce variations in the feature contours (e.g., contact hole contours) on the substrate. In some embodiments, programming variations refer to physical changes made to the mask pattern. Such physical changes to the mask pattern may cause changes to the patterned 3D contours on the physical substrate. Thus, for each programming variation, a corresponding 3D contour can be obtained. In some embodiments, the programming variations and the data generated therefrom are used as training data for training the model discussed herein. In some embodiments, programming variations include changes to the size of the SRAF, its distance to the master pattern, or both, which will affect the CD and the 3D contours. Based on such programming variations, a patterned substrate can be generated and an SEM image of the patterned substrate is captured. In some embodiments, programming variations in the mask pattern are used to generate training data and are different from variations in the mask pattern performed to improve the lithography process (e.g., variations related to OPC).

[0144] In some embodiments, programming variations in the mask pattern can be used during the simulation of a photolithography process configured to generate a simulated 3D contour 1210. For example, mask pattern data (e.g., in GDS format) can be modified according to the programming variations, and the corresponding mask data (e.g., modified GDS) can be employed in a simulation using a deterministic model DPM to generate the simulated 3D contour. For example, as described above, programming variations implemented on the physical mask (such as changes in the size, shape, distance to the master pattern, etc. of the SRAF) can be used in the simulator to generate the corresponding simulated 3D contour. Thus, the correspondence between observed data and simulated data can be established based on the programming variations. In one example, a convolutional neural network (CNN) can be trained to directly map SEM data 1202 to the simulated 3D resist contour 1210. It is understood that the ADI image is for illustrative purposes only.

[0145] Return to reference Figure 11 In some embodiments, method 1100 may further include a process P1110 for applying a trained model 1110. For example, applying model 1110 includes: receiving an SEM image of a structure patterned on a substrate via an SEM tool; and determining depth data associated with the SEM image via a model using the received SEM image as input.

[0146] In some embodiments, the physical properties of the structure of the patterned substrate can be determined based on depth data. Physical properties may include the shape, size, or relative positioning of polygonal shapes at one or more depths of features of the structure relative to each other. In some embodiments, physical properties may be compared with desired physical properties to identify any defects in the structure. Defects may indicate that the structure does not meet design specifications (e.g., CD is within the desired CD limits). Based on defects, one or more parameters of the patterning process (e.g., dosage, focus, resist properties, etc.) may be adjusted to eliminate defects in subsequent processes of the patterning process.

[0147] Figure 13 This is a flowchart of method 1300 for generating model M130, which is configured to estimate depth data of a structure on a patterned substrate. In some embodiments, the model is trained using training data, which includes multiple unpaired SEM images and multiple simulated contours. For example, a particular SEM image is not paired with a particular simulated contour. In some embodiments, each of the multiple SEM images may correspond to one or more simulated contours. In this method, the simulated contours are generated via a calibrated process model (e.g., a calibrated stochastic model), which is configured to account for stochastic variations in the physical properties associated with the structure. In some embodiments, method 1300 may include a process P1302 for obtaining training data and a process P1304 for generating a model for estimating depth data. Processes P1302 and P1304 are discussed in further detail below.

[0148] Process P1302 includes obtaining (i) multiple SEM images 1301 of the structure, (ii) a simulated profile 1303 of the structure, and (iii) key performance indicators (KPIs) associated with the simulated profile 1303. In some embodiments, the multiple SEM images may be obtained from an image capture tool, such as a SEM tool, configured to capture images of the structure on a patterned substrate. The SEM images may be 2D images, such as top-view SEM images. In some embodiments, the simulated profile 1303 of the structure may be a 3D profile of the structure generated via a calibrated process model (SPM) associated with the patterning process. In some embodiments, the multiple SEM images 1301 and the simulated profile 1303 are not paired.

[0149] In some embodiments, the calibrated process model (SPM) is a process model calibrated to generate a 3D profile of a structure such that the 3D structure meets one or more key performance indicators (KPIs) extracted from the 3D structure. In some embodiments, the KPIs may include, but are not limited to, critical dimensions (CD) of the structure, local CD uniformity (LCDU) associated with the structure, line edge roughness (LER) associated with the structure, defect rate associated with the structure, line width roughness (LWR) associated with the line space pattern, contact edge roughness associated with contact holes, random edge placement error (SEPE) associated with the structure, other random variations associated with the geometry of the structure, or combinations thereof. In some embodiments, calibration may be based on a comparison between wafer data under test and model generation data.

[0150] Process P1304 includes generating a model M130 based on (i) multiple SEM images 1301, (ii) a simulated profile 1303, and (iii) KPIs to estimate depth data of the structure such that the KPIs associated with the estimated depth data are within an acceptable threshold for the KPIs associated with the simulated profile 1303. In some embodiments, the estimated depth data includes at least one of the following: height data of the structure, shape parameters of the structure, or a voxel map related to the overhanging structure. In some embodiments, the shape parameters include one or more of the following: a top CD measured at the top of the structure, a bottom CD measured at the bottom of the structure, or sidewall angles of the structure.

[0151] In some embodiments, the generative model M130 may be based on a generative adversarial network, an encoder-decoder network, or other machine learning-related networks. For example, generating model M130 includes: training a first model (e.g., Figure 14 In G1) and the second model (e.g., Figure 14 (DS in the example). In some embodiments, for example, referring to a GAN architecture, the first model may be a generator model, and the second model may be a discriminator model. Training of the first model is performed in conjunction with the second model, such that the first model generates depth data using SEM images from multiple SEM images as input. The second model uses a simulated contour 1303 and simulated KPI reference data associated with the simulated contour 1303 to classify the generated depth data into a first category (e.g., real) or a second category (e.g., fake).

[0152] In some embodiments, model generation is an iterative process. Each iteration includes: inputting SEM images from a plurality of SEM images 1301 into a first model; using the first model, estimating depth data for the input SEM images; extracting KPIs from the estimated depth data; and via a second model, classifying the estimated depth data into a first category or a second category using (i) the extracted KPIs and (ii) a plurality of simulated contours 1303 and their simulated KPIs as references. In some embodiments, the first category indicates that the estimated depth data corresponds to a reference, and the second category indicates that the estimated depth data does not correspond to a reference. Furthermore, the model parameters of both the first and second models can be updated such that the first model estimates the depth data and the second model classifies the estimated depth into the first category (e.g., true).

[0153] In some embodiments, for example, using cycle-GAN training, the first model (e.g., Figure 14 G1 can also be combined with a third model (e.g., Figure 14 Another generator model (G2) is further trained. A third model can be configured to generate SEM images from the estimated depth data. In other words, the first and third models are cycle-consistent. For example, the first and third models are trained such that a loss function (e.g., ...) is used... Figure 14 (The difference between images 1402 and 1406 in the image), the input of the first model is matched with the output of the third model. It should be understood that other forms of cycle-GAN training can be used, for example, via domain adaptation as further described in WO 2023 / 083559A1, which is incorporated herein by reference in its entirety.

[0154] The training of the first and third models can be an iterative process. Each iteration includes: inputting SEM images from a plurality of SEM images 1301 into the first model; using the first model to estimate depth data for the input SEM images; generating a predicted SEM image via the third model using the estimated depth data as input; and updating the model parameters of both the first and third models such that the difference between the input SEM image and the predicted SEM image is within a specified difference threshold. In some embodiments, the first, second, and third models are convolutional neural networks (CNNs) or deep CNNs. In some embodiments, the model parameters are the weights and biases of one or more layers of a CNN or a DCNN.

[0155] Figure 14 The illustration shows an exemplary determination of a model for estimating depth data according to method 1300. Figure 14 In this embodiment, model G1 can be trained according to method 1300 to generate model M130. In this embodiment, a stochastic process model (e.g., a stochastic resist model SPM) can be employed, which eliminates the need for specific target profile variation (e.g., programming variation). According to one example, a one-to-one correlation is impossible between the CH generated by the stochastic resist model and the CH formed on the wafer. In some embodiments, to circumvent this problem, a cycle-consistent generative adversarial network (CGN) can be employed to learn unpaired image-to-image translations between SEM image 1402, simulated depth data 1414, and predicted SEM image 1406 by minimizing a loss function. In some embodiments, the illustrated 3D profiles are merely examples and may not correspond to actual calibrated profiles for the illustrated SEM images. For example, the loss function could be the difference between SEM image 1402 and predicted SEM image 1406. An example of cycle-GAN is discussed in reference "J. Zhu, T. Park, P. Isola and AA Efros, Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks, 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017, pp. 2242-2251", which is incorporated herein by reference in its entirety.

[0156] refer to Figure 14 Measurement or inspection data for the CH array (e.g., SEM 1402) can be obtained from a measurement tool (e.g., SEM) by capturing images or inspecting a patterned substrate. In some embodiments, multiple CH arrays are obtained from ADI for multiple cases, such as using a focal exposure matrix (FEM).

[0157] In some embodiments, a physical depth random resist model (not shown) can be calibrated such that the random model can well predict the CD distribution, LCDU, LER, LWR, defect rate, or other random variations under the same conditions, as well as the contrast profile under the same conditions during planar exposure. The random model can be used to generate simulation profiles with multiple examples of CD and resist high variation (e.g., due to half-open CH), and the simulation profiles are similar to inspection data of a patterned substrate.

[0158] Furthermore, the network (e.g., a GAN) is trained using a dataset of unpaired deep resist outlines 1414 and SEM images 1402. For example, the GAN network includes a first model G1, a second model DS, and a third model G2. In some embodiments, the GAN network is trained to learn an efficient way to transform SEM image 1402 into a true deep resist outline 1404 and then back to a true SEM image 1406. For example, the prediction from SEM image 1402 to deep resist outline 1404 is performed using a trained model G1.

[0159] In the current training example, SEM image 1402 captured by a SEM tool can be input into a first model G1 (e.g., a generator of a GAN) to generate depth data 1404. Depth data 1404 can be represented as a 3D contour of a structure that may be formed on a substrate. For example, depth data 1404 could be a resist contour formed on a resist layer of the substrate. In some embodiments, key performance indicators (KPIs) (e.g., CD, LCDU, LER, LWR) can be extracted from depth data 1404. In some embodiments, depth data 1404 and KPIs can be input into a second model DS (e.g., a discriminator of a GAN). In some embodiments, reference contour 1414 can be generated using a stochastic process model, and reference KPIs can be extracted from reference contour 1414.

[0160] Depth data 1404 and relevant KPIs are input into a second model DS, which is configured to classify the depth data 1404 into a first category (e.g., real) or a second category (e.g., fake) based on reference KPIs and a reference simulation profile 1414. In some embodiments, the first category indicates that the depth data 1404 is real or similar to the reference profile 1414, while the second category indicates that the depth data 1404 is not real or dissimilar to the reference profile 1414.

[0161] In some embodiments, a first model G1 and a second model DS are trained to improve each other. In this way, the first model G1 gradually generates such realistic depth data that the second model DS classifies the depth data generated by the model as belonging to a first category (e.g., real).

[0162] Alternatively or additionally, the first model G1 and the second model DS are further trained in collaboration with a third model G2, which uses depth data 1404 as input to generate a predicted SEM image 1406. During training, the model parameters of the first model G1 and the third model G2 are adjusted to reduce or minimize the difference between the predicted SEM image 1406 and the real SEM image 1402. Therefore, the third model G2 can generate a realistic SEM image 1406 for any depth data. In some embodiments, the model parameters of the second model DS can also be adjusted because modifying G1 causes a change in G1's predictions, which may in turn affect the classification by the second model DS. In other words, when the third model is trained together, it can generate realistic depth data, which is further validated by generating equally realistic predicted SEM images (similar to the real SEM). Thus, according to some embodiments, training data may not need to be paired.

[0163] Return to reference Figure 13 In some embodiments, method 1300 may further include a process P1310 for applying a trained model 1310. For example, applying the model includes: receiving an SEM image of a patterned structure on a substrate via an SEM tool; and determining depth data associated with the SEM image using the received SEM image as input via the model. In some embodiments, the physical properties of the structure of the patterned substrate may be determined based on the depth data. Physical properties include the shape, size, or relative positioning of polygonal shapes at one or more depths of features of the structure.

[0164] In some embodiments, defects in a structure can be detected by comparing physical properties to a desired range. In some embodiments, KPIs associated with the physical properties can be compared to a desired KPI range to determine the defects. Thus, defects can indicate that the structure does not meet design or performance specifications. In some embodiments, based on the defects, parameters of one or more patterning processes can be adjusted to eliminate the defects in subsequent processes of the patterning process.

[0165] Figure 15This is a flowchart of method 1500, which applies model 1110 or model 1310 trained according to method 1100 or 1300 discussed herein. In one example, according to some embodiments, method 1500 for estimating depth data of a structure formed on a substrate uses a convolutional neural network (“CNN”), configured to estimate depth data based on scanning electron microscope (“SEM”) images of the structure. For example, the CNN can be trained according to 1100 or 1300. Example implementations of method 1500 may include procedures P1502 and P1504, which are further discussed below.

[0166] Process P1502 includes receiving an SEM image 1501 of a structure on a patterned substrate. For example, a normal SEM image can be captured by an SEM tool by projecting an electron beam perpendicular to the substrate. Alternatively, an optical image of the substrate can be captured via an optical tool and received in process P1502.

[0167] Process P1504 includes estimating depth data 1510 associated with the structure of the patterned substrate via a CNN, using SEM images 1501 as input. In some embodiments, the CNN can be trained using training data, which includes multiple SEM images of the structure and multiple simulated contours of the structure. In some embodiments, the simulated contours can be generated from a process model associated with the patterning process.

[0168] In some embodiments, a CNN can be a combination of a generator model (e.g., a first model or CNN1) and a discriminator model (e.g., a second model or CNN2) trained together (e.g., as per [reference to...]). Figure 11 (As discussed). For example, CNN training is an iterative process, each iteration including: obtaining multiple SEM images and multiple simulated contours, each SEM image paired with a simulated contour; inputting the SEM images from the multiple SEM images into a generator model; using the generator model, estimating depth data for the input SEM images; classifying the estimated depth data into a first category or a second category via a discriminator model, using the simulated contours paired with the input SEM images as references, where the first category indicates that the estimated depth data corresponds to the reference, and the second category indicates that the estimated depth data does not correspond to the reference; and updating the model parameters of both the generator model and the discriminator model so that the generator model estimates the depth data and the discriminator model classifies the estimated depth into the first category.

[0169] In some embodiments, a CNN can be a combination of a generator model (e.g., a first model or CNN1) and a discriminator model (e.g., a second model or CNN2) trained together (e.g., as opposed to...). Figure 13(As discussed). For example, CNN training is an iterative process; each iteration includes: inputting SEM images from multiple SEM images into a generator model; using the generator model, estimating depth data for the input SEM images; extracting key performance indicators (KPIs) associated with the structure based on the structure-associated intensity values ​​in the estimated depth data; classifying the estimated depth data into a first category and a second category via a discriminator model using (i) the extracted KPIs and (ii) multiple simulated contours and their simulated KPIs as references, where the first category indicates that the estimated depth data corresponds to the reference and the second category indicates that the estimated depth data does not correspond to the reference; and updating the model parameters of both the generator model and the discriminator model so that the generator model estimates the depth data and the discriminator model classifies the estimated depth into the first category.

[0170] In some embodiments, the CNN may also be trained in conjunction with a second generator model (e.g., CNN3), which is configured to generate SEM images from the estimated depth data. Training further includes: inputting SEM images from a plurality of SEM images into the CNN; using the CNN to estimate depth data for the input SEM images; generating a predicted SEM image via the second generator model using the estimated depth data as input; and updating the model parameters of both the CNN and the second generator model such that the difference between the input SEM image and the predicted SEM image is within a specified difference threshold.

[0171] In some embodiments, the process model is a calibrated deterministic process model (DPM) calibrated such that the critical dimension (CD) of the simulated profile of the structure is within a CD threshold of the observation data associated with the patterned substrate, but does not necessarily satisfy the stochastic variation specification associated with the observation data. In some embodiments, the process model is a calibrated stochastic process model calibrated such that key performance indicators (KPIs) extracted from the depth data generated by the model are within a specified threshold of the observation KPIs extracted from inspection data associated with the patterned substrate. In some embodiments, the inspection data is obtained from multiple features of the patterned substrate, which are formed using dose ranges and focus conditions. In some embodiments, the KPIs include one or more of the following: the CD of the structure, the local CD uniformity (LCDU) associated with the structure, the line edge roughness (LER) associated with the structure, the defect rate associated with the structure, and other KPIs indicating the performance of the patterning process.

[0172] As previously mentioned, depth information can also be used to identify structural defects by comparing the CDs of features at different depths and determining whether the CDs meet design specifications. In some embodiments, depth information can be used to determine one or more parameters of a patterning process (such as resist processing, etching processing, or other patterning-related processes) such that the 3D structure of the feature is within the desired design specifications.

[0173] In some embodiments, the base model (e.g., an untrained model) and the trained model can be machine learning models, which include weights and biases as model parameters. During the training process, the weights and biases of the base model are continuously adjusted based on the training data. At the end of training, the base model is referred to as the trained model. In some embodiments, the trained model is a convolutional neural network (e.g., a CNN) or a deep convolutional network (DCNN). The model parameters include the weights and biases of one or more layers of the deep convolutional network.

[0174] Without limiting the scope of this disclosure, examples of applications of supervised machine learning algorithms are described below.

[0175] Supervised learning is a machine learning task that infers a function from labeled training data. Training data consists of a set of training examples. In supervised learning, each example is a pair with an input object (usually a vector) and a desired output value (also called a supervision signal). Supervised learning algorithms analyze the training data and produce an inferred function that can be used to map new examples. An optimal scenario will allow the algorithm to correctly determine class labels for unseen instances. This requires the learning algorithm to generalize from the training data to unseen cases in a "reasonable" way.

[0176] Given form A set of N training examples, such that It is the feature vector of the i-th example, and It is its label (i.e., class) that the learning algorithm seeks the function. Let X be the input space and Y be the output space. A feature vector is an n-dimensional vector representing the numerical features of an object. Many algorithms in machine learning require numerical representations of objects because such representations facilitate processing and statistical analysis. When representing an image, feature values ​​might correspond to the pixels of the image; when representing text, feature values ​​might correspond to the frequency of occurrence of items. The vector space associated with these vectors is usually called the feature space. A function g is an element in some possible function space G, which is usually called the hypothesis space. Sometimes, for convenience, a scoring function can be used. Let g be a variable that is limited to returning the y-value that gives the highest score: Let F denote the space of the scoring functions.

[0177] Although G and F can be any function space, many learning algorithms are probabilistic models, where g employs a conditional probability model. In the form of, or f adopts a joint probability model In the form of [formula missing]. For example, Naive Bayes and Linear Discriminant Analysis are joint probability models, while Logistic Regression is a conditional probability model.

[0178] There are two fundamental methods for choosing between g and f: empirical risk minimization and structural risk minimization. Empirical risk minimization seeks a function that best fits the training data. Structural risk minimization includes a penalty function that controls for the bias / variance tradeoff.

[0179] In both cases, it is assumed that the training set has samples with independent and identically distributed pairs. To measure how well a function fits the training data, the loss function... Limited. For training examples Predicted value The loss is .

[0180] Risk of function g The expected loss is constrained to g. This can be estimated based on the training data. .

[0181] Exemplary models of supervised learning include decision trees, ensemble models (bagging, boosting, random forests), k-NN, linear regression, Naive Bayes, neural networks, logistic regression, perceptrons, support vector machines (SVM), relevance vector machines (RVM), and deep learning.

[0182] SVM is an example of a supervised learning model that analyzes data and identifies patterns, and can be used for classification and regression analysis. Given a training set of examples, each labeled as belonging to one of two classes, the SVM training algorithm builds a model that assigns new examples to one class or the other, making it a non-probabilistic binary linear classifier. The SVM model represents examples as points in space, mapped such that examples of different classes are separated by sharp gaps as wide as possible. New examples are then mapped into the same space, and their class is predicted based on which side of the gap they fall on.

[0183] In addition to performing linear classification, SVMs can also utilize so-called kernel methods to efficiently perform non-linear classification, implicitly mapping their inputs to a high-dimensional feature space.

[0184] Kernel methods involve a user-specified kernel, which is a similarity function between pairs of data points in the original representation. Kernel methods are named for the kernel function they use, which allows them to operate in a high-dimensional implicit feature space without computing the coordinates of the data in that space; instead, they only need to compute the inner product between all pairs of data in the feature space. This operation is generally computationally cheaper than the explicit computation of coordinates. This method is known as the "kernel trick."

[0185] The effectiveness of SVM depends on the choice of kernel, kernel parameters, and soft margin parameter C. A common choice is the Gaussian kernel, which has a single parameter γ. The optimal combination of C and γ is usually achieved through grid search (also known as "parameter scan"), utilizing exponentially growing sequences of C and γ (e.g.: ; Use ) to choose.

[0186] Grid search is an exhaustive search of a manually specified subset of the hyperparameter space of a learning algorithm. Grid search algorithms are guided by performance metrics, which are typically measured by cross-validation on the training set or by evaluation on a reserved validation set.

[0187] Each combination of parameter selections can be checked using cross-validation, and the parameter with the best cross-validation accuracy can be selected.

[0188] Cross-validation (sometimes called rotation estimation) is a model validation technique used to evaluate how the results of a statistical analysis will generalize to independent datasets. It is primarily used in scenarios where the goal is prediction and one wants to estimate the accuracy of a predictive model in practice. In prediction problems, the model is typically given a known dataset for training (training dataset) and an unknown dataset (or data seen for the first time) for testing the model (test dataset). The goal of cross-validation is to constrain the dataset used to "test" the model during the training phase (i.e., the validation dataset) to limit problems such as overfitting and to gain insights into how the model will generalize to independent datasets (i.e., unknown datasets, such as datasets from real-world problems). A round of cross-validation involves splitting a sample of data into complementary subsets, performing analysis on one subset (called the training set), and validating the analysis on the other subset (called the validation or test set). To reduce variability, multiple rounds of cross-validation are performed using different partitions, and the validation results from each round are averaged.

[0189] The final model, which can be used for testing and classifying new data, is then trained on the entire training set using selected parameters.

[0190] Another example of supervised learning is regression. Regression infers a relationship between a dependent variable and one or more independent variables based on the set of values ​​for the dependent variable and their corresponding values. Regression can estimate the conditional expectation of the dependent variable given the independent variables. The inferred relationship can be called a regression function. The inferred relationship can be probabilistic.

[0191] Now for reference Figure 16 , Figure 16 This is an example illustration of training a convolutional neural network (CNN) to generate a depth map of a sample, consistent with embodiments of this disclosure. Figure 16 The illustration shows multiple input images 1601 supplied to a CNN 1602 to output multiple 3D depth maps 1603. The depth maps can be generated by the CNN 1602 for each of the multiple input images 1601. The CNN 1602 can generate the depth maps 1603 as described above and via the loss function or other mathematical operations described above. Input images 1601 can be simulated images of the sample to be inspected and can represent any shape or geometry (e.g., line space, contact holes, etc.). Input images 1601 can be generated by a simulator (e.g., an electron beam simulator), and multiple images can be generated by changing the input parameters of the simulator. Input images 1601 can be generated by changing parameters such as surface roughness and defect size. Defect types can be any obvious defect in the IC. Non-limiting examples of input defects include top loss, scumming, or any other defect that may occur during IC manufacturing. Defect size can vary in dimension (e.g., dy in the Y direction, nm) and height (e.g., zcut in the Z direction, nm). Roughness can vary from 0 to 1.5, where a roughness of 0 indicates an ideally smooth feature surface. Real feature surfaces may have non-zero roughness values.

[0192] After CNN 1602 is trained, input image 1604 can be fed into trained CNN 1605 to generate depth image 1606. Input image 1604 can be a second or more simulated images, which can have different parameters (e.g., roughness, defect type, defect size, etc.) compared to input image 1601. Input image 1604 can also be a non-simulated image of the sample. Input image 1604 can be an experimentally collected image of the sample from an inspection device. In some embodiments, the inspection device can be a charged particle beam device, atomic force microscope, scattering meter, optical imaging device, or any other tool for inspecting ICs. In some embodiments, the inspection device can be an atomic force microscope, scattering meter, focused ion beam scanning electron microscope, scanning electron beam microscope, transmission electron beam microscope, scanning transmission electron beam microscope, or soft X-ray microscope.

[0193] For reference Figure 17 , Figure 17 This is an example illustration of a depth map of a sample generated from an input image, consistent with embodiments of this disclosure. In some embodiments, reference images 1701 and 1703 and input image 1702 may be supplied to a trained CNN (e.g., Figure 16 The trained CNN 1605 in the example. Reference images 1701 and 1703 and input image 1702 can be simulated or non-simulated images. In some embodiments, reference images 1701 and 1703 can be gold images, GDS files, design templates for features to be manufactured, or defect-free manufactured features. Input image 1702 can be an image of interest of a sample and may contain defects. As mentioned above, reference images 1701 and 1703 and input image 1702 are not limited to... Figure 17 The figure illustrates the feature shapes and geometries. Any feature shape or geometries can be envisioned in the embodiments of this disclosure. The generated depth map 1705 corresponds to the input image 1702, the generated depth map 1704 corresponds to the reference image 1701, and the generated depth map 1706 corresponds to the reference image 1703. The depth map 1705 can illustrate a three-dimensional representation of defects present in the input image 1702, defects that may not be easily identifiable in the input image 1702.

[0194] The generated depth maps 1704-1706 can be used to calculate the defect intensity of defects present in depth map 1705 (or input image 1702). In some embodiments, the defect intensity can be calculated using the following formula: (Equation 2)

[0195] In Equation 2, T is a sensitivity threshold parameter that can be tuned to an optimal value. σ21 and σ23 are the standard deviations of 1705-1704 and 1705-1706, respectively. Equation 2 can be understood as determining the difference between depth map 1705 and depth maps generated from reference images (e.g., depth maps 1704 and 1706, and reference images 1701 and 1703), and identifying defects with values ​​greater than or equal to 1. Equation 2 is applied to each location on depth maps 1704-1706 and can be repeated for each location on depth maps 1704-1706. In some embodiments, the defect intensity can be calculated for each pixel in depth maps 1704-1706.

[0196] Now for reference Figure 18A , Figure 18A This is an example illustration of an input image used to train a CNN to output a depth map, consistent with embodiments of this disclosure. Figure 18A Images 1801 and 1802, which may contain defects, are illustrated. Input images 1801 and 1802 can be simulated to contain nine defects with nominal dimensions (e.g., 8 nm or 16 nm z-cut). Possible defects observed in the images are identified by a solid black outline 1803. In some embodiments, image 1801 corresponds to a top-loss defect with a z-cut value of 8 nm. In some embodiments, image 1802 corresponds to a top-loss defect with a z-cut value of 16 nm. Input images 1801 and 1802 can be used to identify possible defects outlined in 1803; however, because images 1801 and 1802 are two-dimensional illustrations, it may be difficult to accurately determine the location of a defect in the images.

[0197] For reference Figures 18B to 18G , Figures 18B to 18G This is an example illustration of a binary map generated by aggregating the defect intensities calculated for each location in the generated depth map, consistent with embodiments of this disclosure. The binary map can be processed after computation. In some embodiments, the binary map can be processed using morphological operators, such that the defect is processed as a square structured object of size D. In some embodiments, the binary map can be processed using Gaussian blur with a filter size of sigma. Figures 18B to 18G Each figure in the figure shows a binary map generated by calculating the defect strength via Equation 2, which has varying parameters (e.g., roughness T, structure object size D, and Gaussian kernel size sigma). Figures 18B to 18G The results are shown for an input image (e.g., a simulated image) with nine defects, each having a nominal defect height (e.g., zcut) ranging from 4 nm to 32 nm. Figures 18B to 18G The binary map shown illustrates the presence of defects using solid white bars. In some embodiments, defects are more clearly illustrated at defect heights above 8 nm.

[0198] Now for reference Figure 19A , Figure 19A This is an example illustration of using a machine learning model to quantify defect size, consistent with embodiments of this disclosure. Figure 19A A two-dimensional input image 1901 is illustrated, which may contain defects that are not easily identified. A depth map 1902 can be generated from the input image 1901 to predict possible defects using a trained CNN as described above. In some embodiments, the input image 1901 may be a simulated image generated via parameters to model possible defects present on the sample. In some embodiments, the input image 1901 may be an image of a fabricated feature or sample collected experimentally. The input image 1901 may correspond to a defect of nominal size (e.g., top loss at a 16 nm z-cut). The depth map 1902 may illustrate a three-dimensional representation of possible defects present on the sample. The depth map 1902 illustrates possible top loss defects, but embodiments of this disclosure are not limited thereto. For example, the depth map 1902 may illustrate scum defects or defects associated with other structural shapes or geometries of the IC. Figure 19A A depth map 1903, which can be generated from a reference image of the sample, is also illustrated. In some embodiments, the reference image may indicate defects present on the sample. In some embodiments, the reference image may be an image of the sample under examination. In some embodiments, the reference image may be historical data of the sample. In some embodiments, the reference image may be coupled with some type of measurement data / process that provides contour information, which may be related to the contour information in the depth map 1902. In some embodiments, the depth map 1903 may be considered a baseline truth. In some embodiments, depth maps 1902 and 1903 may be analyzed to quantify possible defects. In some embodiments, possible defects may be represented by shape, correlation charts, or distribution. In some embodiments, possible defects may be represented by any portion of the shape that creates a deviation or offset from the expected shape (e.g., a feature) in terms of height, depth, or length.

[0199] Now for reference Figure 19B and Figure 19C , Figure 19B and Figure 19C This is an example illustration consistent with embodiments of the present disclosure, used to quantify a machine learning model to determine defect size. Figure 19B The diagram illustrates a depth map (e.g., Figure 19AThe modeled dimension of the defect height (e.g., zcut, height in the z direction) at a location on the depth map (Figure 1902), and the modeled dimension compared to the baseline ground truth (e.g., Figure 19A The depth mapping in Figure 1903 is compared with that in the figure. Figure 19B The line profile shown in the diagram can be mapped using a depth map around the center of the defect (e.g., Figure 19A The signal is obtained by averaging over 6 pixels in the depth maps 1902 and 1903. Figure 19B The illustration shows a cross-sectional view of depth map 1902, where defects are present in both the predicted depth map and the baseline true depth map. The cross-sectional view of the predicted depth map (“predicted”) is observed to match the baseline true depth map (“real”). This indicates that the generated depth map (e.g., depth map 1902) accurately predicts the presence and size of defects on the sample to be inspected. Figure 19C The illustration shows the modeled length (e.g., dy) of a defect consistent with embodiments of this disclosure. Figure 19C The line profile illustrated in the figure can be obtained by averaging the signal over six pixels in the depth map along the length of the defect. The defect length can be modeled as a Gaussian distribution from the depth map (e.g., depth map 1902) and compared with a modeled defect length from a ground truth reference (e.g., depth map 1903). In some embodiments, the defect length can be determined by the full width at half maximum (FWHM) value. Figure 19C As shown, the predicted defect length matches the baseline true defect length. It should be understood that embodiments of this disclosure are not limited to the Gaussian distribution used for modeling defect length. Non-limiting examples include any metric that can model or limit defect length, and any metric can be used to model or limit defect height.

[0200] Such as Figure 19A Depth maps like depth map 1902 can be applied to multiple input images (as described above for...). Figures 18A to 18G The prediction of defects of various sizes (e.g., height and length) is generated based on an input image derived from varying parameters (e.g., roughness, Gaussian kernel size, T). Now refer to... Figure 20 , Figure 20 The illustration shows an example correlation chart of predicted defect size versus baseline true defect size with different characteristic roughness values ​​(e.g., sigma), consistent with some embodiments of this disclosure. The correlation chart can be generated for predicted defects with dimensions (height or length) ranging from 4 nm to 36 nm. Figure 20The figure illustrates the linear correlation between the defect size determined from the baseline true value and the predicted defect size. For defect heights below 8 nm, there is a slight deviation in linearity, and the predicted defect length appears less than ideal for top losses of 4 nm and 8 nm.

[0201] Figure 20 The results shown in the figure illustrate the strong correlation between the predicted defect inspections and measurements and the actual defects present on the sample.

[0202] It should be understood that while the above discussion pertains to top loss defects, the embodiments of this disclosure are not limited thereto. For example, a CNN can be trained based on simulated inspection images of any defects present in the IC structure (e.g., scum, pins, etc.). The depth map can be generated for scum or pin defects in the IC structure. In some embodiments, a binary map can be generated to identify the location of the scum or pin defect, and the defect size can be calculated and correlated with a baseline ground truth scum or pin defect.

[0203] Now for reference Figure 21 , Figure 21 This is a flowchart of an exemplary machine learning method 2100 for defect detection, consistent with embodiments of this disclosure. In step 2101, a convolutional neural network is trained using images of samples to generate a depth map from the images. In some embodiments, the images can be obtained experimentally or simulated. In some embodiments, multiple images can be fed to the CNN for training. Multiple images can be obtained experimentally or generated by a simulator (e.g., an electron beam simulator, an optical simulator) using modified feature roughness, size, structure, or shape.

[0204] In step 2102, the trained CNN generates a depth map from the reference image and the image of interest of the sample. In some embodiments, the reference image and the image of interest may be experimentally obtained images of the sample. In some embodiments, the reference image and the image of interest may be examination images of the sample. In some embodiments, multiple images of interest and multiple reference images may be fed to the trained CNN.

[0205] In step 2103, the defect strength is calculated for a location on the generated depth map among a plurality of locations. In some embodiments, the defect strength may be calculated based on a mathematical operation that correlates the strength of the defect signal with an estimator of the signal-to-noise metric in the generated depth map. In some embodiments, the defect strength may be calculated based on Equation 2. In some embodiments, the defect strength may be calculated based on the signal-to-noise metric. In some embodiments, the defect strength may be calculated based on a threshold.

[0206] In step 2104, the binary map is generated by aggregating multiple locations with the calculated defect intensity. The binary map can be processed by morphological operators such that the defect is processed as a structural object square of size D. In some embodiments, the binary map can be processed using Gaussian blur with a filter size sigma. In step 2105, the generated binary map is used to guide the inspection of an inspection tool. In some embodiments, the generated binary map can be used to guide the measurement of the inspection tool. In some embodiments, the inspection tool can select locations on the generated binary map to inspect for possible defects and collect measurements to quantify the defects (e.g., as shown in the image). Figure 19B and Figure 19C (As depicted in the figure). In some embodiments, the inspection tool is an optical inspection tool. In some embodiments, the inspection tool is a charged particle beam device. In some embodiments, the charged particle beam device is a scanning electron microscope.

[0207] In some embodiments, a system is provided that can use a model to generate 3D data (e.g., depth data) after the system captures an image of a patterned substrate. In some embodiments, the system may be configured to include, for example, a DBM model discussed herein. Figure 22 SEM tools or Figure 23 The measurement tools include, for example, an electron beam generator for capturing an image of a patterned substrate; and one or more processors including a trained model described herein. The one or more processors are configured to use the captured image to execute the trained model to generate depth data. As previously mentioned, the model may be a convolutional neural network.

[0208] In some embodiments, the 3D data generated by the model (e.g., depth data) can be used to improve the patterning process. For example, depth data can be used in simulations of the patterning process, such as to predict contours, CD, edge placement (e.g., edge placement error), etc., in the resist and / or etched images. The goal of the simulation is to accurately predict, for example, the edge placement of the printed pattern and / or the spatial image intensity slope and / or CD, etc. These values ​​can be compared with the expected design, for example, to correct the patterning process, identify the locations of predicted defects, etc. The expected design is typically defined as a pre-OPC design layout, which can be provided in a standard digital file format (such as GDSII or OASIS or other file formats).

[0209] In some embodiments, the inspection or measurement device may be a scanning electron microscope (SEM) that produces images of a structure (e.g., some or all of the structure of a device), the images of which are exposed or transferred onto a substrate. Figure 22 Some embodiments of the SEM tool are depicted. The primary electron beam EBP emitted from the electron source ESO is converged by the focusing lens CL and then passed through the beam deflectors EBD1, E x B deflector EBD2 and objective lens OL to radiate the substrate PSub on the substrate stage ST at the focal point.

[0210] When the substrate PSub is irradiated with the electron beam EBP, secondary electrons are generated from the substrate PSub. These secondary electrons are deflected by the ExB deflector EBD2 and detected by the secondary electron detector SED. A two-dimensional electron beam image can be obtained by detecting the electrons generated from the sample, which is synchronized with, for example, a two-dimensional scan of the electron beam by the beam deflector EBD1, or repeated scanning of the electron beam EBP in the X or Y direction by the beam deflector EBD1, while the substrate PSub is continuously moved in the other direction (X or Y) by the substrate stage ST.

[0211] The signal detected by the secondary electron detector (SED) is converted into a digital signal by an analog-to-digital (A / D) converter (ADC), and the digital signal is sent to the image processing system (IPU). In some embodiments, the image processing system IPU may have a memory (MEM) to store all or part of the digital image for processing by the processing unit (PU). The processing unit (PU) (e.g., specially designed hardware or a combination of hardware and software) is configured to convert or process the digital image into a dataset representing the digital image. Furthermore, the image processing system IPU may have a storage medium (STOR) configured to store the digital image and the corresponding dataset in a reference database. A display device (DIS) may be connected to the image processing system IPU, allowing an operator to perform necessary operations of the device with the aid of a graphical user interface.

[0212] As mentioned above, SEM images can be processed to extract the contours of edges describing objects in the image, representing device structures. These contours are then quantized via a metric such as CD. Therefore, images of device structures are typically compared and quantized using simplified metrics such as edge-to-edge distance (CD) or simple pixel differences between images. Typical contour models for detecting the edges of objects in an image to measure CD use image gradients. In fact, these models rely on strong image gradients. However, in practice, images are often noisy and have discontinuous boundaries. Techniques such as smoothing, adaptive thresholding, edge detection, erosion, and dilation can be used to process the results of image gradient contour models to address the problems of noisy and discontinuous images, but ultimately result in low-resolution quantization of high-resolution images. Therefore, in most cases, mathematical processing of images of device structures to reduce noise and automatically perform edge detection leads to a loss of image resolution, resulting in a loss of information. Thus, the result is low-resolution quantization, which is equivalent to a simplified representation of a complex high-resolution structure.

[0213] Therefore, a mathematical representation of a structure (circuit feature, alignment mask, or measurement target portion (e.g., grating feature)) that retains resolution and describes the general shape of the structure is desired. This structure is generated or is intended to be generated using a patterning process, whether the structure is in a potential resist image, in a developed resist image, or a layer transferred to a substrate, for example, by etching. In the context of photolithography or other patterning processes, the structure can be a fabricated device or a portion thereof, and the image can be an SEM image of the structure. In some cases, the structure can be a feature of a semiconductor device (e.g., an integrated circuit). In this case, the structure can be referred to as a pattern or desired pattern comprising multiple features of the semiconductor device. In some cases, the structure can be: an alignment mark or a portion thereof (e.g., an alignment mark grating), used in an alignment measurement process to determine the alignment of an object (e.g., a substrate) with another object (e.g., a patterning apparatus); or a measurement target or a portion thereof (e.g., a measurement target grating), used to measure parameters of the patterning process (e.g., overlay, focus, dose, etc.). In some embodiments, the measurement target is used to measure, for example, an overlay diffraction grating.

[0214] Figure 23 Another embodiment of the inspection apparatus is schematically illustrated. The system is used to inspect a sample 90 (such as a substrate) on a sample stage 88 and includes a charged particle beam generator 81, a focusing lens module 82, a probe forming objective module 83, a charged particle beam deflection module 84, a secondary charged particle detector module 85, and an image forming module 86.

[0215] A charged particle beam generator 81 generates a primary charged particle beam 91. A focusing lens module 82 converges the generated primary charged particle beam 91. A probe forming objective module 83 focuses the converged primary charged particle beam onto a charged particle beam probe 92. A charged particle beam deflection module 84 scans the formed charged particle beam probe 92 across the surface of the region of interest on the sample 90 fixed on the sample stage 88. In some embodiments, the charged particle beam generator 81, the focusing lens module 82, and the probe forming objective module 83, or their equivalents, alternatives, or any combination thereof, together form a charged particle beam probe generator for generating and scanning the charged particle beam probe 92.

[0216] When the sample surface is bombarded by the charged particle beam probe 92, the secondary charged particle detector module 85 detects secondary charged particles 93 emitted from the sample surface (possibly along with other reflected or scattered charged particles from the sample surface) to generate a secondary charged particle detection signal 94. An image forming module 86 (e.g., a computing device) is coupled to the secondary charged particle detector module 85 to receive the secondary charged particle detection signal 94 from the secondary charged particle detector module 85 and accordingly form at least one scanned image. In some embodiments, the secondary charged particle detector module 85 and the image forming module 86, or their equivalents, alternatives, or any combination thereof, together form an image forming apparatus that forms a scanned image from the detected secondary charged particles emitted due to the bombardment of the sample 90 by the charged particle beam probe 92.

[0217] In some embodiments, the monitoring module 87 is coupled to the image forming module 86 of the image forming apparatus to monitor, control, and / or derive parameters for patterning process design, control, monitoring, etc., using scanned images of the sample 90 received from the image forming module 86. Therefore, in some embodiments, the monitoring module 87 is configured or programmed to cause the execution of the methods described herein. In some embodiments, the monitoring module 87 includes a computing device. In some embodiments, the monitoring module 87 includes a computer program that provides the functionality described herein and is encoded on a computer-readable medium forming or disposed within the monitoring module 87.

[0218] In some embodiments, for example Figure 22 Electron beam inspection tools that use probes to inspect substrates. Figure 23 Electronic current in a system and, for example, such as Figure 22The probe spot is significantly larger than that depicted in the CD SEM, allowing for a sufficiently large probe spot and faster inspection speed. However, due to the large probe spot, the resolution may not be as high as that of the CD SEM. In some embodiments, without limiting the scope of this disclosure, the inspection apparatus discussed above may be a single-beam or multi-beam apparatus.

[0219] From, for example Figure 22 and / or Figure 23 The system's SEM images can be processed to extract contours of edges describing objects in the image, representing device structures. These contours are then typically quantized at user-defined cut lines using a metric such as edge-to-edge distance (CD) measured on the extracted contours. Therefore, images of device structures are often compared and quantized using metrics such as edge-to-edge distance (CD) measured on the extracted contours or simple pixel differences between images.

[0220] In some embodiments, one or more processes of the method may be implemented as instructions (e.g., program code) in a processor of a computer system (e.g., processor 104 of computer system 100). In some embodiments, the program may be distributed across multiple processors (e.g., parallel computing) to improve computational efficiency. In some embodiments, a computer program product including a non-transitory computer-readable medium has instructions recorded on the computer program product that, when executed by a computer hardware system, implement the method described herein.

[0221] As discussed herein, various methods (e.g., methods 500, 700, 1100, and 1300) are provided for training models configured to generate depth information from a single SEM image of a patterned substrate. Therefore, using the models described herein, depth information can be estimated using only a single measurement, thereby saving measurement time. The depth information can also be further employed to configure lithography processes to improve yield or minimize defects.

[0222] According to this disclosure, combinations and sub-combinations of the disclosed elements constitute individual embodiments. For example, a first combination includes determining a model configured to estimate depth data using a single image (e.g., SEM image, AFM data, optical image, etc.). A sub-combination may include using a trained model to determine the depth data. In another combination, the depth data may be used during inspection, with variance data generated by the model used to determine OPC or SMO. In yet another example, the combination includes determining process adjustments to the lithography, resist, or etching processes based on inspection data based on depth information to improve the yield of the patterning process.

[0223] Figure 24This is a block diagram illustrating a computer system 100 that may assist in implementing the methods, processes, or apparatus disclosed herein. The computer system 100 includes a bus 102 or other communication mechanism for transmitting information, and a processor 104 (or multiple processors 104 and 105) coupled to the bus 102 for processing information. The computer system 100 also includes a main memory 106, such as random access memory (RAM) or other dynamic storage devices, coupled to the bus 102, for storing information and instructions to be executed by the processor 104. The main memory 106 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processor 104. The computer system 100 also includes a read-only memory (ROM) 108 or other static storage devices coupled to the bus 102 for storing static information and instructions for the processor 104. A storage device 110, such as a magnetic disk or optical disk, is provided and coupled to the bus 102 for storing information and instructions.

[0224] Computer system 100 can be coupled to display 112, such as a cathode ray tube (CRT), flat panel display, or touch panel display, via bus 102 to display information to the computer user. Input device 114, including alphanumeric keys and other keys, is coupled to bus 102 for transmitting information and command selection to processor 104. Another type of user input device is cursor control 116, such as a mouse, trackball, or arrow keys, for transmitting directional information and command selection to processor 104 and for controlling cursor movement on display 112. This input device typically has two degrees of freedom on two axes, a first axis (e.g., x) and a second axis (e.g., y), allowing the device to specify a position in a plane. Touch panel (screen) displays can also be used as input devices.

[0225] According to one embodiment, portions of one or more methods described herein can be executed by computer system 100 in response to processor 104 executing one or more sequences of one or more instructions contained in main memory 106. Such instructions may be read into main memory 106 from another computer-readable medium, such as storage device 110. Execution of the sequence of instructions contained in main memory 106 causes processor 104 to perform the process steps described herein. One or more processors in a multiprocessor arrangement may also be employed to execute the sequence of instructions contained in main memory 106. In another embodiment, hardwired circuitry may be used in place of or in combination with software instructions. Therefore, the description herein is not limited to any particular combination of hardware circuitry and software.

[0226] As used herein, the term "computer-readable medium" refers to any medium that participates in providing instructions to processor 104 for execution. Such media can take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical discs or magnetic disks, such as storage device 110. Volatile media include dynamic memory, such as main memory 106. Transmission media include coaxial cables, copper wires, and optical fibers, including conductors forming bus 102. Transmission media can also take the form of sound waves or light waves, such as sound waves or light waves generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs, any other optical media, punched cards, paper tapes, any other physical media with a perforated pattern, RAM, PROMs, EPROMs, FLASH-EPROMs, any other memory chips or tapes, carrier waves as described below, or any other media that a computer can read.

[0227] Various forms of computer-readable media can participate in transmitting one or more sequences of one or more instructions to processor 104 for execution. For example, instructions may initially be carried on the disk of a remote computer. The remote computer may load the instructions into its dynamic memory and transmit the instructions over a telephone line using a modem. A modem local to computer system 100 may receive data over the telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector coupled to bus 102 may receive the data transmitted in the infrared signal and place the data on bus 102. Bus 102 transmits the data to main memory 106, from which processor 104 retrieves and executes the instructions. Instructions received in main memory 106 may optionally be stored on storage device 110 before or after execution by processor 104.

[0228] Computer system 100 may also include a communication interface 118 coupled to bus 102. Communication interface 118 provides bidirectional data communication coupled to network link 120, which is connected to local network 122. For example, communication interface 118 may be an Integrated Services Digital Network (ISDN) card or a modem to provide data communication connectivity to a corresponding type of telephone line. As another example, communication interface 118 may be a Local Area Network (LAN) card to provide data communication connectivity to a compatible LAN. A wireless link may also be implemented. In any such implementation, communication interface 118 transmits and receives electrical, electromagnetic, or optical signals that carry streams of digital data representing various types of information.

[0229] Network link 120 typically provides data communication to other data devices via one or more networks. For example, network link 120 may provide connection to host computer 124 or to data devices operated by Internet Service Provider (ISP) 126 via local network 122. ISP 126 then provides data communication services via a global packet data communication network now commonly referred to as the "Internet" 128. Both local network 122 and Internet 128 use electrical, electromagnetic, or optical signals to transmit digital data streams. Signals through various networks and on network link 120 via communication interface 118 are exemplary forms of carrier waves that transmit digital data to or from computer system 100.

[0230] Computer system 100 can send messages and receive data, including program code, via networks(s), network link 120, and communication interface 118. In the Internet example, server 130 can transmit application-requested code via Internet 128, ISP 126, local network 122, and communication interface 118. Such a downloaded application can provide all or part of the methods described herein. The received code can be executed by processor 104 upon receipt and / or stored in storage device 110 or other non-volatile storage for later execution. In this way, computer system 100 can obtain application code in carrier form.

[0231] Figure 25 An exemplary photolithography projection apparatus that can be used in conjunction with the techniques described herein is schematically depicted. The apparatus includes: The irradiation system IL is used to modulate the radiation beam B. In this specific case, the irradiation system also includes a radiation source SO; A first object stage (e.g., a pattern forming apparatus stage) MT is provided with a pattern forming apparatus holder for holding a pattern forming apparatus MA (e.g., a mask) and is connected to a first positioner to accurately position the pattern forming apparatus relative to the item PS. The second object stage (substrate stage) WT is provided with a substrate holder for holding the substrate W (e.g., a silicon wafer coated with resist) and is connected to the second positioner to accurately position the substrate relative to the item PS. A projection system (“lens”) PS (e.g., a refractive, reflective, or catadioptric optical system) images the irradiated portion of the patterning apparatus MA onto a target portion C (e.g., including one or more dies) of the substrate W.

[0232] As described herein, the apparatus is transmissive (i.e., has transmissive patterning equipment). However, it can also typically be, for example, reflective (with reflective patterning equipment). The apparatus can employ a different type of patterning equipment than classical masks; examples include programmable mirror arrays or LCD matrices.

[0233] A source SO (e.g., a mercury lamp or excimer laser, LPP (laser-generated plasma) EUV source) produces a beam of radiation. This beam is fed into the irradiation system (irradiator) IL, either directly or after passing through an adjustment device (such as a beam expander Ex). The irradiator IL may include an adjustment device AD ​​for setting the outer radial range and / or inner radial range (typically referred to as σ-outer and σ-inner, respectively) of the intensity distribution in the beam. Furthermore, it will typically include various other components, such as an integrator IN and a beam concentrater CO. In this way, the beam B striking the patterning device MA has the desired uniformity and intensity distribution in its cross-section.

[0234] about Figure 25 It should be noted that the source SO can be inside the housing of the lithography projection device (which is usually the case when the source SO is, for example, a mercury lamp), but it can also be located away from the lithography projection device, with the radiation beam generated by the source SO being guided into the device (e.g., by means of a suitable guide mirror); the latter scenario is usually the case when the source SO is an excimer laser (e.g., based on KrF, ArF, or F2 lasers).

[0235] Beam B then intercepts the patterning apparatus MA, which is held on the patterning apparatus stage MT. After passing through the patterning apparatus MA, beam B is passed through lens PL, which focuses beam B onto the target portion C of the substrate W. With the aid of a second positioning device (and an interferometric measuring device IF), the substrate stage WT can be accurately moved, for example, to accurately position the different target portions C within the path of beam PB. Similarly, for example, after mechanically acquiring the patterning apparatus MA from the patterning apparatus library or during scanning, a first positioning device can be used to accurately position the patterning apparatus MA relative to the path of beam B. Typically, the movement of the target stages MT and WT is achieved using long-stroke modules (coarse positioning) and short-stroke modules (fine positioning), which... Figure 25 It is not explicitly described in the text. However, in the case of a stepper (as opposed to a stepping scanning tool), the pattern forming equipment stage MT can be connected only to a short-stroke actuator, or it can be fixed.

[0236] The tools described can be used in two different modes: In step mode, the patterning apparatus stage MT is kept essentially stationary, and the entire patterning apparatus image is projected (i.e., a single "flash") onto the target portion C. The substrate stage WT is then shifted in the x and / or y directions so that different target portions C can be illuminated by the beam B; In scanning mode, the same scenario largely applies, except that the given target portion C is not exposed in a single "flash." Instead, the patterning stage MT can move at a speed v in a given direction (the so-called "scanning direction," e.g., the y-direction), causing the projected beam B to scan over the patterning image; simultaneously, the substrate stage WT moves in the same or opposite direction at a speed V = Mv, where M is the magnification of the lens PL (typically M = 1 / 4 or 1 / 5). In this way, a relatively large target portion C can be exposed without compromising resolution.

[0237] Figure 26 Another exemplary photolithography projection device, LA, is schematically depicted that can be used in conjunction with the techniques described herein.

[0238] Photolithography projection equipment (LA) includes: Source collector module SO; Irradiation system (irradiator) IL, the irradiation system IL is configured to modulate the radiation beam B (e.g., EUV radiation). A support structure (e.g., a pattern forming equipment stage) MT is configured to support a pattern forming device (e.g., a mask or mask plate) MA and is connected to a first positioner PM, which is configured to accurately position the pattern forming device. A substrate stage (e.g., a wafer stage) WT, configured to hold a substrate (e.g., a wafer coated with resist) W and connected to a second positioner PW, configured to accurately position the substrate; and A projection system (e.g., a reflective projection system) PS is configured to project a pattern given by a radiation beam B by a patterning device MA onto a target portion C (e.g., including one or more dies) of a substrate W.

[0239] As described herein, the apparatus LA is reflective (e.g., employing a reflective patterning device). It should be noted that since most materials are absorbent in the EUV wavelength range, the patterning device can have multilayer reflectors, which may include, for example, multiple stacks of molybdenum and silicon. In one example, the multilayer stacked reflector has 40 pairs of molybdenum and silicon, with each layer being a quarter wavelength thick. Even smaller wavelengths can be produced using X-ray lithography. Because most materials are absorbent at both EUV and X-ray wavelengths, small patches of patterned absorbing material on the morphology of the patterning device (e.g., a TaN absorber on top of a multilayer reflector) define the locations where features will be printed (positive resist) or not printed (negative resist).

[0240] refer to Figure 26 The irradiator IL receives an extreme ultraviolet (EUV) radiation beam from the source collector module SO. Methods for generating EUV radiation include, but are not necessarily limited to, converting a material into a plasma state having at least one element (e.g., xenon, lithium, or tin) with one or more emission lines in the EUV range. In one such method (often referred to as laser-generated plasma "LPP"), plasma can be generated by radiating a fuel (such as droplets, streams, or clusters of a material having a line-emitting element) using a laser beam. Figure 26 (Not shown) This is part of an EUV radiation system used to provide a laser beam for exciting the fuel. The resulting plasma emits output radiation (e.g., EUV radiation), which is collected using a radiation collector located in the source collector module. For example, when a CO2 laser is used to provide the laser beam for fuel excitation, the laser and the source collector module can be separate entities.

[0241] In this case, the laser is not considered part of the lithography apparatus, and the radiation beam is delivered from the laser to the source collector module via a beam delivery system, which includes, for example, suitable guide mirrors and / or beam expanders. In other cases, such as when the source is a discharge-generated plasma EUV generator (often referred to as a DPP source), the source can be part of the source collector module.

[0242] An irradiator IL may include adjusters for adjusting the angular intensity distribution of the radiation beam. Typically, at least the outer radial range and / or inner radial range (often referred to as σ-outer and σ-inner, respectively) of the intensity distribution in the pupil plane of the irradiator can be adjusted. Furthermore, the irradiator IL may include various other components, such as faceted fields and pupil reflector devices. The irradiator can be used to adjust the radiation beam to have a desired uniformity and intensity distribution in its cross-section.

[0243] A radiation beam B is incident on a patterning apparatus (e.g., a mask) MA held on a support structure (e.g., a patterning apparatus stage) MT and is patterned by the patterning apparatus. After being reflected from the patterning apparatus (e.g., the mask) MA, the radiation beam B is passed through a projection system PS, which focuses the beam onto a target portion C of the substrate W. The substrate stage WT can be accurately moved, for example, to position different target portions C within the path of the radiation beam B, using a second positioner PW and a position sensor PS2 (e.g., an interferometric device, a linear encoder, or a capacitive sensor). Similarly, a first positioner PM and another position sensor PS1 can be used to accurately position the patterning apparatus (e.g., the mask) MA relative to the path of the radiation beam B. The patterning apparatus (e.g., the mask) MA and the substrate W can be aligned using patterning apparatus alignment marks M1, M2 and substrate alignment marks P1, P2.

[0244] The depicted device LA can be used in at least one of the following modes: 1. In step mode, the support structure (e.g., a patterning stage) MT and the substrate stage WT are kept substantially stationary while the entire pattern imparting the radiation beam is projected onto the target portion C at once (i.e., a single static exposure). The substrate stage WT is then shifted in the X and / or Y directions so that different target portions C can be exposed. 2. In scanning mode, the support structure (e.g., a patterning equipment stage) MT and the substrate stage WT are scanned synchronously, while a pattern imparting a radiation beam is projected onto the target portion C (i.e., single dynamic exposure). The velocity and direction of the substrate stage WT relative to the support structure (e.g., a patterning equipment stage) MT can be determined by the (reduced) magnification and image inversion characteristics of the projection system PS. 3. In another mode, the support structure (e.g., a patterning stage) MT is held substantially stationary to hold the programmable patterning apparatus, and the substrate stage WT is moved or scanned while a pattern imparted by a radiation beam is projected onto the target portion C. In this mode, a pulsed radiation source is typically employed, and the programmable patterning apparatus is updated as needed after each movement of the substrate stage WT or between successive radiation pulses during scanning. This operating mode can be readily applied to maskless lithography utilizing programmable patterning apparatuses, such as programmable mirror arrays of the type referenced above.

[0245] Figure 27The apparatus LA is shown in more detail, comprising a source collector module SO, an irradiation system IL, and a projection system PS. The source collector module SO is constructed and arranged such that a vacuum environment can be maintained within the encapsulation structure 220 of the source collector module SO. EUV radiation-emitting plasma 210 can be formed by a plasma source generated by a discharge. EUV radiation can be generated by a gas or vapor, such as xenon (Xe), lithium (Li), or tin (Sn) vapor, in which an extremely hot plasma 210 can be created to emit radiation in the EUV range of the electromagnetic spectrum. The extremely hot plasma 210 is created, for example, by a discharge that causes at least partial ionization of the plasma. For efficient radiation generation, a partial pressure of xenon, lithium, tin vapor, or any other suitable gas or vapor, such as 10 Pa, may be required. In some embodiments, an excited tin (Sn) plasma is provided to generate EUV radiation.

[0246] Radiation emitted by thermal plasma 210 is transferred from source chamber 211 to collector chamber 212 via an optional gas barrier or contaminant trap 230 (also referred to in some cases as a contaminant barrier or foil trap), which is positioned in or after an opening in source chamber 211. Contaminant trap 230 may include a channel structure. Contaminant trap 230 may also include a gas barrier or a combination of a gas barrier and a channel structure. Contaminant traps or contaminant barriers 230 further indicated herein include at least channel structures as known in the art.

[0247] Collector chamber 211 may include a radiation collector CO, which may be a so-called grazing incidence collector. The radiation collector CO has an upstream radiation collector side 251 and a downstream radiation collector side 252. Radiation passing through the collector CO may be reflected from a grating spectral filter 240 to be focused along the optical axis indicated by the dashed line "O" into a virtual source point IF. The virtual source point IF is often referred to as the intermediate focus, and the source collector module is arranged such that the intermediate focus IF is located at or near the opening 221 in the encapsulation structure 220. The virtual source point IF is an image of the radiative emission plasma 210.

[0248] Subsequently, radiation passes through an illumination system IL, which may include a faceted field mirror device 22 and a faceted pupil mirror device 24. The faceted field mirror device 22 and the faceted pupil mirror device 24 are arranged to provide a desired angular distribution of the radiation beam 21 at the patterning apparatus MA, and to provide desired radiation intensity uniformity at the patterning apparatus MA. As the radiation beam 21 is reflected at the patterning apparatus MA, held by the support structure MT, a patterned beam 26 is formed and imaged by the projection system PS via reflective elements 28 and 30 onto the substrate W, held by the substrate stage WT.

[0249] The illumination optics unit IL and projection system PS can typically contain more components than shown. Depending on the type of lithography apparatus, a grating spectral filter 240 may optionally be present. Furthermore, more mirrors than shown may be present, for example, compared to... Figure 21 As shown, there can be 1 to 6 additional reflective elements in the projection system PS.

[0250] like Figure 27 As shown, the collector optics CO is depicted as a nested collector with grazing incidence reflectors 253, 254, and 255, which are merely examples of collectors (or collector mirrors). The grazing incidence reflectors 253, 254, and 255 are arranged axially symmetrically around the optical axis O, and this type of collector optics CO can be used in combination with a plasma source generated by discharge (commonly referred to as a DPP source).

[0251] Alternatively, the source collector module SO can be as follows: Figure 28 Part of the LPP radiation system is shown. A laser LA is arranged to deposit laser energy into a fuel such as xenon (Xe), tin (Sn), or lithium (Li), thereby creating a highly ionized plasma 210 with an electron temperature of tens of eV. Energetic radiation generated during the deexcitation and recombination of these ions is emitted from the plasma, collected by a near-normal incident collector optics CO, and focused onto an opening 221 in the encapsulation structure 220.

[0252] The embodiments may also be described using the following terms: 1. A method for defect inspection, comprising: Two-dimensional images are converted into three-dimensional images using a trained convolutional neural network (CNN); Calculate the defect intensity at the location on the three-dimensional image; A binary mapping is generated by aggregating the calculated defect intensities; The binary mapping is filtered to remove noise; and The binary mapping is used to guide the use of inspection tools to inspect the samples. 2. The method according to Clause 1, wherein the CNN is trained via simulated two-dimensional images. 3. The method according to Clause 1 or 2, wherein the two-dimensional image comprises an image of the sample obtained experimentally. 4. The method according to any one of clauses 1 to 3, wherein the trained CNN converts the first reference image into a first three-dimensional reference image and the second reference image into a second three-dimensional reference image. 5. The method according to Clause 4, wherein the first reference image and the second reference image are coupled with measurement information of the sample. 6. The method according to Clause 4 or 5, wherein the first reference image and the second reference image include design data of the sample. 7. The method according to Clause 4 or 5, wherein the first reference image and the second reference image comprise previously collected images of the sample. 8. The method according to any one of Clauses 1 to 7, wherein the calculation of defect strength is based on a mathematical operation that correlates the defect strength signal with a signal-to-noise ratio metric. 9. The method according to Clause 8, wherein the defect strength is calculated based on a threshold of the signal-to-noise ratio metric. 10. The method according to any one of clauses 1 to 9, wherein the location includes pixels on the three-dimensional image. 11. The method according to any one of clauses 1 to 10, wherein the generated binary mapping predicts the defect size on the sample. 12. The method according to Clause 11, wherein the generated binary map predicts the height or length of the defect. 13. The method according to any one of clauses 1 to 12, wherein filtering comprises: applying a morphological operator to the generated binary map. 14. The method according to Clause 13, wherein the morphological operator applies the shape or size of the structural object to the predicted defect in the binary mapping graph. 15. The method according to any one of clauses 1 to 12, wherein filtering comprises: applying Gaussian blur to the binary map. 16. The method according to any one of Clauses 1 to 15, wherein the defects predicted by the binary mapping include top loss, scum, or bottom defects. 17. The method according to any one of Clauses 1 to 16, wherein the two-dimensional image, the first reference image, and the second reference image comprise inspection images of the sample. 18. The method according to Clause 17, wherein the examined image comprises a scanning electron microscope image. 19. The method according to any one of Clauses 1 to 18, wherein the three-dimensional image, the first reference three-dimensional image, and the second reference three-dimensional image include a depth map. 20. The method according to any one of Clauses 1 to 19, wherein the binary map is used to guide the inspection of the optical inspection tool. 21. The method according to any one of Clauses 1 to 19, wherein the binary mapping is used to guide the inspection of the charged particle beam device. 22. The method according to Clause 21, wherein the charged particle beam device includes a scanning electron microscope. 23. A system comprising: Image inspection apparatus, the image inspection apparatus being configured to scan a sample and generate an image; and One or more processors, the one or more processors being configured to execute instructions to cause the system to perform operations, the operations including: Two-dimensional images are converted into three-dimensional images using a trained convolutional neural network (CNN); Calculate the defect intensity at the location on the three-dimensional image; A binary mapping is generated by aggregating the calculated defect intensities; The binary mapping is filtered to remove noise; and The binary mapping is used to guide the use of inspection tools to inspect the samples. 24. The system according to Clause 23, wherein the CNN is trained via a simulated two-dimensional image. 25. The system according to clause 23 or 24, wherein the two-dimensional image comprises an experimentally obtained image of the sample. 26. The system according to any one of clauses 23 to 25, wherein the trained CNN converts a first reference image into a first three-dimensional reference image and a second reference image into a second three-dimensional reference image. 27. The system according to Clause 26, wherein the first reference image and the second reference image are coupled with measurement information of the sample. 28. The system according to Clause 26 or 27, wherein the first reference image and the second reference image include design data of the sample. 29. The system according to Clause 26 or 27, wherein the first reference image and the second reference image comprise previously collected images of the sample. 30. The system according to any one of clauses 23 to 29, wherein the calculation of defect strength is based on a mathematical operation that correlates the defect strength signal with a signal-to-noise ratio metric. 31. The system according to Clause 30, wherein the defect strength is calculated based on a threshold of the signal-to-noise ratio metric. 32. The system according to any one of clauses 23 to 31, wherein the location includes pixels on the three-dimensional image. 33. The system according to any one of clauses 23 to 32, wherein the generated binary mapping predicts the defect size on the sample. 34. The system according to Clause 33, wherein the generated binary map predicts the height or length of a defect. 35. The system according to any one of clauses 23 to 34, wherein filtering comprises: applying a morphological operator to the generated binary map. 36. The system according to Clause 35, wherein the morphological operator applies the shape or size of the structural object to the predicted defect in the binary mapping graph. 37. The system according to any one of clauses 23 to 34, wherein filtering comprises: applying Gaussian blur to the binary map. 38. The system according to any one of clauses 23 to 37, wherein the defects predicted by the binary mapping include top loss, scum, or bottom defects. 39. The system according to any one of Clauses 23 to 38, wherein the two-dimensional image, the first reference image, and the second reference image comprise inspection images of the sample. 40. The system according to Clause 39, wherein the examined images include scanning electron microscope images. 41. The system according to any one of clauses 23 to 40, wherein the three-dimensional image, the first reference three-dimensional image, and the second reference three-dimensional image include a depth map. 42. The system according to any one of clauses 23 to 41, wherein the binary map is used to guide the inspection of the optical inspection tool. 43. The system according to any one of clauses 23 to 41, wherein the binary mapping is used to guide the inspection of the charged particle beam device. 44. The system according to Clause 43, wherein the charged particle beam device includes a scanning electron microscope. 45. A non-transitory computer-readable medium comprising a set of instructions executable by one or more processors of a computing device to cause the computing device to perform operations for predicting a failure rate of the device, the operations comprising: Two-dimensional images are converted into three-dimensional images using a trained convolutional neural network (CNN); Calculate the defect intensity at the location on the three-dimensional image; A binary mapping is generated by aggregating the calculated defect intensities; The binary mapping is filtered to remove noise; and The binary mapping is used to guide the use of inspection tools to inspect the samples. 46. ​​The non-transitory computer-readable medium as described in Clause 45, wherein the CNN is trained via a simulated two-dimensional image. 47. The non-transitory computer-readable medium according to clause 45 or 46, wherein the two-dimensional image comprises an experimentally obtained image of the sample. 48. A non-transitory computer-readable medium according to any one of clauses 45 to 47, wherein the trained CNN converts a first reference image into a first three-dimensional reference image and a second reference image into a second three-dimensional reference image. 49. The non-transitory computer-readable medium according to Clause 48, wherein the first reference image and the second reference image are coupled with measurement information of the sample. 50. The non-transitory computer-readable medium according to clause 48 or 49, wherein the first reference image and the second reference image include design data of the sample. 51. The non-transitory computer-readable medium according to clause 48 or 49, wherein the first reference image and the second reference image comprise previously collected images of the sample. 52. A non-transitory computer-readable medium according to any one of clauses 45 to 51, wherein the calculation of defect strength is based on a mathematical operation that correlates the defect strength signal with a signal-to-noise ratio metric. 53. The non-transitory computer-readable medium as described in Clause 52, wherein the defect strength is calculated based on a threshold of the signal-to-noise ratio metric. 54. The non-transitory computer-readable medium according to any one of clauses 45 to 53, wherein the location includes pixels on the three-dimensional image. 55. A non-transitory computer-readable medium according to any one of clauses 45 to 54, wherein the generated binary mapping predicts the defect size on the sample. 56. The non-transitory computer-readable medium as described in Clause 55, wherein the generated binary map predicts the height or length of a defect. 57. A non-transitory computer-readable medium according to any one of clauses 45 to 56, wherein filtering comprises: applying a morphological operator to the generated binary map. 58. The non-transitory computer-readable medium according to Clause 57, wherein the morphological operator applies the shape or size of a structural object to the predicted defect in the binary mapping graph. 59. The non-transitory computer-readable medium according to any one of clauses 45 to 56, wherein filtering comprises: applying Gaussian blur to the binary map. 60. A non-transitory computer-readable medium according to any one of clauses 45 to 59, wherein the defects predicted by the binary mapping include top loss, scum, or bottom defects. 61. A non-transitory computer-readable medium according to any one of clauses 45 to 60, wherein the two-dimensional image, the first reference image, and the second reference image comprise inspection images of a sample. 62. The non-transitory computer-readable medium as described in Clause 61, wherein the examination image includes a scanning electron microscope image. 63. The non-transitory computer-readable medium according to any one of clauses 45 to 62, wherein the three-dimensional image, the first reference three-dimensional image, and the second reference three-dimensional image comprise a depth map. 64. A non-transitory computer-readable medium according to any one of clauses 45 to 63, wherein the binary map is used to guide the inspection of an optical inspection tool. 65. A non-transitory computer-readable medium according to any one of clauses 45 to 63, wherein the binary map is used to guide the inspection of a charged particle beam device. 66. The non-transitory computer-readable medium according to Clause 65, wherein the charged particle beam device includes a scanning electron microscope.

[0253] The concepts disclosed in this paper can be used to simulate or mathematically model any general imaging system for imaging subwavelength features, and may be particularly useful for emerging imaging techniques capable of generating increasingly shorter wavelengths. Emerging techniques already in use include EUV (Extreme Ultraviolet) and DUV lithography, which can generate wavelengths of 193 nm using ArF lasers and even 157 nm using fluorine lasers. Furthermore, EUV lithography can generate wavelengths in the range of 20 nm to 5 nm by using synchrotrons or by utilizing high-energy electrons to bombard materials (solid-state or plasma) to generate photons within this range.

[0254] While the concepts disclosed herein can be used for imaging on substrates such as silicon wafers, it should be understood that the disclosed concepts can be used in any type of lithography imaging system, for example, for imaging on substrates other than silicon wafers.

[0255] As used herein, unless otherwise expressly stated, the term "or" covers all possible combinations unless impractical. For example, if a statement indicates that a database may include A or B, then unless otherwise expressly stated or impractical, the database may include A, or B, or A and B. As a second example, if a statement indicates that a database may include A, B, or C, then unless otherwise expressly stated or impractical, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0256] The above description is intended to be illustrative and not restrictive. Therefore, it will be apparent to those skilled in the art that modifications can be made without departing from the scope of the set forth claims.

Claims

1. A method for defect inspection, comprising: Two-dimensional images are converted into three-dimensional images using a trained convolutional neural network (CNN). Calculate the defect intensity at the location shown in the three-dimensional image; A binary mapping map is generated by aggregating the calculated defect intensities; The binary mapping is filtered to remove noise; as well as The binary mapping is used to guide the use of inspection tools to inspect the samples.

2. The method of claim 1, wherein the CNN is trained via a simulated two-dimensional image.

3. The method according to claim 1, wherein the two-dimensional image comprises an image of the sample obtained experimentally.

4. The method of claim 1, wherein the trained CNN converts the first reference image into a first three-dimensional reference image and the second reference image into a second three-dimensional reference image.

5. The method according to claim 4, wherein the first reference image and the second reference image are coupled with the measurement information of the sample.

6. The method according to claim 4 or 5, wherein the first reference image and the second reference image comprise design data of the sample, or comprise previously collected images of the sample.

7. The method of claim 1, wherein the calculation of defect strength is based on a mathematical operation that correlates the defect strength signal with a signal-to-noise ratio metric.

8. The method of claim 7, wherein the defect strength is calculated based on a threshold of the signal-to-noise ratio metric.

9. The method of claim 1, wherein the location comprises pixels on the three-dimensional image.

10. The method of claim 1, wherein the generated binary map predicts the defect size on the sample.

11. The method of claim 1, wherein filtering comprises: Apply morphological operators to the generated binary map, or apply Gaussian blur to the binary map.

12. The method of claim 11, wherein the morphological operator applies the shape or size of the structural object to the predicted defect in the binary map.

13. The method of claim 1, wherein the defects predicted by the binary mapping include top loss, scum, or bottom defects.

14. The method of claim 1, wherein the two-dimensional image, the first reference image, and the second reference image include inspection images of the sample.

15. The method of claim 1, wherein the three-dimensional image, the first reference three-dimensional image, and the second reference three-dimensional image comprise a depth map.

16. A system comprising: An image inspection device configured to scan a sample and generate an image; as well as One or more processors, the one or more processors being configured to execute instructions to cause the system to perform operations, the operations including: Two-dimensional images are converted into three-dimensional images using a trained convolutional neural network (CNN). Calculate the defect intensity at the location shown in the three-dimensional image; A binary mapping map is generated by aggregating the calculated defect intensities; The binary mapping is filtered to remove noise; and The binary mapping is used to guide the use of inspection tools to inspect the samples.

17. A non-transitory computer-readable medium comprising a set of instructions executable by one or more processors of a computing device to cause the computing device to perform operations for fault checking, the operations including: Two-dimensional images are converted into three-dimensional images using a trained convolutional neural network (CNN). Calculate the defect intensity at the location shown in the three-dimensional image; A binary mapping map is generated by aggregating the calculated defect intensities; The binary mapping is filtered to remove noise; as well as The binary mapping is used to guide the use of inspection tools to inspect the samples.

Images