Defect detection from a scanning electron microscope derived depth map with neural network and defect strength thresholding

Abstract

Described herein are machine learning based system, method, and non-transitory computer readable medium for determining three-dimensional (3D) information of a structure of a patterned substrate. The 3D information can be determined using a neural network to convert a two-dimensional image to a 3D image. In a method, the neural network is trained by supplying simulated images of a sample with varied parameters to model a real fabricated IC structure. The trained network generates a depth map from newly supplied images of a sample to predict defect location The depth map may be converted to a binary map to predict defect dimension and location that may be used to guide inspection of a sample using an inspection tool.

Description

DEFECT DETECTION FROM A SCANNING ELECTRON MICROSCOPE DERIVED DEPTH MAP WITH NEURAL NETWORK AND DEFECT STRENGTH THRESHOLDING CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority of US application 63 / 542,286 which was filed on 3 October 2023 and which is incorporated herein in its entirety by reference. TECHNICAL FIELD

[0002] The description herein relates generally to improved metrology systems and methods. More particularly, methods for determining and employing a model configured to determine three- dimensional data of a structure patterned on a substrate using a trained neural network. BACKGROUND

[0003] An image inspection apparatus (e.g., a charged-particle beam apparatus or an optical beam apparatus) is able to produce a two-dimensional (2D) image of a wafer substrate by detecting particles (e.g., photons, secondary electrons, backscattered electrons, mirror electrons, or other kinds of electrons) from a surface of a wafer substrate upon impingement by a beam (e.g., a charged-particle beam or an optical beam) generated by a source associated with the inspection apparatus. Various image inspection apparatuses are used on semiconductor wafers in the semiconductor industry for various purposes such as wafer processing (e.g., e-beam direct write lithography system), process monitoring (e.g., critical dimension scanning electron microscope (CD-SEM)), wafer inspection (e.g., e-beam inspection system), or defect analysis (e.g., defect review SEM, or say DR-SEM and Focused Ion Beam system, or say FIB).

[0004] To control quality of manufactured structures on the wafer substrate, the 2D image of the wafer substrate may be analyzed to detect potential defects in the wafer substrate. In some applications, the 2D image may be compared with a simulation image. The simulation image may be generated by a simulation technique configured to simulate an image measured by the image inspection apparatus. In some applications, 2D geometric features (e.g., edges) or three-dimensional (3D) geometric features may be extracted from the 2D image based on the simulation image. The quality of the simulation image may be an important factor for performance and accuracy of those applications. SUMMARY

[0005] With the advancement of lithography and other patterning process technologies, the dimensions of functional elements have continually been reduced while the amount of the functional elements, such as transistors, per device has been steadily increased over decades. In the meanwhile,the requirement of accuracy in terms of critical dimension (CD), height, etc. has become more and more stringent. Errors in shape and size of the structure may cause a problem in terms of the functioning of the device, including failure of the device to function or one or more electrical problems of the functioning device. Accordingly, it is desirable to be able to measure the three- dimensional structure of the functional elements to characterize reduce or minimize one or more of defects in the device. However, measuring three-dimensional (3D) structure of the functional elements using existing metrology tools and methods is time consuming, and inaccurate which negatively affects the yield of the patterning process.

[0006] Some embodiments of the present disclosure provide a method for defect inspection. The method comprises converting a two-dimensional image to a three-dimensional image via a trained convolutional neural network (CNN), calculating a defect strength at a location from a plurality of locations on the three-dimensional image, generating a binary map by aggregating the calculated defect strengths from the plurality of locations, filtering the binary map to remove noise, and utilizing the binary map to guide inspection of a sample using an inspection tool.

[0007] In some embodiments, a system is provided. The system comprises an image inspection apparatus configured to scan a sample and generate an image, and one or more processors configured to perform instructions to cause the system to perform operations. The operations comprise converting a two-dimensional image to a three-dimensional image via a trained convolutional neural network (CNN), calculating a defect strength at a location from a plurality of locations on the three- dimensional image, generating a binary map by aggregating the calculated defect strengths from the plurality of locations, filtering the binary map to remove noise, and utilizing the binary map to guide inspection of a sample using an inspection tool.

[0008] In some embodiments, a non-transitory computer readable medium comprising a set of instructions that is executable by one or more processors of a computing device to cause the computing device to perform operations for defect inspection is a provided. The operations comprise converting a two-dimensional image to a three-dimensional image via a trained convolutional neural network (CNN), calculating a defect strength at a location from a plurality of locations on the three- dimensional image, generating a binary map by aggregating the calculated defect strengths from the plurality of locations, filtering the binary map to remove noise, and utilizing the binary map to guide inspection of a sample using an inspection tool.

[0009] Other advantages of the present disclosure will become apparent from the following description taken in conjunction with the accompanying drawings wherein are set forth, by way of illustration and example, certain embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above aspects and other aspects and features will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments in conjunctionwith the accompanying figures, wherein:

[0011] Figure 1 shows a block diagram of various subsystems of a lithography system, according to some embodiments;

[0012] Figure 2A illustrates an example SEM image of a line having a depth of 10nm in a z- direction from a top of a substrate, according to some embodiments;

[0013] Figure 2B illustrate another example SEM image of a line having a depth of 100nm in a z- direction from a top of a substrate, according to some embodiments;

[0014] Figure 2C illustrates a SEM image of contact holes patterned on a substrate and corresponding depth map, according to some embodiments;

[0015] Figure 2D illustrates a SEM image of a set of lines patterned on a substrate and corresponding depth map, according to some embodiments;

[0016] Figure 2E illustrates a SEM image of another set of lines patterned on a substrate and corresponding depth map, according to some embodiments;

[0017] Figure 3 is a block diagram of a training process of a model, according to some embodiments;

[0018] Figure 4 illustrates a correlation between height and disparity data associated with stereo images of a patterned substrate, according to some embodiments;

[0019] Figure 5 is a flow chart of a method for training a model for estimating depth information of a structure of a patterned substrate, according to some embodiments;

[0020] Figure 6A is a block diagram of an example determination of depth information of a structure formed on a substrate using a model trained according to Figure 5, according to some embodiments;

[0021] Figure 6B is a flow chart of a method for determining depth information of a structure formed on a substrate using a model trained according to Figure 5, according to some embodiments;

[0022] Figure 7 is a flow chart of a method for generating a model configured to estimate depth data of a structure of a patterned substrate, according to some embodiments;

[0023] Figure 8A illustrates exemplary training of a model using simulated SEM data, according to some embodiments;

[0024] Figure 8B illustrates exemplary fine-tuning of the model (of Figure 8A) using metrology data, according to some embodiments;

[0025] Figure 8C illustrates exemplary scaling of the estimated depth data generated by the model of Figure 8A based on observed metrology data, the scaled estimated data can be further used in fine- tuning the model, according to some embodiments;

[0026] Figures 9A, 9B, and 9C illustrate SEM image with charging effect, predicted depth data with charging effect, and predicted depth data without charging effect, respectively, according to some embodiments;

[0027] Figure 10A is a flow chart of a method for estimating depth data by using a trained model(of Figure 7) using a single SEM image as input, according to some embodiments;

[0028] Figure 10B illustrates exemplary depth data estimated using the trained model, according to some embodiments;

[0029] Figure 11 is a flow chart of another method for training a model for estimating depth information of a structure of a patterned substrate, according to some embodiments;

[0030] Figure 12 illustrates exemplary training of a CNN based on method of Figure 11, where training data generated by a deterministic process simulator is used, according to some embodiments;

[0031] Figure 13 is a flow chart of yet another method for training a model for estimating depth information of a structure of a patterned substrate, according to some embodiments;

[0032] Figure 14 illustrates exemplary training of a CNN based on method of Figure 13, where training data generated by a deterministic process simulator is used, according to some embodiments;

[0033] Figure 15 is a flow chart of a method for estimating depth data by using a trained model (of Figure 11 or 13) and using a single SEM image as input, according to some embodiments;

[0034] Figure 16 is an example illustration of training a convolutional neural network (CNN) to generating a depth map of a sample, according to some embodiments.

[0035] Figure 17 is an example illustration of depth maps of a sample generate from input images, according to some embodiments.

[0036] Figure 18A is an example illustration of an input image to train a CNN to output a depth map, according to some embodiments.

[0037] Figure 18B to 18G are example illustrations of binary maps generated by aggregating a calculated defect strength per location in a generated depth map, according to some embodiments.

[0038] Figure 19A is an example illustration of quantifying a defect dimension using a machine learning model, according to some embodiments.

[0039] Figure 19B and 19C are example illustrations of metrics to quantify a machine learning model to determine a defect dimension, according to some embodiments.

[0040] Figure 20 illustrates example correlation plots of predicted defect dimension to ground truth defect dimension with different parameter values, according to some embodiments.

[0041] Figure 21 is a flowchart of an exemplary machine learning method for defect detection, according to some embodiments.

[0042] Figure 22 schematically depicts an example of a scanning electron microscope (SEM), according to some embodiments;

[0043] Figure 23 schematically depicts an example of an electron beam inspection apparatus, according to some embodiments;

[0044] Figure 24 is a block diagram of an example computer system, according to some embodiments;

[0045] Figure 25 is a schematic diagram of a lithographic projection apparatus, according to some embodiments;

[0046] Figure 26 is a schematic diagram of another lithographic projection apparatus, according to some embodiments;

[0047] Figure 27 is a more detailed view of the apparatus in Figure 25, according to some embodiments; and

[0048] Figure 28 is a more detailed view of the source collector module SO of the apparatus of Figure 26 and Figure 27, according to some embodiments. DETAILED DESCRIPTION

[0049] An integrated circuit (IC) chip used in a device (e.g., phone, laptop, computer memory, etc.) includes complex circuit patterns. During manufacturing of such circuit patterns, images of the printed circuit patterns are captured to determine whether desired circuit patterns are printed accurately. The ultimate performance of the fabricated device is critically dependent on the accuracy of the positioning and sizing of the various features of the product structure formed via lithography and other processing steps. These features are three-dimensional (3D) structures having a predetermined depth and shape at nanometer scale. The product structure made by an imperfect lithography procedure or other processing steps will result in a slightly different structure than an ideal or nominal desired structure.

[0050] In order to inspect the size of various features, three-dimensional information (e.g., height of features) can be very beneficial to ensure a feature at one layer connects to feature at another layer. However, obtaining 3D information of structures at nanometer scale is not a trivial task. In existing technology, 3D information may be obtained via tilted beam scanning electron microscope (SEM), in which two or more images targeted at the same location are needed in order to infer proper depth information. However, the use of multiple images for 3D metrology has several limitations. For example, capturing a pair of stereo images decreases a throughput of a patterning process or a metrology process due to having to switch between beam tilt angles. A proper alignment between the images is required to capture the stereo images. Processing the stereo images to determine depth information may be computationally expensive and prone to noise in the images and drifts in a metrology hardware. Thus, extraction of 3D information using existing technology can slow down the chip manufacturing and metrology process.

[0051] In some embodiments, machine learning may be employed in the generation of inspection images, reference images or other images associated with apparatus 100, 104A, or 104B. For example, in some embodiments a machine learning system may be operated in association with, e.g., controller 109 or 296, image processing system 199 or 290, image acquirer 120 or 292, or storage unit 130 or 294 of FIGs.1-2B. In some embodiments, machine learning may be employed in the real time recipe tuning method, e.g., method 300 of FIG.3 or 800 of FIG.8, in association with, e.g., an offline computation platform 301 of FIG.3, which will be described below. In some embodiments, a machine learning system may comprise a discriminative model. In some embodiments, a machinelearning system may include a generative model. For example, learning can feature two types of mechanisms: discriminative learning that may be used to create classification and detection algorithms, and generative learning that may be used to actually create models that, in the extreme, can render images. For example, as described further below, a generative model may be configured for generating an image from a design clip that resembles a corresponding location on a wafer in a SEM image. This may be performed by 1) training the generative model with design clips and the associated actual SEM images from those locations on the wafer; and 2) using the model in inference mode to feed the model design clips in locations for which simulated SEM images are desired. Such simulated images can be used as reference images in, e.g., die-to-database inspection.

[0052] If the model(s) include one or more discriminative models, the discriminative model(s) may have any suitable architecture and / or configuration known in the art. Discriminative models, also called conditional models, are a class of models used in machine learning for modeling the dependence of an unobserved variable “y” on an observed variable “x.” Within a probabilistic framework, this may be done by modeling a conditional probability distribution P(y|x), which can be used for predicting y based on x. Discriminative models, as opposed to generative models, may not allow one to generate samples from the joint distribution of x and y. However, for tasks such as classification and regression that do not require the joint distribution, discriminative models may yield superior performance. On the other hand, generative models are typically more flexible than discriminative models in expressing dependencies in complex learning tasks. In addition, most discriminative models are inherently supervised and cannot easily be extended to unsupervised learning. Application specific details ultimately dictate the suitability of selecting a discriminative versus generative model.

[0053] A generative model can be generally defined as a model that is probabilistic in nature. In other words, a “generative” model is not one that performs forward simulation or rule-based approaches and, as such, it may not be necessary to model the physics of the processes involved in generating an actual image or output (for which a simulated image or output is being generated). Instead, the generative model can be learned (in that its parameters can be learned) based on a suitable training set of data. Such generative models may have a number of advantages for the embodiments described herein. In addition, the generative model may be configured to have a deep learning architecture in that the generative model may include multiple layers, which may perform a number of algorithms or transformations. The number of layers included in the generative model may depend on the particular use case. For practical purposes, a suitable range of layers is from 2 layers to a few tens of layers.

[0054] Deep learning is a type of machine learning. Machine learning can be generally defined as a type of artificial intelligence (AI) that provides computers with the ability to learn without being explicitly programmed. Machine learning focuses on the development of computer programs that can teach themselves to grow and change when exposed to new data. Machine learning explores the studyand construction of algorithms that can learn from and make predictions on data—such algorithms overcome following strictly static program instructions by making data driven predictions or decisions, through building a model from sample inputs.

[0055] The machine learning described herein may be further performed as described in “Introduction to Statistical Machine Learning,” by Sugiyama, Morgan Kaufmann, 2016, 534 pages; “Discriminative, Generative, and Imitative Learning,” Jebara, MIT Thesis, 2002, 212 pages; and “Principles of Data Mining (Adaptive Computation and Machine Learning)” Hand et al., MIT Press, 2001, 578 pages; which are incorporated by reference as if fully set forth herein. The embodiments described herein may be further configured as described in these references.

[0056] In some embodiments, a machine learning system may comprise a neural network. For example, a model may be a deep neural network with a set of weights that model the world according to the data that it has been fed to train it. Neural networks can be generally defined as a computational approach which is based on a relatively large collection of neural units loosely modeling the way a biological brain solves problems with relatively large clusters of biological neurons connected by axons. Each neural unit is connected with many others, and links can be enforcing or inhibitory in their effect on the activation state of connected neural units. These systems are self-learning and trained rather than explicitly programmed and excel in areas where the solution or feature detection is difficult to express in a traditional computer program.

[0057] Neural networks typically consist of multiple layers, and the signal path traverses from front to back. The goal of the neural network is to solve problems in the same way that the human brain would, although several neural networks are much more abstract. Modern neural network projects typically work with a few thousand to a few million neural units and millions of connections. The neural network may have any suitable architecture and / or configuration known in the art.

[0058] In a further embodiment, a model may comprise convolutional and deconvolution neural network. For example, the embodiments described herein can take advantage of learning concepts such as a convolution and deconvolution neural network to solve the normally intractable representation conversion problem (e.g., rendering). The model may have any convolution and deconvolution neural network configuration or architecture known in the art.

[0059] In a further embodiment, a model may comprise convolutional and deconvolution neural network. For example, the embodiments described herein can take advantage of learning concepts such as a convolution and deconvolution neural network to solve the normally intractable representation conversion problem (e.g., rendering). The model may have any convolution and deconvolution neural network configuration or architecture known in the art.

[0060] Although specific reference may be made in this text to the manufacture of ICs, it should be explicitly understood that the description herein has many other possible applications. For example, it may be employed in the manufacture of integrated optical systems, guidance and detection patterns for magnetic domain memories, liquid-crystal display panels, thin-film magnetic heads, etc.The skilled artisan will appreciate that, in the context of such alternative applications, any use of the terms “reticle”, “wafer” or “die” in this text should be considered as interchangeable with the more general terms “mask”, “substrate” and “target portion”, respectively. The substrate referred to herein may be processed, before or after exposure, in for example a track (a tool that typically applies a layer of resist to a substrate and develops the exposed resist) or a metrology or inspection tool. Where applicable, the disclosure herein may be applied to such and other substrate processing tools. Further, the substrate may be processed more than once, for example in order to create a multi-layer IC, so that the term substrate used herein may also refer to a substrate that already contains multiple processed layers.

[0061] A critical dimension (CD) of a device refers the smallest width of a line or hole or the smallest space between two lines or two holes. Thus, the CD determines the overall size and density of the designed device. Of course, one of the goals in device fabrication is to faithfully reproduce the original design intent on the substrate (via the patterning device).

[0062] In the present document, the terms “radiation” and “beam” may be used to encompass all types of electromagnetic radiation, including ultraviolet radiation (e.g. with a wavelength of 365, 248, 193, 157 or 126 nm) and EUV (extreme ultra-violet radiation, e.g. having a wavelength in the range of about 5-100 nm).

[0063] The term “mask” or “patterning device” as employed in this text may be broadly interpreted as referring to a generic patterning device that can be used to endow an incoming radiation beam with a patterned cross-section, corresponding to a pattern that is to be created in a target portion of the substrate; the term “light valve” can also be used in this context. Besides the classic mask (transmissive or reflective; binary, phase-shifting, hybrid, etc.), examples of other such patterning devices include: -a programmable mirror array. An example of such a device is a matrix-addressable surface having a viscoelastic control layer and a reflective surface. The basic principle behind such an apparatus is that (for example) addressed areas of the reflective surface reflect incident radiation as diffracted radiation, whereas unaddressed areas reflect incident radiation as undiffracted radiation. Using an appropriate filter, the said undiffracted radiation can be filtered out of the reflected beam, leaving only the diffracted radiation behind; in this manner, the beam becomes patterned according to the addressing pattern of the matrix-addressable surface. The required matrix addressing can be performed using suitable electronic means. -a programmable LCD array. An example of such a construction is given in U.S. Patent No. 5,229,872, which is incorporated herein by reference.

[0064] As a brief introduction, Figure 1 illustrates an exemplary lithographic projection apparatus 10A. Major components are a radiation source 12A, which may be a deep-ultraviolet excimer laser source or other type of source including an extreme ultra violet (EUV) source (as discussed above, the lithographic projection apparatus itself need not have the radiation source), illumination optics which,e.g., define the partial coherence (denoted as sigma) and which may include optics 14A, 16Aa and 16Ab that shape radiation from the source 12A; a patterning device 18A; and transmission optics 16Ac that project an image of the patterning device pattern onto a substrate plane 22A. An adjustable filter or aperture 20A at the pupil plane of the projection optics may restrict the range of beam angles that impinge on the substrate plane 22A, where the largest possible angle defines the numerical aperture of the projection optics NA= n sin(Θmax), wherein n is the refractive index of the media between the substrate and the last element of the projection optics, and Θmaxis the largest angle of the beam exiting from the projection optics that can still impinge on the substrate plane 22A.

[0065] In a lithographic projection apparatus, a source provides illumination (i.e. radiation) to a patterning device and projection optics direct and shape the illumination, via the patterning device, onto a substrate. The projection optics may include at least some of the components 14A, 16Aa, 16Ab and 16Ac. An aerial image (AI) is the radiation intensity distribution at substrate level. A resist layer on the substrate is exposed and the aerial image is transferred to the resist layer as a latent “resist image” (RI) therein. The resist image (RI) can be defined as a spatial distribution of solubility of the resist in the resist layer. A resist model can be used to calculate the resist image from the aerial image, an example of which can be found in U.S. Patent Application Publication No. US 2009-0157360, the disclosure of which is hereby incorporated by reference in its entirety. The resist model is related only to properties of the resist layer (e.g., effects of chemical processes which occur during exposure, PEB and development). Optical properties of the lithographic projection apparatus (e.g., properties of the source, the patterning device and the projection optics) dictate the aerial image. Since the patterning device used in the lithographic projection apparatus can be changed, it may be desirable to separate the optical properties of the patterning device from the optical properties of the rest of the lithographic projection apparatus including at least the source and the projection optics.

[0066] Once a semiconductor chip is manufactured, measurements may be performed to determine dimension of structures fabricated on the substrate. For example, inspection of the fabricated structure may be based on critical dimension (CD) measurements obtained using a metrology tool (e.g., SEMs, AFM, optical tool, etc). In some embodiments, lateral dimensions (e.g., in x, y plane) of the structure may be extracted from the CD measurements. Besides the lateral dimensions of features, the measurements (e.g., CD-SEM) may also include information on 3D information (e.g., a height, CD values at different heights, height profile, etc.) of the features. In some embodiments, the height of the feature refers to a depth in a z-direction perpendicular to the x-y plane.

[0067] Figures 2A and 2B illustrate two different SEM images of a line having same CDs but different heights. Figure 2A illustrates a SEM image I1 of a line having a depth of 10nm in z-direction (perpendicular to the plane of the figures 2A and 2B) from a top of a substrate and Figure 2B illustrate another SEM image I2 of a line having a depth of 100nm in z-direction from a top of a substrate. These figures show that depth has a clear impact on the captured SEM images. However, although 3Dinformation (e.g., height information) is of interest, it is rarely extracted. In existing technology, only lateral information such as CD may be extracted, but not height.

[0068] For many problems related to semiconductor manufacturing there may be a correlation between input and output, but the correlation may be too complex for humans to model or identify. According to the present disclosure, a model is trained to extract depth information from a single image (e.g., SEM image) of the substrate. For example, a deep-learning algorithm may be applied to train a machine learning model such as a convolutional neural network (e.g., CNN) using training data and a training method described herein. In some embodiments, the methods described herein enable extraction of height of features from a single image of a substrate. For example, Figures 2C-2E illustrate different SEM images I3, I4, and I5, for each SEM image a depth map can be extracted from the SEM image, according to the present disclosure. In Figures 2C-2E, input SEM images I3, I4, and I5 are depicted at the top row and corresponding depth maps 3D3, 3D4, and 3D5, respectively, at the bottom row.

[0069] In some embodiments, a beam-tilt functionality of a SEM tool may be employed to retrieve depth information from a single CD-SEM image. An unsupervised training of a model may be performed using stereo image pairs (e.g., obtained by the beam-tilt SEM image) as training dataset. When semiconductor topologies are measured with a CD-SEM, the electron beam is commonly at normal incidence with respect to the substrate, resulting in top down images. This measurement method can resolve the substrate measurements with a lateral resolution close to 1 nm, whereas the axial resolution is often neglected due to the ill-posed problem of how to translate a SEM signal to a topology depth. To solve this ill-posed problem, tilted beam SEMs may be used to generate information about the depth of a structure. For example, “C. Valade, et al., Tilted beam SEM, 3D metrology for industry, Proc. SPIE 10959, Metrology, Inspection, and Process Control for Microlithography XXXIII, 109590Y (26 March 2019)” describes example tilted beam SEMs used to perform 3D metrology. In such systems stereo image pairs created with tilted electron beams from different directions are typically used.

[0070] For example, a SEM tool is capable of directing an electron beam at a desired angle (e.g., up to 12º with respect to a perpendicular to the substrate) to catch images at different angles, giving access to more information. From the analysis of such images, pattern height and sidewall angles can be consequently determined using geometric considerations.

[0071] However, in tilted beam SEM, two or more images targeted at the same location are needed in order to infer proper depth information of a structure. For example, two images are obtained for the same structure on a substrate, a first image at a normal beam incidence and a second image with the beam incident at a tilted angle. The use of multiple images for 3D metrology has several limitations. For example, capturing a pair of stereo images decreases a throughput of a patterning process or a metrology process due to having to switch between beam tilt angles. A proper alignment between the images is required to capture the stereo images. Processing the stereo images to determine depthinformation may be computationally expensive and prone to noise in the images and drifts in a metrology hardware. Capturing the stereo images require the metrology hardware to facilitate ease-of- use in changing the tilt direction of the beam. Capturing image at a tilted angle reduces an effective field-of-view, because only areas that are present in all images can be used for depth inference.

[0072] In some embodiments, the present disclosure describes mechanisms to estimate depth of a structure using a single image (e.g., a SEM image) captured at a first tilt angle (e.g., perpendicular to a substrate). As such, this potentially relieves most of the disadvantages associated with stereo images mentioned above. For example, once a model is trained according to the present disclosure, depth information can be determined using a single SEM image thereby increasing the throughput, reducing metrology time, and reducing computational resources compared to using stereo image based methods for determining depth information.

[0073] In some embodiments, the present disclosure describes an unsupervised training method used for training a machine learning model (e.g., CNN). In some embodiments, the term “unsupervised” refers to a training approach where a ground truth depth data is not used when training the model or manual interventions is not involved during the training process. The approach herein has sufficient capability by itself to learn how to perform depth estimation. In some embodiments, a convolutional neural network (CNN) with a U-net-like architecture (e.g., an encoder, a decoder, and skip connections between the two) may be employed for training the CNN.

[0074] Figure 3 is a block diagram of a training process of a model M0 according to some embodiments of the present disclosure. After training, a trained model may be referred as a model M1. As depicted in Figure 3, a model M0 (e.g., CNN) receives an image 301 of a substrate as input, and outputs disparity data 311 associated with the inputted image. The inputted image 301 may be either a normal SEM image or a tilted SEM image of a patterned substrate. In the present embodiment, the term “normal image” or “normal SEM image” refers to a top-view image of a substrate that captures a first topology of the structure viewed from a top. In some embodiments, the normal image can be obtained via a metrology tool configured to direct e-beam perpendicular to the substrate. The term “tilted image” or “tilted SEM image” refers to an angled-view image of the substrate that captures a second topology of the structure viewed at an angle with respect to perpendicular to the substrate. In some embodiments, tilted image can be obtained via the metrology tool configured to direct an e-beam at an angle with respect to a substrate.

[0075] In Figure 3, the model M0 is configured to output data that characterizes a difference between two images of the same structure. From such difference data, depth information of the structure can be determined. In some embodiments, the model M0 generates disparity data 311 that can be converted to depth information 315 of the structure on the substrate. For example, the disparity data 311 can be converted to depth information 315 by multiplying the magnitude of the disparity data 311 with a conversion function such as a scaling factor k. In some embodiments, the conversion function may be a linear function, a non-linear function, or a constant determined by comparing theresult of applying the conversion function with one of the two images. In Figure 3, the disparity data 311 is visually represented as a disparity map or image, as an example. In the present disclosure, disparity data 311 refers to a difference in coordinates of similar features within two stereo images (e.g., two SEM images of the same structure). In some embodiments, a magnitude of displacement vectors in the disparity map, is directly proportional to a depth of a structure.

[0076] For training the model M0, the disparity data 311 may be transformed, via a transform operation T1, to another image 321 that correspond to the input image 301 of the model M0. The transform operation T1 represents any transformation function that modified a tilted image 302 using the disparity data 311. For example, the transformation function can be composition, or convolution operating between the disparity data 311 and the tilted image 302 that results in another image that should correspond to the normal image.

[0077] In the present example, the disparity data 311 is combined with the tilted image 302, measured at a specified beam tilt (other than normal), to map the pixels of the tilted image 302 with the normal SEM image. In some embodiments, the tilted image 302 may be represented as a function ^ and the disparity data 311 (e.g., a map) may be represented as another function ^. By combining the disparity data 311 with the tilted image 302, a reconstructed image may be obtained. For example, the reconstructed image is represented by a function obtained from a composition of functions ^ and ^. In some embodiments, the reconstructed image may be represented as ^ ∘ ^, where the symbol ∘ denotes the composition operation between functions e.g., ^(^) ∘ ^(^) = ^(^(^)), ^ being a vector of xy-coordinates of an image. If the estimated disparity data 311 is accurate, then the reconstructed image is expected to be very similar to the inputted SEM image. For example, the reconstructed image is more than 95% similar to the normal SEM image inputted to the model M0. If the reconstructed image is not similar, the model M0 is modified or trained to cause the reconstructed image to be similar to the inputted SEM image. The modification of the model M0 may involve adjusting of one or more parameters (e.g., weights and biases) of the model M0 until a satisfactory reconstructed image is generated by the model. In some embodiments, the adjustment of the one or more parameters is based on a difference between the reconstructed image and the inputted SEM image. In some embodiments, a performance function may be used to guide the adjustment of the one or more parameters of the model M0, the performance function being a difference between the reconstructed image and the inputted SEM image.

[0078] In some embodiments, training of the model M0 involves adjusting model parameters to cause minimization of a performance function. At the end of the training process, the model M0 is referred as the model M1 or the trained model M1. In some embodiments, the adjusting comprises determining a gradient map of the performance function by taking a derivative of the performance function with respect to the one or more model parameters. The gradient map guides the adjustment of the one or more model parameters in a direction that minimizes or brings the performance functionvalues within a specified threshold. For example, the specified threshold may be that the difference of the performance function values between a current iteration and a subsequent iteration is less than 1%.

[0079] In some embodiments, the performance function comprises a similarity loss ^^^^(^, ^ ∘ ^) indicative of similarity between the reconstructed image and the inputted SEM image (e.g., represented as a function ^). In some embodiments, the similarity loss function may be computed for a plurality of images of the structure, each image obtained at different angles. For example, the performance function may be modified to include a loss function computed as a sum of similarity between an image of the plurality of images and a corresponding reconstructed image.

[0080] Additionally, the performance function may include another loss function ^^^^^^(^)determined based on prior information about the disparity characteristic of a pair of stereo SEM images. The prior information includes, but is not limited to, disparity being characterized as piecewise smooth function, disparity being characterized as piecewise constant function, or disparity being characterized as a function allowed to jump at edges of features in the normal image. For example, the edges of the features may be detected applying a gradient operation to an image (e.g., the normal image). The gradient operation identifying a sharp change in slope of an intensity profile of the image (e.g., normal image) at one or more locations, such location may be characterized as edges at which disparity may jump from one function type to another.

[0081] As mentioned earlier, based on prior stereo images of one or more prior patterned substrate a disparity function may be determined. For example, the disparity may be a piecewise smooth function, wherein a derivative of the disparity is piecewise continuous. For example, disparity related to structures having non-vertical walls will be piecewise continuous. In another example, the disparity may be piecewise constant. For example, disparity related to structures having vertical walls will be piecewise constant. In yet another example, the disparity may be a function having a jump at edges of a structure within an image, the edges being detected based a gradient of an intensity profile within the image. For example, disparity related to structures having both vertical and non-vertical walls, where the location of the jumps is per-determined from the SEM images. Thus, the present method enables incorporation of disparity data from prior patterned substrate in form of the disparity function to train the machine learning model. Use of such disparity function enables faster convergence or a faster training of the model M0 along with more accurate results.

[0082] In some embodiments, the model M0 (e.g., CNN) may be trained based on the performance function expressed as a sum of similarity loss function and another loss function (see equation below) associated with prior patterned substrates. In an example, a performance function ^ is minimized during training by modifying one or more model parameters of the CNN, as expressed below: ^ = argmin ^ ^^^^(^^, ^^∘ ^(^^, ^^, ^)) + ^^^^^^(^(^^, ^^, ^)) ^ ^

[0083] According to the present disclosure, referring to Figure 4, inventors identified, based on an experiment, that depth information of a structure patterned on a substrate can be inferred from a disparity between two or more SEM images captured at different tilt angles. For example, the experiment involved generating a depth map and feed it into a Monte-Carlo SEM simulator to compute a stereo pair of denoised SEM images: one at 0 degrees from the normal (top-view image) and one at 5.7 degrees from the normal (tilted image). An example Monte-Carlo simulator configured to generate SEM images is discussed in “L. van Kessel, and C.W. Hagen, Nebula: Monte Carlo simulator of electron–matter interaction, SoftwareX, Volume 12, 2020, 100605, ISSN 2352-7110.” In the experiment, in two SEM images, a set of points were identified and related disparity data was analyzed. For example, the set of points corresponding to step transitions in a height profile of a structure that results in a peak in the SEM signal. For the set of points, disparity data (e.g., displacement in the x-coordinate) of each point between the two SEM images was calculated. The height or depth of each point is plotted against the disparity, as shown in Figure 4. Based on a line 401 fitted between the disparity data and the height at the set of points, it can be concluded that there is a strong correlation between the disparity between two SEM images and the height of the structure.

[0084] In some embodiments, once the model M1 is trained predictions of depth information may be performed based on a single image (e.g., SEM image) of any patterned substrate. Figure 6A illustrates estimating of height or depth information associated with a structure using a single SEM image 351. In Figure 6A, the model M1 (e.g., trained as in Figure 3) uses a normal SEM image 351 of the structure as input to estimate disparity data 355. The disparity data 355 is further converted, by applying a conversion function k to the disparity data 355, to height data 360 of a structure patterned on the substrate. For example, the magnitude of the estimated disparity data can be multiplied by a constant to generate the depth information. As such, no tilted image is required to infer height information of the structure.

[0085] Figure 5 is an exemplary flow chart of a method 500 for determining a model for estimating depth information of a structure of a patterned substrate. In an example implementation, the method 500 includes following processes P502, P504, P506 and P508, discussed in detail below.

[0086] Process P502 involves obtaining a pair of images IM1 and IM2 of a structure of a patterned substrate. The pair of images includes a first image IM1 captured at a first angle with respect to the patterned substrate, and a second image IM2 captured at a second angle different from the first angle. As an example, the first image IM1 captures top view details of the structure, and the second image IM2 captures details of an angle-view details of the same structure from another angle (e.g., at an angle between 1 to 15º from a perpendicular to the substrate). An example of the first image IM1 can be a normal image 301 (see Figure 3) and the second image IM2 can be a tilted image 302 (see Figure 3). In some embodiments, the first image IM1 is a normal image associated with an e-beam directed perpendicular to the patterned substrate, and the second image IM2 is associated with an e-beam directed at an angle more 90º or less than 90º with respect to the patterned substrate. In someembodiments, a plurality of pair of images may be obtained and used as training dataset. In some embodiments, the pair of images obtained, via a metrology tool, includes a plurality of pairs of SEM images of a patterned substrate. Each pair including a first SEM image associated with a first e-beam tilt setting of the metrology tool, and a second SEM image associated with a second e-beam tilt setting of the metrology tool. In some embodiments, each image of the pair of images is captured from different SEM tools. For example, the first SEM image is captured by a first SEM tool configured to capture an normal image of a structure, and the second SEM image is captured by a second SEM tool configured to capture an image of the same structure at a second e-beam tilt setting.

[0087] Process P504 involves generating, via a model M0 using the first image IM1 as input, disparity data DD1 between the first image IM1 and the second image IM2, the disparity data DD1 being indicative of depth information associated with the first image IM1. In some embodiments, the disparity data DD1 comprises difference in coordinates of similar features within the first image IM1 and the second image IM2. In some embodiments, after the training process, the model M1 is considered as a trained model of M0. In some embodiments, the model M0 or M1 may be a machine learning model configured to predict disparity data using a single image of a substrate. For example, the model M0 or M1 may be a convolutional neural network CNN, deep CNN, or other machine learning model.

[0088] Process P506 involves applying the disparity data DD1 to the second image IM2 to generate a reconstructed image corresponding to the first image IM1. In some embodiments, the reconstructed image is generated by performing a composition operation between the disparity data DD1 and the second image IM2 to generate the reconstructed image. For example, Figure 3 illustrates the reconstructed image 321 generated based on the disparity data (represented as the disparity map 311) and the tilted image 302.

[0089] Process P508 involves adjusting, based on a performance function one or more parameters of the model M1 causing the performance function to be within a specified performance threshold. The performance function may be a function of the disparity data DD1, the reconstructed image, and the first image IM1. The model M1 is configured to generate data convertible to depth information of a structure of a patterned substrate.

[0090] In some embodiments, the performance function further comprises a loss function computed based on disparity characteristics associated with a pair of stereo images of prior one or more patterned substrate, and the disparity data DD1 predicted by the model M1. In an example, the disparity characteristics may include disparity characterized as a piecewise smooth function, wherein a derivative of the disparity is piecewise continuous. In another example, the disparity characteristics may include disparity being piecewise constant. In yet another example, the disparity characteristics may include disparity characterized as a function having a jump at edges of a structure within an image, the edges being detected based a gradient of an intensity profile within the image. An example performance function ^ is discussed above with respect to Figure 3.

[0091] In some embodiments, the training is based on a plurality of images of the structure obtained at different angles. Accordingly, the performance function may include a loss function related to a sum of similarity between an image of the plurality of images and a corresponding reconstructed image.

[0092] In an example, the performance function is minimized during training by modifying one or more model parameters of the CNN. For example, the performance function L computed as a sum of loss function related to reconstructed image and the first image, and disparity characteristic of prior stereo images of prior patterned substrate. For example, the performance function can be computed as ∑^^^^^(^^, ^^∘ ^(^^, ^^, ^)) + ^^^^^^(^(^^, ^^, ^)) , as discussed earlier. parameters of the model M0 is an iterativeprocess, function based on the disparity data DD1, and the reconstructed image. The iteration further includes determining whether the performance function is within the specified performance threshold; and in response to the performance function not being within the specified difference threshold, adjusting the one or more parameters of the model M0 to cause the performance function to be within the specified performance threshold. The adjusting may be based on a gradient of the performance function with respect to the one or more parameters. Once the model M0 is trained, the model M1 can be applied to determine depth information of any structure based on a single image (e.g., a normal SEM image) of any patterned substrate.

[0094] In some embodiments, the method 500 may further include steps for applying the trained model M1 during measurement or inspection of a patterned substrate. In some embodiments, the method 500 may include a process for obtaining, via the metrology tool, a SEM image of a patterned substrate at the first e-beam tilt setting of the metrology tool. The SEM image may be a normal image obtained by directing an e-beam approximately perpendicular to the patterned substrate. The method 500 may further include executing the model M1 using the SEM image as input to generate disparity data associated with the SEM image; and applying a conversion function (e.g., a linear function, a constant conversion factor, or a non-linear function) to the disparity data to generate depth information of the structure in the SEM image. The method 500 includes determining, based on the depth information, physical characteristics of the structure of the patterned substrate. In some embodiments, the physical characteristics may include a shape, a size, or relative positioning of polygon shapes with respect to each other at one or more depths of features of the structure.

[0095] In some embodiments, the processes described herein may be stored in the form of instructions on a non-transitory computer-readable medium that, when executed by one or more processors, cause steps of the present methods. For example, the medium includes operations including receiving, via a metrology tool (e.g., SEM tool), a normal SEM image of a structure on a patterned substrate. The normal SEM image being associated with an e-beam directed perpendicular to a pattered substrate. Further, the medium includes operations to execute a model using the normalSEM image to determine depth information of the structure on the patterned substrate. As discussed herein, the model may be trained to estimate depth information from only a single SEM image and stored on the medium. For example, a model M1 (e.g., a CNN) may be trained using the method of Figure 5.

[0096] In some embodiments, the medium is further configured to determine, based on the depth information, physical characteristics of the structure of the patterned substrate. For example, the physical characteristics include, but not limited to, a shape, a size, or relative positioning of polygon shapes with respect to each other at one or more depth of features of the structure.

[0097] In some embodiments, the medium is further configured to determine, based on the physical characteristics, a defect in the structure, the defect indicative of the structure not satisfying a design specification. Based on the defect, a parameter of one or more patterning process (e.g., resist or etching related parameters) may be adjusted to eliminate the defect in a subsequent process of the patterning process.

[0098] Figure 6B is a flow chart of a method 600 for determining depth information of a structure formed on a substrate using a model trained according to Figure 5, according to some embodiments. The depth information is determined based on a scanning electron microscope (“SEM”) image of the structure. The depth information is determined using a convolutional neural network (CNN) trained to mimic a parallax effect present in pairs of stereo SEM images. The method 600 includes processes P602, P604, and P606. Process P602 involves receiving, via a SEM tool, a SEM image SEM1 of a structure patterned on a substrate.

[0099] Process P604 involves inputting the SEM image SEM1 to a CNN1 (an example of the model M1 trained according to the method 500) to predict disparity data 605 associated with the SEM image SEM1. In some embodiments, the CNN1 may be trained according to Figure 5. For example, the CNN1 may be trained by: obtaining, via the SEM tool, a stereo pair of SEM images of a patterned substrate, the stereo pair including a first SEM image obtained at a first e-beam tilt setting of the SEM tool, and a second SEM image obtained at a second e-beam tilt setting of the SEM tool; generating using disparity data between the first SEM image and the second SEM image; combining the disparity data with the second SEM image to generate a reconstructed image of the first SEM image; and comparing the reconstructed image and the first SEM image.

[0100] Process P606 involves generating, based on the predicted disparity data 605, depth information 610 associated with the structure patterned on the substrate.

[0101] In some embodiments, the depth information can be further used to determine a defectiveness of structure by comparing CDs of the features at different depths and determining whether the CDs satisfy design specification. In some embodiments, the depth information may be used to determine one or more parameters of a patterning process such as resist process, etching process, or other patterning related process so that the 3D structure of the feature is within desired design specification.

[0102] In some embodiment, methods and systems are described herein for semi-supervised training of depth estimation model for structures formed on substrates and employing the trained model to predict depth data from a single SEM image. In some systems, to obtain depth information related to a structure, two images are obtained for the structure, a first at a normal beam incidence and a second with the beam at a tilt. However, adjusting alignment, in addition to being difficult and imprecise, is time consuming, which negatively affects manufacturing throughput.

[0103] The present disclosure overcomes these problems by using a machine learning model such as a convolutional neural network (“CNN”) that uses a SEM image as input, to predict depth data associated with the SEM image. However, the use of a machine learning model poses an additional technical problem in that training data may be sparse and not properly diverse, resulting in poor predictions. To overcome this problem, the machine learning model described herein is trained to predict depth data for inputted SEM images based on plurality of simulated SEM images that are labeled with simulated depth data. For example, by using simulated depth data the system may fully control the amount and diversity.

[0104] While the use of simulated SEM images overcomes the technical problem of sparse training data, and using the simulated SEM images as input to the model provided qualitatively reasonable depth information, the depth information still does not typically match observed depth information. To overcome this second technical issue, the machine learning model as described herein comprises a second supervised calibration step. In particular, the machine learning model is calibrated to scale predicted depth data to correspond to observed depth data, as determined using a SEM tool, of actual SEM images. For example, to correct for this issue, the system scales predicted depth information to the depth information observed using a SEM tool or other metrology tool, and uses the scaled predicted depth information as a new ground truth for the subsequent training procedure.

[0105] Figure 7 is a flow chart of a method for generating a model 710 configured to estimate depth data of a structure of a patterned substrate, according to some embodiments. In some embodiments, the method 700 is a semi-supervised training of depth estimation model 710 for structures formed on substrates from SEM images of the structures. For example, in the semi- supervised training method, initially the depth estimation model 710 is trained using simulated data, further observed depth data of structures is used to modify model-predicted depth data, and the modified depth data is used as reference to further fine-tune the depth estimation model 710. The method 700 is further discussed in detail with respect to process P702, P704, P706, and P710.

[0106] Process P702 includes obtaining, via a simulator 701, a plurality of simulated metrology images 703 of a structure. Each of the plurality of simulated metrology images 703 is associated with depth data used by the simulator 701. For example, depth data used in the simulator 701 (also referred as simulator depth data) may be a shape, a size, a side wall angle, relative positioning of polygon shapes, a material, or depth related parameter associated with each layer of the substrate. The simulator 701 is configured to perturb the simulator depth data to generate simulated metrologyimages. For example, the simulator depth data that can be adjusted includes, but not limited to the shape of a resist profile of the structure in a resist layer, and the side wall angle of the resist profile. For each adjusted simulator depth data, a corresponding metrology image (e.g., SEM image) may be generated.

[0107] In some embodiments, the simulator 701 is a Monte-Carlo simulator configured to generate the simulated metrology images 703 of the structure by varying depth related parameters defined in the Monte-Carlo simulator. An example Monte-Carlo simulator configured to generate SEM images is discussed in “L. van Kessel, and C.W. Hagen, Nebula: Monte Carlo simulator of electron–matter interaction, SoftwareX, Volume 12, 2020, 100605, ISSN 2352-7110.” The Monte-Carlo simulator is provided as an example, but the present disclosure is not limited to a particular simulator.

[0108] Process P704 includes generating, based on the plurality of simulated metrology images 703 and the corresponding simulated depth data, a model M70 configured to predict depth data from an inputted image.

[0109] In some embodiments, generating the model M70 is an iterative process. Each iteration includes predicting depth data, comparing the predicted data with the simulator data, and generating the model based on the comparison. For example, the process P704 includes inputting the plurality of simulated metrology images 703 to the model M70 to predict depth data associated with each of the plurality of simulated metrology images 703. The predicted depth data is compared with the simulator depth data. Based on the comparison, model parameters of the model M70 are adjusted so that the predicted depth data is within a specified matching threshold of the simulator depth data. For example, the predicted depth data has more than 90% matching with simulator depth data. For example, a height profile of the predicted depth data and the height profile of the simulator depth data are more than 90% similar. The model M70 is further fine-tuned based on observed data, which is a part of the semi-supervised training.

[0110] Process P706 includes obtaining a captured image 707 and observed depth data 708 of the structure patterned on a substrate. In some embodiments, the captured image 707 is obtained via an image capturing tool, and the observed depth data 708 are obtained from one or more metrology tools. In some embodiments, the image capture tool is a SEM tool and the captured images are SEM images. In some embodiments, the observed depth data is obtained from the metrology tool. The metrology tool being one or more of: an optical metrology tool configured to measure structure of the patterned substrate and extract depth information based on diffraction-based measurements of the patterned substrate, or an atomic force microscope (AFM). For example, the observed depth data 708 comprises height profile of the structure captured by an atomic force microscope tool (AFM), or shape parameter data captured by an optical scatterometry tool (e.g., Yieldstar). In some embodiments, the metrology tool is an optical metrology tool configured to measure structure of the patterned substrate and extract depth information based on diffraction-based measurements of the patterned substrate.

[0111] In some embodiments, the observed depth data includes one-dimensional height data of the structure traced from the captured image 707. Similarly, simulator depth data may include one- dimensional height data determined from simulated image of the structure on the patterned substrate. In some embodiments, the observed depth data includes two-dimensional height data of the structure traced from the captured image 707. Similarly, simulator depth data may include two-dimensional height data extracted from the simulated image of the structure on the patterned substrate. In another embodiment, the observed depth data includes shape parameters obtained from the optical metrology tool used to measure structure of the patterned substrate. Similarly, shape parameters may be extracted from the simulator depth data. As an example, one-dimensional height data includes height profile (e.g., height profile 3DD1 illustrated in Figure 8B) of the structure along a cut line of the captured image 707. In some embodiments, the two-dimensional height data comprises the height of the structure along a first direction and a second direction on the captured image 707. In some embodiments, the shape parameters comprises one or more of a top CD measured at a top of the structure, a bottom CD measured at a bottom of the structure, or a side wall angle of the structure. For example, shape parameter data 3DD2 is illustrated in Figure 8B.

[0112] Process P710 includes calibrating, based on the captured image 707 and the observed depth data 708, the model M70 to cause the predicted depth data to be within a specified matching threshold of the observed depth data 708. Upon calibration, the model M70 may be referred as the model 710, a trained model 710, or a calibrated model 710.

[0113] In some embodiments, calibrating the model M70 is an iterative process. Each iteration includes adjusting the predicted depth data in accordance with the observed data to generate modified predicted data for training. For example, each iteration includes inputting the captured image 707 to the model M70 to predict depth data. The predicted depth data is adjusted by comparing the predicted depth data and the observed depth data 708. Based the adjusted predicted depth data, model parameters of the model M70 may be adjusted to cause the model M70 to generate depth data that is within a matching threshold of observed depth data 708.

[0114] In some embodiments, adjusting of the predicted depth data includes: extracting, from the predicted depth data, one dimensional height profile of the structure along a given direction; comparing the predicted height profile with one dimensional height profile of the observed depth data 708 of the structure along the given direction; and modifying the predicted height profile to match the height profile of the observed depth data 708 of the structure.

[0115] In some embodiments, adjusting of the predicted depth data includes: extracting predicted shape parameters of the structure from the predicted depth data, and real shape parameters from the observed depth data 708; comparing the predicted shape parameters with the real shape parameters of the structure; and modifying the predicted shape parameters to match the real shape parameters.

[0116] In some embodiments, adjusting of the predicted depth data includes: deriving a predicted average height of the structure from the predicted depth data of the structure, and a real average heightof the structure from the observed depth data 708; and scaling the predicted average height to match the real average height. For example, a scaling factor (e.g., a factor SF1 in Figure 8C) computed as a ratio of an average height computed from the predicted depth data and an average height obtained from the optical scatterometry tool (e.g., Yieldstar).

[0117] Figure 8A illustrates an example of generating an initial model M70 using simulated metrology image and simulated depth data (e.g., generated via a Monte-Carlo SEM simulator). In Figure 8A, a convolutional neural network M70 (e.g., with a encoder-decoder, U-net, or Res-net architecture) is trained based on a sufficiently large dataset generated from a Monte-Carlo SEM model. The advantage of using simulated data at this step is it provides full control over the amount and diversity (e.g. SEM settings, feature geometries and feature materials) of training examples.

[0118] As an example, a synthetic or simulated SEM image SSEM1 generated by the SEM model and corresponding simulator data SDEP can be used to train the initial CNN M70. In some embodiments, the CNN M70 predicts depth data, which is compared with the simulated data SDEP. Based on the comparison, one or more weights and biases of one or more layers of the CNN are adjusted so that the predicted data matches the simulated data SDEP. In some embodiments, a difference between the predicted depth data and depth data extracted from the simulated data SDEP may be computed. During the training of the CNN M70, in each iteration, the weights and biases are modified so that the difference between predicted and simulated data is reduced. In some embodiments, the training processes terminates when the difference cannot be reduced further in subsequent iterations. At the end of the training the model M70 is sufficiently trained to predict depth data from an input SEM image. However, as mentioned earlier, the predicted data by the CNN M70 may not accurately correspond to observed data. As such, the CNN M70 is further fine-tuned as discussed with respect to Figures 8B and 8C.

[0119] Figure 8B illustrate example of observed depth data obtained from different metrology tools (e.g., AFM, SEM, or Yieldstar) that can be used to fine-tune the model CNN M70. In the present example, a subset of paired experimental SEM images SEM80 and metrology depth data 3DD1 or 3DD2 (e.g., from metrology tool Yieldstar or AFM) is generated. The experimental SEM images SEM80 as input to the CNN M70 to provide qualitatively reasonable depth data 3DM such as height maps. However, in some embodiments, predicted height maps 3DM may not match the observed height maps 3DD1 or 3DD2. For example, the height map 3DD1 is a 1D height map across a cut line of an AFM data, and height map 3DD2 is a 2D height map obtained from optical scatterometry tool (e.g., Yieldstar). To correct for this, the predicted height maps may be scaled to the observed height data 3DD1 or 3DD2. The scaled predicted height maps serve as new ground truth for the subsequent training procedure.

[0120] In some embodiments, the CNN M70 may be layers that may for instance include convolutions with kernels that model the finiteness of an AFM tool tip. In some embodiments, additional layers corresponding to a metrology tool may be augmented in the CNN M70. Thus, duringfine-tuning model parameters associated with such particular layers augmented in the CNN M70 may be adjusted. In some embodiments, the experimental dataset will typically be limited by the limitation of the metrology tool (e.g., AFM or YS), as such only a subset of the degrees of freedom (e.g., layers) of the CNN M70 is optimized while the rest of the layers are kept unchanged. In one embodiment, a correction bias in a latent space variable may be employed in training CNN M70. For example, the latent space variable ^ may be computed as follows: ^^= ^ + Δ^,

[0121] In the above equation, Δ^ is a vector of the same dimension as ^. Δ^ (or the norm thereof) can be used as a measure for the difference between the synthetically trained model (e.g., CNN M70) and the calibrated model (e.g., 710). Ideally, this distance will be relatively small. In another embodiment, the weights and biases of last encoder layer are used as degrees of freedom.

[0122] Figure 8C illustrate an example adjustment of the predicted depth data based on the observed data from an optical metrology tool (e.g., Yieldstar) configured to measure a patterned substrate. In the present example, an experimental SEM image SSEM2 is input to the CNN M70 to generate a qualitatively reasonable height map PDD2, but an average height does not match the observed height YSh provided by an optical tool (e.g., Yieldstar). To correct for this, the predicted height map PDD2 may be scaled using a scaling factor SF1 to the height observed YSh. For example, the scaling factor may be computed as a ratio of an average height of the height observed YSh and an average height of the structure computed from the predicted height map PDD2. Multiplying the predicted height map PDD2 with the scaling factor SF1 generates a scaled predicted height map SPD2, which is used as new ground truth for the subsequent training procedure to generate a fully trained CNN 710 (e.g., in Figure 8B).

[0123] In some embodiments, a get / set experiment results show that predicted depth data using the calibrated model 710 closely follow programmed depth data used in an experiment. For example, the depth of individual contacts holes predicted by the model 710 (get depth) closely follows the depth programmed in the geometry (set depth) that was used to generate the simulated SEM image. Similarly, extracted side-wall angle of edges from simulated data closely follow the simulated SEM image. In some embodiments, brightness and contrast settings in simulated images may be varied during the training process of the model 710, so that predictions from the deep-learning network are independent of brightness and contrast settings.

[0124] The calibrated model 710 is capable of handling various physical effects caused during SEM image capture or during a patterning process. For example, the model 710 is capable of making good predictions even in a presence of a charging effect in a SEM image, as illustrated in Figures 9A- 9C.

[0125] Figure 9A is an exemplary SEM image 902 including charging artifacts. For example, in the SEM image 902 of line-space features, SEM charging artifacts are seen just right of trenches where the SEM signal is reduced compared to at the edge of the trench. When such SEM image 902 having charging effect is inputted to the initial model M70 (e.g., in Figure 7), the predicted height maps are affected by this reduction in SEM intensity, as shown (see right of the trenches) in the predicted height map 904 in Fig.9B. To correct for this, a fine-tuning or further calibration step was employed in the training process of the model M70, as discussed earlier. In such calibration step, the predicted data is adjusted to assume the lines for the new training set to have a flat surface. When such adjusted predicted data is used to generate calibrated model 710 (e.g., in Figure 7), the predicted height maps (by the model 701) do not suffer from charging anymore, but do have a flat surface on top of the lines, as shown the predicted height map 906 in Figure 9C. This example serves as a proof-of-principle that, in some embodiments, a deep-learning network (e.g., CNN) is taught to recognize artifacts not present in the simulated dataset, and perform a non-linear correction based on the observed data.

[0126] Figure 10A is a flow chart of a method for estimating depth data by using the trained model 710 (of Figure 7) using a single SEM image as input. The method 800 includes processes P802, and P804. In some embodiments, the process P802 includes receiving a SEM image SEM1 of a structure patterned on a substrate. In some embodiments, process P804 includes predicting, via a the CNN (e.g., the model 710) using the SEM image as input, depth data 810 associated with the SEM image. As discussed earlier, the CNN (e.g., the model 710) may be trained to predict depth data for inputted SEM images based on plurality of simulated SEM images that are labeled with simulated depth data; and further calibrated using scaled predicted depth data, where the scaled predicted data correspond to observed depth data, as determined using a SEM tool, of captured SEM images of the structures patterned on the substrate.

[0127] For example, training of the CNN 710 includes receiving, via a SEM tool, a captured SEM image and observed depth data associated with the captured SEM image of the structure patterned on the substrate; inputting the captured SEM image to the model to predict depth data; adjusting the predicted depth data by comparing the predicted depth data and the observed depth data; and tuning, based on the adjusted predicted depth data, the model to cause the predicted depth data to be within a specified matching threshold of the observed depth data.

[0128] In some embodiments, the estimated depth data comprises physical characteristics of the structure of the patterned substrate. For example, the physical characteristics comprising a shape (e.g., resist profile), a size, a side wall angle (e.g., an angle of the resist profile), or relative positioning of polygon shapes with respect to each other at one or more depth of features of the structure.

[0129] Figure 10B illustrates example of using the trained model 710 to predict depth data 825 (e.g., a height map) using a SEM image 821 as input. As shown, the model 710 predicts 3D information from the 2D data in the SEM image.

[0130] In some embodiments, instead of using a SEM simulator to train a depth estimation model with paired data, a calibrated deterministic or stochastic process model (e.g., resist model) may be employed to generate training data. In case an accuracy of the process model may be insufficient, the process models may be fine-tuned with a limited set of experimental depth data (e.g. from AFM or Yieldstar). In some embodiments, depending on the type of the process model (deterministic or stochastic), two separate methods are described herein.

[0131] In some embodiments, using a resist model instead of a SEM model for generating training data (e.g., depth maps and corresponding SEM images) has several advantages. For example, the SEM images corresponding to simulated resist profiles are experimental images, while the SEM images generated by the SEM model are synthetic. A process model (e.g., a resist model) can be well calibrated to wafer data, either by including or not a SEM simulator.

[0132] Figure 11 is a flow chart of a method 1100 for generating a model configured to estimate depth data of a structure of a patterned substrate. For example, depth data of structures including one or more features such as lines, bars, contact holes, other features or a combination of features. In some embodiments, the model is trained using training data generated as a pair of SEM images and corresponding simulated 3D profiles that may be formed on a substrate. The simulated profiles may be generated via a calibrated process model (e.g., a calibrated resist model). In some embodiments, the calibration is performed to meet a critical dimension criteria associated with the structures. In some embodiments, the process model calibration may involve comparing height of 3D simulated geometry to height extracted from atomic force microscopy (AFM) data or optical metrology data. In some embodiments, the model is a CNN trained using a generative adversarial network (GAN), encoder-decoder network, or other machine learning related training methods. For example, the model includes a first model (e.g., a generator model or encoder model) and a second model (e.g., discriminator model or decoder model) that are trained together. In some embodiments, both the first model and the second model may be CNN models. An example implementation of the method 1100 includes process P1102 for obtaining training data and process P1104 for generating the model based on the training data. The processes P1102 and P1104 are discussed in detail below.

[0133] Process P1102 includes obtaining (i) a plurality of SEM images 1101 of structures associated with programmed variations PV1 in a mask pattern and (ii) simulated profiles 1103 of the structures based on the programmed variations PV1, each SEM image of the plurality of SEM images 1101 being paired with a simulated profile corresponding to a programmed variation in the mask pattern.

[0134] In some embodiments, the programmed variations PV1 includes variations in one or more of: assist feature variations associated with the mask pattern, main feature variations associated with the mask pattern, or resist coated thickness variations. In some embodiments, the assist feature variations includes modifying a size of an assist feature of the mask pattern, a distance of the assist feature to a main feature of the mask pattern, or both. In an example, a first programmed variationmay be change in size of the assist feature, and a mask including the first programmed variation may be manufactured. Employing the mask having the first programmed variation, a substrate may be patterned (e.g., via a lithographic apparatus). Further, a SEM image of the patterned substrate may be captured via a SEM tool. As such, the captured SEM image includes features related to the first programmed variation in the mask. In another example, a second programmed variation in the mask pattern may be modification of a shape of a main feature of the mask pattern. A substrate may be patterned using a mask having the second programmed variation, and another SEM image of the patterned substrate may be captured so that the SEM image includes variations related to the second programmed variation. Such SEM images corresponding to the first, second, third, etc. programmed variations PV1 may be obtained and included in the training data.

[0135] In some embodiments, the training data may further include the simulated profiles 1103 of the structures. In some embodiments, the simulated profiles 1103 may be 3D profiles (e.g., resist profiles) that may be formed on the substrate. Such 3D profiles may be generated by a process model (e.g., resist model). In some embodiments, the simulated profiles 1103 are generated by a calibrated deterministic process model DPM associated with a patterning process using the programmed variations PV1 in the mask pattern. For example, the first programmed variation (e.g., change in assist feature size), the second programmed variation (e.g., change in shape of the main feature), or other programmed variation may be inputted to the calibrated process model resulting in the simulated profiles 1103. Each of these simulated profiles 1103 is a 3D profile may include variations in 3D profile caused by a corresponding programmed variation in the mask pattern. As such, the SEM images 1101 and the simulated profiles 1103 may be paired according to the programmed variations PV1.

[0136] In some embodiments, the calibrated deterministic process model DPM is a process model calibrated using inspection data of patterned substrate (also referred as wafer data) such that the simulated process model generates simulated 3D profiles of the structures that is within a desired range of measured critical dimensions (CD) of structures of the patterned substrate. In some embodiments, the deterministic process model DPM may not be calibrated to meet local CD uniformity, line edge roughness (LER), line width roughness (LWR) related to line space features, contact edge roughness related to contact hole features, or stochastic variations associated with the structure. In other words, the calibrated deterministic process model DPM may simulate profiles of the structure that meet CD requirements. In another embodiment, a process model may be calibrated to meet the LCDU, LER, LWR, or other stochastic variations associated with physical characteristics of the structure.

[0137] Process P1104 includes generating, based on the plurality of SEM images 1101 paired with the corresponding simulated profiles 1103, a model M110 to estimate depth data of a structure such that the estimated depth data is within acceptable threshold of depth data associated with thesimulated profiles 1103. For example, the model M110 may be trained such that the estimated height map is more than 90% similar to the height map extracted from the simulated profiles 1103.

[0138] In some embodiments, the generated depth data may include height data of the structure, shape parameters of the structure, voxel map of overhanging features, or other depth characteristic of the structure. In some embodiments, the shape parameters comprises one or more of a top CD measured at a top of the structure, a bottom CD measured at a bottom of the structure, or a side wall angle of the structure.

[0139] In some embodiments, generating of the model M110 may be based on a generative adversarial network architecture, encoder-decoder network, Res-net, or other machine learning architectures. For example, conditional GANs learn a mapping from observed image x (e.g., SEM images 1101) and random noise vector z, to output y (e.g., simulated profiles 1103), which is represented as G : {x, z} → y. In some embodiments, the generator G is trained to produce outputs that cannot be distinguished from reference images (e.g., “real” images) by an adversarial trained discriminator D, which is trained to perform as well as possible at classifying the generated images as “fakes”. For example, in GAN training, G tries to minimize an objective against an adversarial D that tries to maximize it. The generator may be an encoder-decoder, or U-net.

[0140] In some embodiments, generating of the model M110 includes: training a first model (e.g., CNN1) in conjunction with a second model (e.g., CNN2) such that the first model generates depth data using a SEM image of the plurality of image as input, and the second model to classify the generated depth data as a first category (e.g., real) or a second category (e.g., fake) based on the paired simulated profile. In some embodiments, the first model and the second model are convolutional neural network (CNN), or deep CNN. After the training process, the first model (e.g., CNN1) may be employed as the model M110 to generate depth data for any inputted SEM image.

[0141] In some embodiments, generating of the model M110 includes inputting a SEM image of the plurality of SEM images 1101 to the first model; estimating, by executing the first model, the depth data for inputted SEM images; classifying, via a second model using the simulated profiles 1103 as reference, the estimated depth data as a first category (e.g., real) indicative that the estimated depth data correspond to the reference or a second category (e.g., fake) indicative that the estimated depth data does not correspond to the reference; and updating model parameters of both the first model and the second model to cause the first model to estimate depth data such that the second model classifies the estimated depth as the first category. In some embodiments, the model parameters are weights and biases of one or more layers of the CNN or DCNN.

[0142] Figure 12 illustrates an exemplary training of a model 1205 using an encoder decoder architecture and training data including paired SEM image 1202 and simulated profile data 1210. The simulated profile 1210 here is only an example and may not correspond to exactly the same mask pattern as the SEM image neither the simulated profile 1210 is calibrated. In some embodiments, the model 1205 may be trained using GAN architecture. In some embodiments, the simulated profile data1210 represents an averaged 3D resist profile 1210 that describes a mean 3D behavior of inspected features of a patterned structure (e.g., the features may be an array of contact holes). In the present case, the deterministic resist model (an example of the calibrated deterministic process model DPM) may be employed for generating simulated 3D resist profiles 1210 of a structure.

[0143] To introduce local profile variations for training the model 1205, programmed variations in a mask pattern (e.g., sub-resolution assist features (SRAF) variations) within a field of view may be performed to cause variations in the feature profiles (e.g., contact hole profiles) on the substrate. In some embodiments, the programmed variations refer to physical changes made to the mask pattern. Such physical changes to the mask pattern may cause changes the 3D profile patterned on a physical substrate. Thus, for each programmed variation, a corresponding 3D profile may be obtained. In some embodiments, the programmed variations and data generated therefrom is employed as training data for training models discussed herein. In some embodiments, the programmed variations include changes in a size, a distance-to-main-pattern of the SRAF, or both that would affect CD and 3D profile. Based on such programmed variations, patterned substrates may be produced and SEM images of the patterned substrate captured. In some embodiments, the programmed variations in the mask pattern are used for generating training data, and are different from variations in the mask pattern (e.g., OPC related) performed to improve a lithographic process.

[0144] In some embodiments, the programmed variations in the mask pattern may be used in a simulation process of a lithographic process configured to generate simulated 3D profiles 1210. For example, mask pattern data (e.g., in GDS format) can be changed according to the programmed variations and the corresponding mask data (e.g., modified GDS) can be employed in the simulation using the deterministic model DPM to generate simulated 3D profile. For example, the programmed variations such as changes in size, shape, distance-to-main-pattern of the SRAF etc. that are implemented on the physical mask, discussed above, can be used in the simulator to generate corresponding simulated 3D profile. As such, a correspondence between observed data and simulated data may be established based on the programmed variations. In an example, a convolutional neural network (CNN) may be trained for directly mapping the SEM data 1202 to the simulated 3D resist profiles 1210. It can be understood that ADI images are for illustration purposes only.

[0145] Referring back to Figure 11, in some embodiments, the method 1100 may further include a process P1110 for applying the trained model 1110. For example, applying of the model 1110 includes receiving, via a SEM tool, a SEM image of a structure patterned on a substrate; and determining, via the model using the received SEM image as input, depth data associated with the SEM image.

[0146] In some embodiments, based on the depth data, physical characteristics of the structure of the patterned substrate may be determined. The physical characteristics may include a shape, a size, or relative positioning of polygon shapes with respect to each other at one or more depth of features of the structure. In some embodiments, the physical characteristics may be compared with desiredphysical characteristics to determine any defect in the structure. The defect may be indicative of the structure not satisfying a design specification (e.g., CD within desired CD limit). Based on the defect, one or more parameter (e.g., dose, focus, resist characteristics, etc.) of a patterning process may be adjusted to eliminate the defect in a subsequent process of the patterning process.

[0147] Figure 13 is a flow chart of a method 1300 for generating a model M130 configured to estimate depth data of a structure of a patterned substrate. In some embodiments, the model is trained using training data including a plurality of SEM images and a plurality of simulated profiles that are unpaired. For example, a particular SEM image is not paired with a particular simulated profile. In some embodiments, each SEM image of the plurality of SEM images may correspond to one or more simulated profiles. In the present method, the simulated profiles being generated via a calibrated process model (e.g., a calibrated stochastic model) that is configured to account for stochastic variations in physical characteristics associated with a structure. In some embodiments, the method 1300 may include process P1302 for obtaining training data and P1304 for generating the model for estimating depth data. These processes P1302 and P1304 are further discussed in detail below.

[0148] Process P1302 includes obtaining (i) a plurality of SEM images 1301 of structures, (ii) simulated profiles 1303 of the structures, and (iii) key performance indicators (KPIs) associated with the simulated profiles 1303. In some embodiments, the plurality of SEM images may be obtained from an image capture tool such as SEM tool configured to capture an image of structures on a patterned substrate. The SEM images may be 2D images such as a top-view SEM image. In some embodiments, the simulated profiles 1303 of the structures may be 3D profiles of the structures that are generated via a calibrated process model SPM associated with a patterning process. In some embodiments, the plurality of SEM images 1301 and the simulated profiles 1303 are unpaired.

[0149] In some embodiments, the calibrated process model SPM is a process model calibrated to generate 3D profiles of structures such that the 3D structures satisfy on or more KPIs extracted from the 3D structures. In some embodiments, the KPIs may include, but not limited to, critical dimension (CD) of a structure, local CD uniformity (LCDU) associated with the structure, line edge roughness (LER) associated with the structure, defect rates associated with the structure, line width roughness (LWR) related to line space patterns, contact edge roughness related to contact holes, stochastic edge placement error (SEPE) associated with the structures, other stochastic variations associated with geometry of the structure, or a combination thereof. In some embodiments, the calibration may be based on comparison between inspected wafer data and model generated data.

[0150] Process P1304 includes generating, based on (i) the plurality of SEM images 1301, (ii) the simulated profiles 1303, and (iii) the KPIs, a model M130 to estimate depth data of a structure such that KPIs associated with the estimated depth data is within acceptable threshold of the KPIs associated with the simulated profiles 1303. In some embodiments, the estimated depth data includes at least one of: height data of the structure, shape parameters of the structure, or voxel map related to overhanging structures. In some embodiments, the shape parameters comprises one or more of a topCD measured at a top of the structure, a bottom CD measured at a bottom of the structure, or a side wall angle of the structure.

[0151] In some embodiments, generating the model M130 may be based on generative adversarial network, encoder-decoder network, or other machine learning related networks. For example, generating of the model M130 includes training a first model (e.g., G1 in Figure 14) and a second model (e.g., DS in Figure 14). In some embodiments, for example, referring to a GAN architecture, the first model may be a generator model, and the second model may be a discriminator model. The training of the first model is performed in conjunction with the second model such that the first model generates depth data using a SEM image of the plurality of SEM image as input. The second model classifies the generated depth data into a first category (e.g., real) or a second category (e.g., fake) using the simulated profiles 1303 and the simulated KPIs associated with the simulated profiles 1303 reference data.

[0152] In some embodiments, the generating of the model is an iterative process. Each iteration includes inputting a SEM image of the plurality of SEM images 1301 to the first model; estimating, using the first model, the depth data for inputted SEM image; extracting KPI from the estimated depth data; classifying, via the second model using (i) the extracted KPI and (ii) the plurality of simulated profiles 1303 and simulated KPI of the plurality of simulated profiles 1303 as reference, the estimated depth data into the first category or the second category. In some embodiments, the first category- indicates that the estimated depth data correspond to the reference and the second category indicates that the estimated depth data do- not correspond to the reference. Further, model parameters of both the first model and the second model may be updated to cause the first model to estimate depth data such that the second model classifies the estimated depth as the first category (e.g., real).

[0153] In some embodiments, for example, using a cycle-GAN training, the first model (e.g., G1 of Figure 14) may be further trained in conjunction with third model (e.g., another generator model G2 of Figure 14). The third model may be configured to generate a SEM image from the estimated depth data. In other words, the first model and the third model are cycle-consistent. For example, the first model and the third model are trained so that the input of the first model closely matches the output of the third model using a loss function (e.g., a difference between images 1402 and 1406 in Figure 14). It is appreciated that other forms of cycle-GAN training can be utilized, such as, for example, via domain adaptation, is described further in WO 2023 / 083559A1, which is incorporated herein by reference in its entirety.

[0154] The training of the first model and the third model may be an iterative process. Each iteration includes inputting a SEM image of the plurality of SEM images 1301 to the first model; estimating, using the first model, the depth data for inputted SEM image; generating, via the third model using the estimated depth data as input, a predicted SEM image; and updating model parameters of both the first model and the third model to cause a difference between the inputted SEM image and the predicted SEM image to be within a specified difference threshold. In someembodiments, the first model, the second model, and the third model are convolutional neural network (CNN), or deep CNN. In some embodiments, the model parameters are weights and biases of one or more layers of the CNN or DCNN.

[0155] Figure 14 illustrates an exemplary determination of a model for estimating depth data, according to the method 1300. In Figure 14, a model G1 may be trained according to method 1300 to generate the model M130. In this embodiment, a stochastic process model (e.g., the stochastic resist model SPM) may be employed that eliminates the need for specific targeted profile variations (e.g., programmed variations). According to an example, one-to-one correlation is not possible between the CHs generated by a stochastic resist model and the CHs formed the wafer. In some embodiments, to circumvent this problem, cycle-consistent generative adversarial networks (cycle-GANs) may be adopted for learning the unpaired image to image translation between SEM image 1402, to simulated depth data 1414 to predicted SEM image 1406 by minimizing a loss function. In some embodiments, the 3D profiles showed are just examples and may not correspond to actual calibrated profiles against the showed SEM images For example, the loss function may be a difference between the SEM image 1402 and the predicted SEM image 1406. An example of cycle-GAN is discussed in reference “J. Zhu, T. Park, P. Isola and A. A. Efros, Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks, 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017, pp.2242-2251.” which is incorporated herein in its entirety by reference.

[0156] Referring to Figure 14, metrology data or inspection data (e.g., SEM 1402) for CH arrays may be obtained from a metrology tool (e.g., SEM) by capturing an image or inspecting a patterned substrate. In some embodiments, multiple CH arrays are obtained from ADI for multiple cases, e.g., using a focus exposure matrix (FEM).

[0157] In some embodiments, a physical depth stochastic resist model (not illustrated) may be calibrated such that the stochastic model predicts well the CD distributions, LCDU, LER, LWR, defect rates, or other stochastic variations for the same conditions, as well a contrast curve for the same conditions during flat exposure. The stochastic model may be used to generate simulated profiles that have multiple examples of CD and resist height variations (e.g., due to the semi-open CHs), similar to the inspection data of the patterned substrate.

[0158] Further, a network (e.g., GAN) is trained using a dataset of unpaired depth resist profiles 1414 and SEM images 1402. For example, the GAN network includes a first model G1, a second model DS, and a third model G2. In some embodiments, the GAN network is trained to learn an effective way to transform SEM images 1402 to realistic depth resist profiles 1404 and then back to realistic SEM images 1406. For example, predictions from SEM image 1402 to depth resist profile 1404 by using the trained model G1.

[0159] In the present training example, the SEM image 1402 captured by a SEM tool may be input to a first model G1 (e.g., a generator of GAN) to generate depth data 1404. The depth data 1404 may be represented as a 3D profile of structures that may be formed on the substrate. For example, thedepth data 1404 may be resist profile formed on a resist layer of a substrate. In some embodiments, KPIs (e.g., CD, LCDU, LER, LWR) may be extracted from the depth data 1404. In some embodiments, the depth data 1404 and the KPIs may be input to a second model DS (e.g., a discriminator of a GAN). In some embodiments, reference profiles 1414 may be generated using the stochastic process model, and reference KPIs may be extracted from the reference profiles 1414.

[0160] The depth data 1404 and the related KPIs is input to the second model DS, which is configured to classify the depth data 1404 into a first category (e.g., real) or a second category (e.g., fake) based on the reference KPIs and reference simulated profiles 1414. In some embodiments, the first category is indicative that the depth data 1404 is realistic or similar to the reference profiles 1414, while the second category is indicative that the depth data 1404 is not realistic or not similar to the reference profiles 1414.

[0161] In some embodiments, the first model G1 and the second model DS are trained to improve each other. As such, the first model G1 progressively generates depth data that is so realistic that the second model DS classifies the model generated depth data as belonging to the first category (e.g., real).

[0162] Alternatively or additionally, the first model G1 and the second model DS is further trained in cooperation of a third model G2 that generates a predicted SEM image 1406 using the depth data 1404 as input. During the training, model parameters of the first model G1 and the third model G2 are adjusted to reduce or minimize a difference between the predicted SEM image 1406 and the real SEM image 1402. Thus, the third model G2 may generate realistic SEM images 1406 for any depth data. In some embodiments, the model parameters of the second model DS may also be adjusted, since modifying G1 causes changes in G1 predictions, which in turn may affect classification by the second model DS. In other words, the third model when trained together may generate realistic depth data that is further verified by generating the predicted SEM image that is also realistic (similar to real SEM). As such, no pairing between training data may be needed, according to some embodiments.

[0163] Referring back to Figure 13, in some embodiments, the method 1300 may further include a process P1310 for applying the trained model 1310. For example, applying the model include receiving, via a SEM tool, a SEM image of a structure patterned on a substrate; and determining, via the model using the received SEM image as input, depth data associated with the SEM image. In some embodiments, based on the depth data, physical characteristics of the structure of the patterned substrate may be determined. The physical characteristics comprising a shape, a size, or relative positioning of polygon shapes with respect to each other at one or more depth of features of the structure.

[0164] In some embodiments, comparing the physical characteristics with desired ranges, a defect in the structure may be detected. In some embodiments, KPIs associated with the physical characteristics may be compared with the desired KPI ranges to determine a defect. As such, thedefect may be indicative of the structure not satisfying a design specification or performance specification. In some embodiments, based on the defect, a parameter of one or more patterning process may be adjusted to eliminate the defect in a subsequent process of the patterning process.

[0165] Figure 15 is a flowchart of a method 1500 applying the model 1110 or the model 1310 trained according to the method 1100 or 1300 discussed herein. In an example, the method 1500 for estimating depth data for structures formed on substrates uses a convolutional neural network (“CNN”) configured to estimate depth data based on a scanning electron microscope (“SEM”) image of the structures, according to some embodiments. For example, the CNN may be trained according to 1100 or 1300. An example implementation of the method 1500 may include processes P1502, and P1504, further discussed below.

[0166] Process P1502 includes receiving a SEM image 1501 of a structure on a patterned substrate. For example, a normal SEM image may be captured by a SEM tool by projecting an e-beam perpendicular to the substrate. Alternatively, an optical image of the substrate may be captured via an optical tool and received in the process P1502.

[0167] Process P1504 includes estimating, via the CNN using the SEM image 1501 as input, depth data 1510 associated the structure of the patterned substrate. In some embodiments, the CNN may be trained using training data comprising a plurality of SEM images of structures and a plurality of simulated profiles of the structures. In some embodiments, the simulated profiles may be generated by a process model associated with a patterning process.

[0168] In some embodiments, the CNN may be a combination of a generator model (e.g., a first model or CNN1) and a discriminator model (e.g., a second model or CNN2) that are trained together (e.g., as discussed with respect to Figure 11). For example, the training of the CNN is an iterative process, each iteration includes obtaining the plurality of SEM images and the plurality of simulated profiles, each SEM image paired with a simulated profile; inputting a SEM image of the plurality of SEM images to the generator model; estimating, using the generator model, the depth data for inputted SEM images; classifying, via a discriminator model using a simulated profile paired with the inputted SEM image as reference, the estimated depth data as a first category indicative that the estimated depth data correspond to the reference or a second category indicative that the estimated depth data does not correspond to the reference; and updating model parameters of both the generator model and the discriminator model to cause the generator model to estimate depth data such that the discriminator model classifies the estimated depth as first category.

[0169] In some embodiments, the CNN may be a combination of a generator model (e.g., a first model or CNN1) and a discriminator model (e.g., a second model or CNN2) that are trained together (e.g., as discussed with respect to Figure 13). For example, the training of the CNN is an iterative process; each iteration includes inputting a SEM image of the plurality of SEM images to the generator model; estimating, using the generator model, the depth data for inputted SEM images; extracting, based on intensity values associated with the structure in the estimated depth data, keyperformance indicator (KPI) associated with the structure; classifying, via the discriminator model using (i) the extracted KPI and (ii) the plurality of simulated profiles and simulated KPI of the plurality of simulated profiles as reference, the estimated depth data as first category indicative that the estimated depth data correspond to the reference or a second category indicative that the estimated depth data does not correspond to the reference; and updating model parameters of both the generator model and the discriminator model to cause the generator model to estimate depth data such that the discriminator model classifies the estimated depth as the first category.

[0170] In some embodiments, the CNN may be further trained in conjunction with a second generator model (e.g., CNN3), the second generator model being configured to generate a SEM image from the estimated depth data. The training further including inputting a SEM image of the plurality of SEM images to the CNN; estimating, using the CNN, the depth data for inputted SEM images; generating, via the second generator model using the estimated depth data as input, a predicted SEM image; and updating model parameters of both the CNN and the second generator model to cause a difference between the inputted SEM image and the predicted SEM image to be within a specified difference threshold.

[0171] In some embodiments, the process model is a calibrated deterministic process model DPM that is calibrated such that critical dimension (CD) of simulated profiles of the structure is within a CD threshold of observed data associated with the patterned substrate, but not necessarily meet stochastic variation specifications associated with the observed data. In some embodiments, the process model is a calibrated stochastic process model that is calibrated such that key performance indicator (KPI) extracted from the model generated depth data is within a specified threshold of observed KPI extracted from inspection data associated with the patterned substrate. In some embodiments, the inspection data is obtained from a plurality of features of the patterned substrate, the plurality of features formed using a range of dose and focus conditions. In some embodiments, the KPI includes one or more of: CD of the structure, local CD uniformity (LCDU) associated with the structure, line edge roughness (LER) associated with the structure, defect rates associated with the structure, other KPIs indicative of performance of the patterning process.

[0172] As discussed earlier, the depth information can be further used to determine a defectiveness of structure by comparing CDs of the features at different depths and determining whether the CDs satisfy design specification. In some embodiments, the depth information may be used to determine one or more parameters of a patterning process such as resist process, etching process, or other patterning related process so that the 3D structure of the feature is within desired design specification.

[0173] In some embodiments, a base model (e.g., an untrained model) and the trained model may be a machine learning model comprises weights and biases as model parameters. During the training process, the weights and biases of the base model are continuously adjusted based on the training data. At the end of the training, the base model is referred as the trained model. In some embodiments, the trained model is convolutional neural network (e.g., CNN), a deep convolutional network(DCNN). The model parameters comprises weights and biases of one or more layers of the deep convolutional network.

[0174] Without limiting the scope of the present disclosure, applications of example supervised machine learning algorithms are described below.

[0175] Supervised learning is the machine learning task of inferring a function from labeled training data. The training data includes a set of training examples. In supervised learning, each example is a pair having an input object (typically a vector) and a desired output value (also called the supervisory signal). A supervised learning algorithm analyzes the training data and produces an inferred function, which can be used for mapping new examples. An optimal scenario will allow the algorithm to correctly determine the class labels for unseen instances. This requires the learning algorithm to generalize from the training data to unseen situations in a “reasonable” way.

[0176] Given a set of N training examples of the form such that is the feature vector of the i-th example and is its label (i.e., class), a learning algorithm seeks a function , where is the input space and is the output space. A feature vector is an n-dimensional vector of numerical features that represent some object. Many algorithms in machine learning require a numerical representation of objects, since such representations facilitate processing and statistical analysis. When representing images, the feature values might correspond to the pixels of an image, when representing texts perhaps term occurrence frequencies. The vector space associated with these vectors is often called the feature space. The function is an element of some space of possible functions , usually called the hypothesis space. It is sometimes convenient to represent using a scoring function such that is defined as returning the value that gives the highest score: . Let denote the space of scoring functions.

[0177] Although and can be any space of functions, many learning algorithms are probabilistic models where takes the form of a conditional probability model , or takes the form of a joint probability model . For example, naive Bayes and linear discriminant analysis are joint probability models, whereas logistic regression is a conditional probability model.

[0178] There are two basic approaches to choosing or : empirical risk minimization and structural risk minimization. Empirical risk minimization seeks the function that best fits the training data. Structural risk minimization includes a penalty function that controls the bias / variance tradeoff.

[0179] In both cases, it is assumed that the training set has a sample of independent and identically distributed pairs . In order to measure how well a function fits the training data, a loss function is defined. For training example , the loss of predicting the value is .

[0180] The risk of function is defined as the expected loss of . This can be estimated from the training data as .

[0181] Exemplary models of supervised learning include decision trees, ensembles (bagging, boosting, random forest), k-NN, linear regression, naive Bayes, neural networks, logistic regression, perceptron, support vector machine (SVM), relevance vector machine (RVM), and deep learning.

[0182] SVM is an example of supervised learning model, which analyzes data and recognizes patterns and can be used for classification and regression analysis. Given a set of training examples, each marked as belonging to one of two categories, a SVM training algorithm builds a model that assigns new examples into one category or the other, making it a non-probabilistic binary linear classifier. A SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on which side of the gap they fall on.

[0183] In addition to performing linear classification, SVMs can efficiently perform a non-linear classification using what is called the kernel methods, implicitly mapping their inputs into high- dimensional feature spaces.

[0184] Kernel methods involve a user-specified kernel, i.e., a similarity function over pairs of data points in raw representation. Kernel methods owe their name to the use of kernel functions, which enable them to operate in a high-dimensional, implicit feature space without ever computing the coordinates of the data in that space, but rather by simply computing the inner products between the images of all pairs of data in the feature space. This operation is often computationally cheaper than the explicit computation of the coordinates. This approach is called the “kernel trick.”

[0185] The effectiveness of SVM depends on the selection of kernel, the kernel’s parameters, and soft margin parameter C. A common choice is a Gaussian kernel, which has a single parameter γ. The best combination of C and γ is often selected by a grid search (also known as “parameter sweep”) with exponentially growing sequences of C and γ, for example, ; .

[0186] A grid search is an exhaustive searching through a manually specified subset of the hyperparameter space of a learning algorithm. A grid search algorithm is guided by some performance metric, typically measured by cross-validation on the training set or evaluation on a held-out validation set.

[0187] Each combination of parameter choices may be checked using cross validation, and the parameters with best cross-validation accuracy are picked.

[0188] Cross-validation, sometimes called rotation estimation, is a model validation technique for assessing how the results of a statistical analysis will generalize to an independent data set. It is mainly used in settings where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice. In a prediction problem, a model is usually given a dataset of known data on which training is run (training dataset), and a dataset of unknown data (or first seendata) against which the model is tested (testing dataset). The goal of cross validation is to define a dataset to “test” the model in the training phase (i.e., the validation dataset), in order to limit problems like overfitting, give an insight on how the model will generalize to an independent data set (i.e., an unknown dataset, for instance from a real problem), etc. One round of cross-validation involves partitioning a sample of data into complementary subsets, performing the analysis on one subset (called the training set), and validating the analysis on the other subset (called the validation set or testing set). To reduce variability, multiple rounds of cross-validation are performed using different partitions, and the validation results are averaged over the rounds.

[0189] The final model, which can be used for testing and for classifying new data, is then trained on the entire training set using the selected parameters.

[0190] Another example of supervised learning is regression. Regression infers the relationships between a dependent variable and one or more independent variables, from a set of values of the dependent variables and corresponding values of the independent variables. Regression may estimate the conditional expectation of the dependent variable given the independent variables. The inferred relationships may be called the regression function. The inferred relationships may be probabilistic.

[0191] Reference is now made to Figure 16, which is an example illustration of training a convolutional neural network (CNN) to generating a depth map of a sample, consistent with embodiments of the present disclosure. Figure 16 illustrates a plurality of input images 1601 supplied to a CNN 1602 to output a plurality of three-dimensional depth maps 1603 from the plurality of the input images 1601. A depth map may be generated by CNN 1602 per input image of the plurality of input images 1601. CNN 1602 may be as described above and may generate depth map 1603 via a loss function or other mathematical operations as described above. Input image 1601 may be a simulated image of a sample to be inspected and may represent any shape or geometry (e.g., line space, contact holes, etc.). Input image 1601 may be generated by a simulator (e.g., e-beam simulator), and a plurality of images may be generated by varying input parameters to the simulator. Input images 1601 may be generated by varying parameters such as surface roughness and defect size. The type of defect may be any defect apparent in an IC. Non-limiting examples of input defects include top loss, scumming, or any other defect that may occur during IC fabrication. Defect size may be varied in size (e.g., dy, nm in Y-direction) and height (e.g., zcut, nm in Z-direction). Roughness may be varied from 0 to 1.5, where a roughness of 0 indicates an ideally smooth feature surface. Realistic features surfaces may have non-zero roughness values.

[0192] After CNN 1602 is trained, input images 1604 may be supplied to trained CNN 1605 to generate depth images 1606. Input images 1604 may be a second plurality of simulation images, which may have different parameters (e.g., roughness, defect type, defect size, etc.) compared to input images 1601. Input images 1604 may be non-simulation images of a sample. Input images 1604 may be experimentally collected images of a sample collected from an inspection apparatus. In some embodiments, the inspection apparatus may be a charged particle beam apparatus, an atomic forcemicroscope, a scatterometer, an optical imaging apparatus, or any other tool used for inspection of an IC. In some embodiments, the inspection apparatus may be an atomic force microscope, a scatterometer, a focused ion beam scanning electron microscope, a scanning electron beam microscope, a transmission electron beam microscope, a scanning transmission electron beam microscope, or a soft-xray microscope.

[0193] Reference is now made to Figure 17, which is an example illustration of depth maps of a sample generate from input images, consistent with embodiments of the present disclosure. In some embodiments, reference images 1701 and 1703 and input image 1702 may be supplied to a trained CNN (e.g., trained CNN 1605 in Figure 16). Reference images 1701 and 1703 and input image 1702 may be a simulated image or a non-simulated image. In some embodiments, reference images 1701 and 1703 may be a golden image, a GDS file, a design template of a feature to be fabricated, or a defect-free fabricated feature. Input image 1702 may be an image of interest of a sample and may contain defects. As described above, reference images 1701 and 1703 and input image 1702 are not limited to the feature shapes and geometries as illustrated in Figure 17. Any feature shape or geometry may be contemplated in embodiments of the present disclosure. Generated depth map 1705 corresponds to input image 1702, generated depth map 1704 corresponds to reference image 1701, and generated depth map 1706 corresponds to reference image 1703. Depth map 1705 may illustrate a three-dimensional representation of defects present in input image 1702 that may not be as easily identifiable in input image 1702.

[0194] The generated depth maps 1704-1706 may be used to calculate a defect strength of a defect present in depth map 1705 (or input image 1702). In some embodiments, the defect strength may be calculated by the following formula: !^!"# $#%!&'#ℎ =()*+,-)*+.)∗()*+,-)*+0)(1∗23))∗(1∗ 234)≥ 1 (Equation 2)

[0195] In a may be tuned to an optimal value. σ21 and σ23 are the standard deviation values of 1705-1704 and 1705-1706. Equation 2 may be understood to determine a difference between depth map 1705 to the depth maps generated from a reference image (e.g., depth maps 1704 and 1706 and reference images 1701 and 1703), and identify a defect with a value greater than or equal to 1. Equation 2 is applied per a location on depth maps 1704-1706 and may be repeated for each location on depth maps 1704-1706. In some embodiments, a defect strength may be calculated for each pixel in depth maps 1704-1706.

[0196] Reference is now made to Figure 18A, which is an example illustration of an input image to train a CNN to output a depth map, consistent with embodiments of the present disclosure. Figure 18A illustrates image 1801 and 1802 which may contain defects. Input images 1801 and 1802 may be simulated to contain nine defects with a nominal size (e.g., 8 nm or 16 nm zcut). The possible defectsobserved in the images are identified by solid black outlines 1803. In some embodiments, inspection image 1801 corresponds to a toploss defect with a zcut value of 8 nm. In some embodiments, inspection image 1802 corresponds to a toploss defect with a zcut value of 16 nm. Input images 1801 and 1802 may be used to identify the possible defects outlined in 1803, but since images 1801 and 1802 are two-dimensional illustrations, it may be difficult to accurately determine if a defect is present at a location in the images.

[0197] Reference is now made to Figure 18B - Figure 18G, which are example illustrations of binary maps generated by aggregating a calculated defect strength per location in a generated depth map, consistent with embodiments of the present disclosure. A binary map may be processed after the calculation. In some embodiments, the binary map may be processed with a morphological operator such than a defect is processed as a structural object square of size D. In some embodiments, a binary map may be processed with a Gaussian blur with a filter size sigma. Each of Figure 18B - Figure 18G displays a generated binary map generated by calculating defect strengths via Equation 2 with varying parameters (e.g., roughness, T, structural object size D, and Gaussian kernel size sigma). Figure 18B - Figure 18G displays results for input images (e.g., simulated images) with nine defects with a nominal defect height (e.g., zcut) ranging from 4 nm to 32 nm. The binary maps displayed in Figure 18B - Figure 18G illustrate a defect is present by the solid white bars. In some embodiments, a defect is more clearly illustrated at defect heights above 8 nm.

[0198] Reference is now made to Figure 19A, which is an example illustration of quantifying a defect dimension using a machine learning model, consistent with embodiments of the present disclosure. Figure 19A illustrates a two-dimensional input image 1901 that may have defects present that may not be as easily identified. A depth map 1902 may be generated from input image 1901 to predict possible defects using a trained CNN as described above. In some embodiments, input image 1901 may be a simulated image generated via parameters to model possible defects present on the sample. In some embodiments, input image 1901 may be an experimentally collected image of a fabricated feature or sample. Input image 1901 may correspond to a defect of a nominal size (e.g., toploss zcut of 16 nm). Depth map 1902 may illustrate a three-dimensional representation of possible defects present on the sample. Depth map 1902 illustrates possible top loss defects, but embodiments of the present disclosure are not so limited. For example, depth map 1902 may illustrate scumming defects or defects associated with other structure shapes or geometries of an IC. Figure 19A also illustrates a depth map 1903 that may be generated from a reference image of the sample. In some embodiments, the reference image may indicate the defects present on the sample. In some embodiments, the reference image may be an image of the inspected sample. In some embodiments, the reference image may be historical data of the sample. In some embodiments, the reference image may be coupled with some type of metrology data / process that provides profile information that can be related to profile information in depth map 1902. In some embodiments, depth map 1903 may be considered the ground truth. In some embodiments, depth maps 1902 and 1903 may be analyzed toquantify a possible defect. In some embodiments, possible defects may be represented by a shape, correlation plot, or distribution. In some embodiments, a possible defect may be represented by any part of a shape that creates an excursion or deviation from an expected shape (e.g., feature) in terms of the height, depth, or length of the expected shape.

[0199] Reference is now made to Figure.19B and Figure 19C, which is an example illustration of metrics to quantify a machine learning model to determine a defect dimension, consistent with embodiments of the present disclosure. Figure 19B illustrates a modeled dimension of a defect height (e.g., zcut, height in z-direction) at a location on a depth map (e.g., depth map 1902 in Figure 19A) compared to a ground truth (e.g., depth map 1903 in Figure 19A). The line profile illustrated in Figure 19B may be obtained by averaging signal over 6 pixels in a depth map (e.g., depth map 1902 and depth map 1903 in Figure 19A) around the center of a defect. Figure 19B illustrates a cross-sectional view of depth map 1902 where a defect is present in a predicted depth map and a ground truth depth map. The cross-sectional view of the predicted depth map (“predicted”) is observed to match with the ground truth depth map (“real”). This indicates the generated depth map (e.g., depth map 1902) may accurately predict the existence and a dimension of a defect on a sample to be inspected. Figure 19C illustrates a modeled length of a defect (e.g., dy), consistent with embodiments of the present disclosure. The line profile illustrated in Figure 19C may be obtained by averaging signal over 6 pixels in a depth map along the length of a defect. The defect length may be modeled as a Gaussian distribution from the depth map (e.g., depth map 1902) and compared to the modeled defect length from the ground truth (e.g., depth map 1903). In some embodiments, the defect length may be determined by a full width half maximum value. As illustrates in Figure 19C, the predicted defect length matches with the ground truth defect length. It is appreciated that embodiments of the present disclosure are not limited to a Gaussian distribution to model defect length. Non-limited examples include any metric that may model or define a defect length, and any metric may be used to model or define a defect height.

[0200] Depth maps such as depth map 1902 in Figure 19A may be generated for a plurality of input images that predict defects at various dimensions (e.g., height and length) as described above for Figure 18A - Figure 18G and from input images generated from varying parameters (e.g., roughness, Gaussian kernel size, T). Reference is now made to Figure 20, which illustrates example correlation plots of predicted defect dimension to ground truth defect dimension with different feature roughness values (e.g., sigma), consistent with some embodiments of the present disclosure. The correlation plots may be generated for predicted defects with a dimension (height or length) ranging from 4 nm to 36 nm. Figure 20 illustrates a linear correlation between the defect dimension determined from the ground truth and the predicted defect dimension. There is slight deviation from linearity for defect height below 8 nm, and predicted defect length appears to be less than ideal for a toploss of 4 and 8 nm.

[0201] The results illustrated in Figure 20 illustrate a strong correlation between predicted defect inspection and metrology and actual defects present on a sample.

[0202] It is appreciated that while the discussion above is directed to toploss defects, embodiments of the present disclosure are not so limited. For example, a CNN may be trained based on simulated inspection images of any defect present in an IC structure (e.g., scumming, footing, etc.). Depth maps may be generated for scumming or footing defects in an IC structure. In some embodiments, binary maps may be generated to identify a location for a scumming or footing defect, and defect dimensions may be calculated and correlated with ground truth summing or footing defects.

[0203] Reference is now made to Figure 21, which is a flowchart of an exemplary machine learning method 2100 for defect detection, consistent with embodiments of the present disclosure. In step 2101, a convolutional neural network is trained with an image of a sample to generate a depth map from the image. In some embodiments, the image may be experimentally obtained or simulated. In some embodiments, a plurality of images may be supplied to the CNN for training. The plurality of images may be experimentally obtained or may be generated by a simulator (e.g., e-beam simulator, optical simulator) with varied feature roughness, dimension, structure, or shape.

[0204] In step 2102, the trained CNN generated a depth map from a reference image and an image of interest of a sample. In some embodiments, the reference image and the image of interest may be experimentally obtained images of a sample. In some embodiments, the reference image and the image of interest may be inspection images of a sample. In some embodiments, a plurality of images of interest and a plurality of reference images may be supplied to the trained CNN.

[0205] In step 2103, a defect strength is calculated for a location on the generated depth map from a plurality of locations on the generated depth map. In some embodiments, the defect strength may be calculated based on a mathematical operation that relates strength of defect signal to an estimator of a signal over noise metric in the generated depth map. In some embodiments, the defect strength may be calculated based on equation 2. In some embodiments, the defect strength may be calculated based on a signal over noise metric. In some embodiments, the defect strength may be calculated based on a threshold value.

[0206] In step 2104, a binary map is generated by aggregating the plurality of locations with a calculated defect strength. A binary map may be processed by a morphological operator such that a defect is processed as a structural object square of size D. In some embodiments, a binary map may be processed with a Gaussian blur with a filter size sigma. In step 2105, the generated binary map is used to guide inspection of an inspection tool. In some embodiments, the generated binary map may be used to guide metrology measurements of an inspection tool. In some embodiments, an inspection tool may select a location on the generated binary map to inspect for a possible defect and collect a metrology measurement to quantify the defect (e.g., as described in Figure 19B and 19C). In some embodiments, the inspection tool is an optical inspection tool. In some embodiments, the inspectiontool is a charged particle beam apparatus. In some embodiments, the charged particle beam apparatus is a scanning electron microscope.

[0207] In some embodiments, there is provided a system that can use the model to generate 3D data (e.g., depth data) after the system captures images of a patterned substrate. In some embodiments, the system can be, for example, a SEM tool of Figure 22 or an inspection tool of Figure 23 that are configured to include the model DBM discussed herein. For example, the metrology tool includes an e-beam generator to capture an image of a patterned substrate; and one or more processors including a trained model described herein. The one or more processors are configured to execute the trained model using the captured image to generate depth data. As mentioned earlier, the model may be a convolutional neural network.

[0208] In some embodiments, the model generated 3D data (e.g., depth data) can be used to improve patterning process. For example, the depth data can be used in simulation of the patterning process, for example, to predict contours, CDs, edge placement (e.g., edge placement error), etc. in the resist and / or etched image. The objective of the simulation is to accurately predict, for example, edge placement, and / or aerial image intensity slope, and / or CD, etc. of the printed pattern. These values can be compared against an intended design to, e.g., correct the patterning process, identify where a defect is predicted to occur, etc. The intended design is generally defined as a pre-OPC design layout which can be provided in a standardized digital file format such as GDSII or OASIS or other file format.

[0209] In some embodiments, the inspection apparatus or the metrology apparatus may be a scanning electron microscope (SEM) that yields an image of a structure (e.g., some or all the structure of a device) exposed or transferred on the substrate. Figure 22 depicts some embodiments of a SEM tool. A primary electron beam EBP emitted from an electron source ESO is converged by condenser lens CL and then passes through a beam deflector EBD1, an E x B deflector EBD2, and an objective lens OL to irradiate a substrate PSub on a substrate table ST at a focus.

[0210] When the substrate PSub is irradiated with electron beam EBP, secondary electrons are generated from the substrate PSub. The secondary electrons are deflected by the E x B deflector EBD2 and detected by a secondary electron detector SED. A two-dimensional electron beam image can be obtained by detecting the electrons generated from the sample in synchronization with, e.g., two dimensional scanning of the electron beam by beam deflector EBD1 or with repetitive scanning of electron beam EBP by beam deflector EBD1 in an X or Y direction, together with continuous movement of the substrate PSub by the substrate table ST in the other of the X or Y direction.

[0211] A signal detected by secondary electron detector SED is converted to a digital signal by an analog / digital (A / D) converter ADC, and the digital signal is sent to an image processing system IPU. In some embodiments, the image processing system IPU may have memory MEM to store all or part of digital images for processing by a processing unit PU. The processing unit PU (e.g., specially designed hardware or a combination of hardware and software) is configured to convert or process thedigital images into datasets representative of the digital images. Further, image processing system IPU may have a storage medium STOR configured to store the digital images and corresponding datasets in a reference database. A display device DIS may be connected with the image processing system IPU, so that an operator can conduct necessary operation of the equipment with the help of a graphical user interface.

[0212] As noted above, SEM images may be processed to extract contours that describe the edges of objects, representing device structures, in the image. These contours are then quantified via metrics, such as CD. Thus, typically, the images of device structures are compared and quantified via simplistic metrics, such as an edge-to-edge distance (CD) or simple pixel differences between images. Typical contour models that detect the edges of the objects in an image in order to measure CD use image gradients. Indeed, those models rely on strong image gradients. But, in practice, the image typically is noisy and has discontinuous boundaries. Techniques, such as smoothing, adaptive thresholding, edge-detection, erosion, and dilation, may be used to process the results of the image gradient contour models to address noisy and discontinuous images, but will ultimately result in a low-resolution quantification of a high-resolution image. Thus, in most instances, mathematical manipulation of images of device structures to reduce noise and automate edge detection results in loss of resolution of the image, thereby resulting in loss of information. Consequently, the result is a low-resolution quantification that amounts to a simplistic representation of a complicated, high- resolution structure.

[0213] So, it is desirable to have a mathematical representation of the structures (e.g., circuit features, alignment mark or metrology target portions (e.g., grating features), etc.) produced or expected to be produced using a patterning process, whether, e.g., the structures are in a latent resist image, in a developed resist image or transferred to a layer on the substrate, e.g., by etching, that can preserve the resolution and yet describe the general shape of the structures. In the context of lithography or other pattering processes, the structure may be a device or a portion thereof that is being manufactured and the images may be SEM images of the structure. In some instances, the structure may be a feature of semiconductor device, e.g., integrated circuit. In this case, the structure may be referred as a pattern or a desired pattern that comprises a plurality of feature of the semiconductor device. In some instances, the structure may be an alignment mark, or a portion thereof (e.g., a grating of the alignment mark), that is used in an alignment measurement process to determine alignment of an object (e.g., a substrate) with another object (e.g., a patterning device) or a metrology target, or a portion thereof (e.g., a grating of the metrology target), that is used to measure a parameter (e.g., overlay, focus, dose, etc.) of the patterning process. In some embodiments, the metrology target is a diffractive grating used to measure, e.g., overlay.

[0214] Figure 23 schematically illustrates a further embodiment of an inspection apparatus. The system is used to inspect a sample 90 (such as a substrate) on a sample stage 88 and comprises a charged particle beam generator 81, a condenser lens module 82, a probe forming objective lensmodule 83, a charged particle beam deflection module 84, a secondary charged particle detector module 85, and an image forming module 86.

[0215] The charged particle beam generator 81 generates a primary charged particle beam 91. The condenser lens module 82 condenses the generated primary charged particle beam 91. The probe forming objective lens module 83 focuses the condensed primary charged particle beam into a charged particle beam probe 92. The charged particle beam deflection module 84 scans the formed charged particle beam probe 92 across the surface of an area of interest on the sample 90 secured on the sample stage 88. In some embodiments, the charged particle beam generator 81, the condenser lens module 82 and the probe forming objective lens module 83, or their equivalent designs, alternatives or any combination thereof, together form a charged particle beam probe generator which generates the scanning charged particle beam probe 92.

[0216] The secondary charged particle detector module 85 detects secondary charged particles 93 emitted from the sample surface (maybe also along with other reflected or scattered charged particles from the sample surface) upon being bombarded by the charged particle beam probe 92 to generate a secondary charged particle detection signal 94. The image forming module 86 (e.g., a computing device) is coupled with the secondary charged particle detector module 85 to receive the secondary charged particle detection signal 94 from the secondary charged particle detector module 85 and accordingly forming at least one scanned image. In some embodiments, the secondary charged particle detector module 85 and image forming module 86, or their equivalent designs, alternatives or any combination thereof, together form an image forming apparatus which forms a scanned image from detected secondary charged particles emitted from sample 90 being bombarded by the charged particle beam probe 92.

[0217] In some embodiments, a monitoring module 87 is coupled to the image forming module 86 of the image forming apparatus to monitor, control, etc. the patterning process and / or derive a parameter for patterning process design, control, monitoring, etc. using the scanned image of the sample 90 received from image forming module 86. So, in some embodiments, the monitoring module 87 is configured or programmed to cause execution of a method described herein. In some embodiments, the monitoring module 87 comprises a computing device. In some embodiments, the monitoring module 87 comprises a computer program to provide functionality herein and encoded on a computer readable medium forming, or disposed within, the monitoring module 87.

[0218] In some embodiments, like the electron beam inspection tool of Figure 22 that uses a probe to inspect a substrate, the electron current in the system of Figure 23 is significantly larger compared to, e.g., a CD SEM such as depicted in Figure 22, such that the probe spot is large enough so that the inspection speed can be fast. However, the resolution may not be as high as compared to a CD SEM because of the large probe spot. In some embodiments, the above discussed inspection apparatus may be single beam or a multi-beam apparatus without limiting the scope of the present disclosure.

[0219] The SEM images, from, e.g., the system of Figure 22 and / or Figure 23, may be processed to extract contours that describe the edges of objects, representing device structures, in the image. These contours are then typically quantified via metrics, such as CD, at user-defined cut-lines. Thus, typically, the images of device structures are compared and quantified via metrics, such as an edge-to- edge distance (CD) measured on extracted contours or simple pixel differences between images.

[0220] In some embodiments, the one or more processes of the methods can be implemented as instructions (e.g., program code) in a processor of a computer system (e.g., process 104 of computer system 100). In some embodiments, the procedures may be distributed across a plurality of processors (e.g., parallel computation) to improve computing efficiency. In some embodiments, the computer program product comprising a non-transitory computer readable medium has instructions recorded thereon, the instructions when executed by a computer hardware system implementing the method described herein.

[0221] As discussed herein, there are provided different approaches (e.g., methods 500, 700, 1100, and 1300) for training models configured to generate depth information from a single SEM image of a patterned substrate. As such, using the models described herein, depth information can be estimated using only one measurement thereby saving metrology time. The depth information can be further employed to configure a lithographic process such that yield is improved or defects minimized.

[0222] According to present disclosure, the combination and sub-combinations of disclosed elements constitute separate embodiments. For example, a first combination includes determining a model configured to estimate depth data using a single image (e.g., SEM image, AFM data, optical images, etc.). The sub-combination may include determining a depth data using the trained model. In another combination, the depth data can be employed in an inspection process, determining OPC, or SMO based on model-generated variance data. In another example, the combination includes determining, based on inspection data based on the depth data, process adjustments to a lithography process, resist process, or etch process to improve the yield of the patterning process.

[0223] Figure 24 is a block diagram that illustrates a computer system 100 which can assist in implementing the methods, flows or the apparatus disclosed herein. Computer system 100 includes a bus 102 or other communication mechanism for communicating information, and a processor 104 (or multiple processors 104 and 105) coupled with bus 102 for processing information. Computer system 100 also includes a main memory 106, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 102 for storing information and instructions to be executed by processor 104. Main memory 106 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 104. Computer system 100 further includes a read only memory (ROM) 108 or other static storage device coupled to bus 102 for storing static information and instructions for processor 104. A storage device 110, such as a magnetic disk or optical disk, is provided and coupled to bus 102 for storing information and instructions.

[0224] Computer system 100 may be coupled via bus 102 to a display 112, such as a cathode ray tube (CRT) or flat panel or touch panel display for displaying information to a computer user. An input device 114, including alphanumeric and other keys, is coupled to bus 102 for communicating information and command selections to processor 104. Another type of user input device is cursor control 116, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 104 and for controlling cursor movement on display 112. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane. A touch panel (screen) display may also be used as an input device.

[0225] According to one embodiment, portions of one or more methods described herein may be performed by computer system 100 in response to processor 104 executing one or more sequences of one or more instructions contained in main memory 106. Such instructions may be read into main memory 106 from another computer-readable medium, such as storage device 110. Execution of the sequences of instructions contained in main memory 106 causes processor 104 to perform the process steps described herein. One or more processors in a multi-processing arrangement may also be employed to execute the sequences of instructions contained in main memory 106. In an alternative embodiment, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, the description herein is not limited to any specific combination of hardware circuitry and software.

[0226] The term “computer-readable medium” as used herein refers to any medium that participates in providing instructions to processor 104 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non- volatile media include, for example, optical or magnetic disks, such as storage device 110. Volatile media include dynamic memory, such as main memory 106. Transmission media include coaxial cables, copper wire and fiber optics, including the wires that comprise bus 102. Transmission media can also take the form of acoustic or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD- ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.

[0227] Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 104 for execution. For example, the instructions may initially be borne on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 100 can receive the data on the telephone line and use an infraredtransmitter to convert the data to an infrared signal. An infrared detector coupled to bus 102 can receive the data carried in the infrared signal and place the data on bus 102. Bus 102 carries the data to main memory 106, from which processor 104 retrieves and executes the instructions. The instructions received by main memory 106 may optionally be stored on storage device 110 either before or after execution by processor 104.

[0228] Computer system 100 may also include a communication interface 118 coupled to bus 102. Communication interface 118 provides a two-way data communication coupling to a network link 120 that is connected to a local network 122. For example, communication interface 118 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 118 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 118 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

[0229] Network link 120 typically provides data communication through one or more networks to other data devices. For example, network link 120 may provide a connection through local network 122 to a host computer 124 or to data equipment operated by an Internet Service Provider (ISP) 126. ISP 126 in turn provides data communication services through the worldwide packet data communication network, now commonly referred to as the “Internet” 128. Local network 122 and Internet 128 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 120 and through communication interface 118, which carry the digital data to and from computer system 100, are exemplary forms of carrier waves transporting the information.

[0230] Computer system 100 can send messages and receive data, including program code, through the network(s), network link 120, and communication interface 118. In the Internet example, a server 130 might transmit a requested code for an application program through Internet 128, ISP 126, local network 122 and communication interface 118. One such downloaded application may provide all or part of a method described herein, for example. The received code may be executed by processor 104 as it is received, and / or stored in storage device 110, or other non-volatile storage for later execution. In this manner, computer system 100 may obtain application code in the form of a carrier wave.

[0231] Figure 25 schematically depicts an exemplary lithographic projection apparatus in conjunction with the techniques described herein can be utilized. The apparatus comprises: - an illumination system IL, to condition a beam B of radiation. In this particular case, the illumination system also comprises a radiation source SO; - a first object table (e.g., patterning device table) MT provided with a patterning device holder to hold a patterning device MA (e.g., a reticle), and connected to a first positioner to accurately position the patterning device with respect to item PS;- a second object table (substrate table) WT provided with a substrate holder to hold a substrate W (e.g., a resist-coated silicon wafer), and connected to a second positioner to accurately position the substrate with respect to item PS; - a projection system (“lens”) PS (e.g., a refractive, catoptric or catadioptric optical system) to image an irradiated portion of the patterning device MA onto a target portion C (e.g., comprising one or more dies) of the substrate W.

[0232] As depicted herein, the apparatus is of a transmissive type (i.e., has a transmissive patterning device). However, in general, it may also be of a reflective type, for example (with a reflective patterning device). The apparatus may employ a different kind of patterning device to classic mask; examples include a programmable mirror array or LCD matrix.

[0233] The source SO (e.g., a mercury lamp or excimer laser, LPP (laser produced plasma) EUV source) produces a beam of radiation. This beam is fed into an illumination system (illuminator) IL, either directly or after having traversed conditioning means, such as a beam expander Ex, for example. The illuminator IL may comprise adjusting means AD for setting the outer and / or inner radial extent (commonly referred to as σ-outer and σ-inner, respectively) of the intensity distribution in the beam. In addition, it will generally comprise various other components, such as an integrator IN and a condenser CO. In this way, the beam B impinging on the patterning device MA has a desired uniformity and intensity distribution in its cross-section.

[0234] It should be noted with regard to Figure 25 that the source SO may be within the housing of the lithographic projection apparatus (as is often the case when the source SO is a mercury lamp, for example), but that it may also be remote from the lithographic projection apparatus, the radiation beam that it produces being led into the apparatus (e.g., with the aid of suitable directing mirrors); this latter scenario is often the case when the source SO is an excimer laser (e.g., based on KrF, ArF or F2 lasing).

[0235] The beam PB subsequently intercepts the patterning device MA, which is held on a patterning device table MT. Having traversed the patterning device MA, the beam B passes through the lens PL, which focuses the beam B onto a target portion C of the substrate W. With the aid of the second positioning means (and interferometric measuring means IF), the substrate table WT can be moved accurately, e.g. so as to position different target portions C in the path of the beam PB. Similarly, the first positioning means can be used to accurately position the patterning device MA with respect to the path of the beam B, e.g., after mechanical retrieval of the patterning device MA from a patterning device library, or during a scan. In general, movement of the object tables MT, WT will be realized with the aid of a long-stroke module (coarse positioning) and a short-stroke module (fine positioning), which are not explicitly depicted in Figure 25. However, in the case of a stepper (as opposed to a step-and-scan tool) the patterning device table MT may just be connected to a short stroke actuator, or may be fixed.

[0236] The depicted tool can be used in two different modes: - In step mode, the patterning device table MT is kept essentially stationary, and an entire patterning device image is projected in one go (i.e., a single “flash”) onto a target portion C. The substrate table WT is then shifted in the x and / or y directions so that a different target portion C can be irradiated by the beam PB; - In scan mode, essentially the same scenario applies, except that a given target portion C is not exposed in a single “flash”. Instead, the patterning device table MT is movable in a given direction (the so-called “scan direction”, e.g., the y direction) with a speed v, so that the projection beam B is caused to scan over a patterning device image; concurrently, the substrate table WT is simultaneously moved in the same or opposite direction at a speed V = Mv, in which M is the magnification of the lens PL (typically, M = 1 / 4 or 1 / 5). In this manner, a relatively large target portion C can be exposed, without having to compromise on resolution.

[0237] Figure 26 schematically depicts another exemplary lithographic projection apparatus LA in conjunction with the techniques described herein can be utilized.

[0238] The lithographic projection apparatus LA comprises: - a source collector module SO - an illumination system (illuminator) IL configured to condition a radiation beam B (e.g. EUV radiation). - a support structure (e.g. a patterning device table) MT constructed to support a patterning device (e.g. a mask or a reticle) MA and connected to a first positioner PM configured to accurately position the patterning device; - a substrate table (e.g. a wafer table) WT constructed to hold a substrate (e.g. a resist coated wafer) W and connected to a second positioner PW configured to accurately position the substrate; and - a projection system (e.g. a reflective projection system) PS configured to project a pattern imparted to the radiation beam B by patterning device MA onto a target portion C (e.g. comprising one or more dies) of the substrate W.

[0239] As here depicted, the apparatus LA is of a reflective type (e.g. employing a reflective patterning device). It is to be noted that because most materials are absorptive within the EUV wavelength range, the patterning device may have multilayer reflectors comprising, for example, a multi-stack of Molybdenum and Silicon. In one example, the multi-stack reflector has a 40 layer pairs of Molybdenum and Silicon where the thickness of each layer is a quarter wavelength. Even smaller wavelengths may be produced with X-ray lithography. Since most material is absorptive at EUV and x-ray wavelengths, a thin piece of patterned absorbing material on the patterning device topography (e.g., a TaN absorber on top of the multi-layer reflector) defines where features would print (positive resist) or not print (negative resist).

[0240] Referring to Figure 26, the illuminator IL receives an extreme ultra violet radiation beamfrom the source collector module SO. Methods to produce EUV radiation include, but are not necessarily limited to, converting a material into a plasma state that has at least one element, e.g., xenon, lithium or tin, with one or more emission lines in the EUV range. In one such method, often termed laser produced plasma ("LPP") the plasma can be produced by irradiating a fuel, such as a droplet, stream or cluster of material having the line-emitting element, with a laser beam. The source collector module SO may be part of an EUV radiation system including a laser, not shown in Figure 26, for providing the laser beam exciting the fuel. The resulting plasma emits output radiation, e.g., EUV radiation, which is collected using a radiation collector, disposed in the source collector module. The laser and the source collector module may be separate entities, for example when a CO2 laser is used to provide the laser beam for fuel excitation.

[0241] In such cases, the laser is not considered to form part of the lithographic apparatus and the radiation beam is passed from the laser to the source collector module with the aid of a beam delivery system comprising, for example, suitable directing mirrors and / or a beam expander. In other cases the source may be an integral part of the source collector module, for example when the source is a discharge produced plasma EUV generator, often termed as a DPP source.

[0242] The illuminator IL may comprise an adjuster for adjusting the angular intensity distribution of the radiation beam. Generally, at least the outer and / or inner radial extent (commonly referred to as σ-outer and σ-inner, respectively) of the intensity distribution in a pupil plane of the illuminator can be adjusted. In addition, the illuminator IL may comprise various other components, such as facetted field and pupil mirror devices. The illuminator may be used to condition the radiation beam, to have a desired uniformity and intensity distribution in its cross section.

[0243] The radiation beam B is incident on the patterning device (e.g., mask) MA, which is held on the support structure (e.g., patterning device table) MT, and is patterned by the patterning device. After being reflected from the patterning device (e.g. mask) MA, the radiation beam B passes through the projection system PS, which focuses the beam onto a target portion C of the substrate W. With the aid of the second positioner PW and position sensor PS2 (e.g. an interferometric device, linear encoder or capacitive sensor), the substrate table WT can be moved accurately, e.g. so as to position different target portions C in the path of the radiation beam B. Similarly, the first positioner PM and another position sensor PS1 can be used to accurately position the patterning device (e.g. mask) MA with respect to the path of the radiation beam B. Patterning device (e.g. mask) MA and substrate W may be aligned using patterning device alignment marks M1, M2 and substrate alignment marks P1, P2.

[0244] The depicted apparatus LA could be used in at least one of the following modes: 1. In step mode, the support structure (e.g. patterning device table) MT and the substrate table WT are kept essentially stationary, while an entire pattern imparted to the radiation beam is projected onto a target portion C at one time (i.e. a single static exposure). The substrate table WT is then shifted in the X and / or Y direction so that a different target portion C can be exposed.2. In scan mode, the support structure (e.g. patterning device table) MT and the substrate table WT are scanned synchronously while a pattern imparted to the radiation beam is projected onto a target portion C (i.e. a single dynamic exposure). The velocity and direction of the substrate table WT relative to the support structure (e.g. patterning device table) MT may be determined by the (de-)magnification and image reversal characteristics of the projection system PS. 3. In another mode, the support structure (e.g. patterning device table) MT is kept essentially stationary holding a programmable patterning device, and the substrate table WT is moved or scanned while a pattern imparted to the radiation beam is projected onto a target portion C. In this mode, generally a pulsed radiation source is employed and the programmable patterning device is updated as required after each movement of the substrate table WT or in between successive radiation pulses during a scan. This mode of operation can be readily applied to maskless lithography that utilizes programmable patterning device, such as a programmable mirror array of a type as referred to above.

[0245] Figure 27 shows the apparatus LA in more detail, including the source collector module SO, the illumination system IL, and the projection system PS. The source collector module SO is constructed and arranged such that a vacuum environment can be maintained in an enclosing structure 220 of the source collector module SO. An EUV radiation emitting plasma 210 may be formed by a discharge produced plasma source. EUV radiation may be produced by a gas or vapor, for example Xe gas, Li vapor or Sn vapor in which the very hot plasma 210 is created to emit radiation in the EUV range of the electromagnetic spectrum. The very hot plasma 210 is created by, for example, an electrical discharge causing at least partially ionized plasma. Partial pressures of, for example, 10 Pa of Xe, Li, Sn vapor or any other suitable gas or vapor may be required for efficient generation of the radiation. In some embodiments, a plasma of excited tin (Sn) is provided to produce EUV radiation.

[0246] The radiation emitted by the hot plasma 210 is passed from a source chamber 211 into a collector chamber 212 via an optional gas barrier or contaminant trap 230 (in some cases also referred to as contaminant barrier or foil trap) which is positioned in or behind an opening in source chamber 211. The contaminant trap 230 may include a channel structure. Contamination trap 230 may also include a gas barrier or a combination of a gas barrier and a channel structure. The contaminant trap or contaminant barrier 230 further indicated herein at least includes a channel structure, as known in the art.

[0247] The collector chamber 211 may include a radiation collector CO which may be a so-called grazing incidence collector. Radiation collector CO has an upstream radiation collector side 251 and a downstream radiation collector side 252. Radiation that traverses collector CO can be reflected off a grating spectral filter 240 to be focused in a virtual source point IF along the optical axis indicated by the dot-dashed line ‘O’. The virtual source point IF is commonly referred to as the intermediate focus, and the source collector module is arranged such that the intermediate focus IF is located at or near an opening 221 in the enclosing structure 220. The virtual source point IF is an image of the radiationemitting plasma 210.

[0248] Subsequently the radiation traverses the illumination system IL, which may include a facetted field mirror device 22 and a facetted pupil mirror device 24 arranged to provide a desired angular distribution of the radiation beam 21, at the patterning device MA, as well as a desired uniformity of radiation intensity at the patterning device MA. Upon reflection of the beam of radiation 21 at the patterning device MA, held by the support structure MT, a patterned beam 26 is formed and the patterned beam 26 is imaged by the projection system PS via reflective elements 28, 30 onto a substrate W held by the substrate table WT.

[0249] More elements than shown may generally be present in illumination optics unit IL and projection system PS. The grating spectral filter 240 may optionally be present, depending upon the type of lithographic apparatus. Further, there may be more mirrors present than those shown in the figures, for example there may be 1- 6 additional reflective elements present in the projection system PS than shown in Figure 21.

[0250] Collector optic CO, as illustrated in Figure 27, is depicted as a nested collector with grazing incidence reflectors 253, 254 and 255, just as an example of a collector (or collector mirror). The grazing incidence reflectors 253, 254 and 255 are disposed axially symmetric around the optical axis O and a collector optic CO of this type may be used in combination with a discharge produced plasma source, often called a DPP source.

[0251] Alternatively, the source collector module SO may be part of an LPP radiation system as shown in Figure 28. A laser LA is arranged to deposit laser energy into a fuel, such as xenon (Xe), tin (Sn) or lithium (Li), creating the highly ionized plasma 210 with electron temperatures of several 10's of eV. The energetic radiation generated during de-excitation and recombination of these ions is emitted from the plasma, collected by a near normal incidence collector optic CO and focused onto the opening 221 in the enclosing structure 220.

[0252] The embodiments may further be described using the following clauses: 1. A method for defect inspection, comprising: converting a two-dimensional image to a three-dimensional image via a trained convolutional neural network (CNN); calculating a defect strength at a location on the three-dimensional image; generating a binary map by aggregating the calculated defect strength; filtering the binary map to remove noise; and utilizing the binary map to guide inspection of a sample using an inspection tool. 2. The method of clause 1, wherein the CNN is trained via simulated two-dimensional images. 3. The method of clause 1 or 2, wherein the two-dimensional image comprises an experimentally obtained image of a sample. 4. The method of any one of clauses 1 to 3, wherein the trained CNN converts a firstreference image to a first three-dimensional reference image and a second reference image to a second three-dimensional reference image. 5. The method of clause 4, wherein the first reference image and a second reference image are coupled with metrology information of a sample. 6. The method of clause 4 or 5, wherein the first reference image and the second reference image comprise design data of a sample. 7. The method of clause 4 or 5, wherein the first reference image and the second reference image comprise previously collected images of a sample. 8. The method of any one of clauses 1 to 7, wherein calculating a defect strength is based on a mathematical operation that relates a defect strength signal to a signal over noise metric. 9. The method of clause 8, wherein calculating the defect strength is based on a threshold value of the signal over noise metric. 10. The method of any one of clauses 1 to 9, wherein the location comprises a pixel on the three-dimensional image. 11. The method of any one of clauses 1 to 10, wherein the generated binary map predicts a defect dimension on a sample. 12. The method of clause 11, wherein the generated binary map predicts a height of a defect or a length of a defect. 13. The method of any one of clauses 1 to 12, wherein filtering comprises applying a morphological operator to the generated binary map. 14. The method of clause 13, wherein the morphological operator applies a structural object shape or size to a predicted defect in the binary map. 15. The method of any one of clauses 1 to 12, wherein filtering comprises applying a Gaussian blur to the binary map. 16. The method of any one of clauses 1 to 15, wherein the defect predicted by the binary map comprises a toploss, scumming, or a footing defect. 17. The method of any one of clauses 1 to 16, wherein the two-dimensional image, the first reference image, the second reference image comprise inspection images of a sample. 18. The method of clause 17, wherein the inspection images comprise scanning electron microscope images. 19. The method of any one of clauses 1 to 18, wherein the three-dimensional image, the first reference three-dimensional image, and the second reference three-dimensional image comprise depth maps. 20. The method of any one of clauses 1 to 19, wherein the binary map is utilized to guide inspection of an optical inspection tool. 21. The method of any one of clauses 1 to 19, wherein the binary map is utilized to guide inspection of a charged particle beam apparatus.22. The method of clause 21, wherein the charged particle beam apparatus comprises a scanning electron microscope. 23. A system, comprising: an image inspection apparatus configured to scan a sample and generate an image; and one or more processors configured to perform instructions to cause the system to perform operations comprising: converting a two-dimensional image to a three-dimensional image via a trained convolutional neural network (CNN); calculating a defect strength at a location on the three-dimensional image; generating a binary map by aggregating the calculated defect strength; filtering the binary map to remove noise; and utilizing the binary map to guide inspection of a sample using an inspection tool. 24. The system of clause 23, wherein the CNN is trained via simulated two-dimensional images. 25. The system of clause 23 or 24, wherein the two-dimensional image comprises an experimentally obtained image of a sample. 26. The system of any one of clauses 23 to 25, wherein the trained CNN converts a first reference image to a first three-dimensional reference image and a second reference image to a second three-dimensional reference image. 27. The system of clause 26, wherein the first reference image and a second reference image are coupled with metrology information of a sample. 28. The system of clause 26 or 27, wherein the first reference image and the second reference image comprise design data of a sample. 29. The system of clause 26 or 27, wherein the first reference image and the second reference image comprise previously collected images of a sample. 30. The system of any one of clauses 23 to 29, wherein calculating a defect strength is based on a mathematical operation that relates a defect strength signal to a signal over noise metric. 31. The system of clause 30, wherein calculating the defect strength is based on a threshold value of the signal over noise metric. 32. The system of any one of clauses 23 to 31, wherein the location comprises a pixel on the three-dimensional image. 33. The system of any one of clauses 23 to 32, wherein the generated binary map predicts a defect dimension on a sample. 34. The system of clause 33, wherein the generated binary map predicts a height of a defect or a length of a defect. 35. The system of any one of clauses 23 to 34, wherein filtering comprises applying a morphological operator to the generated binary map.36. The system of clause 35, wherein the morphological operator applies a structural object shape or size to a predicted defect in the binary map. 37. The system of any one of clauses 23 to 34, wherein filtering comprises applying a Gaussian blur to the binary map. 38. The system of any one of clauses 23 to 37, wherein the defect predicted by the binary map comprises a toploss, scumming, or a footing defect. 39. The system of any one of clauses 23 to 38, wherein the two-dimensional image, the first reference image, the second reference image comprise inspection images of a sample. 40. The system of clause 39, wherein the inspection images comprise scanning electron microscope images. 41. The system of any one of clauses 23 to 40, wherein the three-dimensional image, the first reference three-dimensional image, and the second reference three-dimensional image comprise depth maps. 42. The system of any one of clauses 23 to 41, wherein the binary map is utilized to guide inspection of an optical inspection tool. 43. The system of any one of clauses 23 to 41, wherein the binary map is utilized to guide inspection of a charged particle beam apparatus. 44. The system of clause 43, wherein the charged particle beam apparatus comprises a scanning electron microscope. 45. A non-transitory computer readable medium comprising a set of instructions that is executable by one or more processors of a computing device to cause the computing device to perform operations for predicting a failure rate for a device, the operations comprising: converting a two-dimensional image to a three-dimensional image via a trained convolutional neural network (CNN); calculating a defect strength at a location on the three-dimensional image; generating a binary map by aggregating the calculated defect strength; filtering the binary map to remove noise; and utilizing the binary map to guide inspection of a sample using an inspection tool. 46. The non-transitory computer readable medium of clause 45, wherein the CNN is trained via simulated two-dimensional images. 47. The non-transitory computer readable medium of clause 45 or 46, wherein the two- dimensional image comprises an experimentally obtained image of a sample. 48. The non-transitory computer readable medium of any one of clause 45 to 47, wherein the trained CNN converts a first reference image to a first three-dimensional reference image and a second reference image to a second three-dimensional reference image. 49. The non-transitory computer readable medium of clause 48, wherein the first reference image and a second reference image are coupled with metrology information of a sample.50. The non-transitory computer readable medium of clause 48 or 49, wherein the first reference image and the second reference image comprise design data of a sample. 51. The non-transitory computer readable medium of clause 48 or 49, wherein the first reference image and the second reference image comprise previously collected images of a sample. 52. The non-transitory computer readable medium of any one of clauses 45 to 51, wherein calculating a defect strength is based on a mathematical operation that relates a defect strength signal to a signal over noise metric. 53. The non-transitory computer readable medium of clause 52, wherein calculating the defect strength is based on a threshold value of the signal over noise metric. 54. The non-transitory computer readable medium of any one of clauses 45 to 53, wherein the location comprises a pixel on the three-dimensional image. 55. The non-transitory computer readable medium of any one of clauses 45 to 54, wherein the generated binary map predicts a defect dimension on a sample. 56. The non-transitory computer readable medium of clause 55, wherein the generated binary map predicts a height of a defect or a length of a defect. 57. The non-transitory computer readable medium of any one of clauses 45 to 56, wherein filtering comprises applying a morphological operator to the generated binary map. 58. The non-transitory computer readable medium of clause 57, wherein the morphological operator applies a structural object shape or size to a predicted defect in the binary map. 59. The non-transitory computer readable medium of any one of clauses 45 to 56, wherein filtering comprises applying a Gaussian blur to the binary map. 60. The non-transitory computer readable medium of any one of clauses 45 to 59, wherein the defect predicted by the binary map comprises a toploss, scumming, or a footing defect. 61. The non-transitory computer readable medium of any one of clauses 45 to 60, wherein the two-dimensional image, the first reference image, the second reference image comprise inspection images of a sample. 62. The non-transitory computer readable medium of clause 61, wherein the inspection images comprise scanning electron microscope images. 63. The non-transitory computer readable medium of any one of clauses 45 to 62, wherein the three-dimensional image, the first reference three-dimensional image, and the second reference three-dimensional image comprise depth maps. 64. The non-transitory computer readable medium of any one of clauses 45 to 63, wherein the binary map is utilized to guide inspection of an optical inspection tool. 65. The non-transitory computer readable medium of any one of clauses 45 to 63, wherein the binary map is utilized to guide inspection of a charged particle beam apparatus. 66. The non-transitory computer readable medium of clause 65, wherein the chargedparticle beam apparatus comprises a scanning electron microscope.

[0253] The concepts disclosed herein may simulate or mathematically model any generic imaging system for imaging sub wavelength features, and may be especially useful with emerging imaging technologies capable of producing increasingly shorter wavelengths. Emerging technologies already in use include EUV (extreme ultra violet), DUV lithography that is capable of producing a 193nm wavelength with the use of an ArF laser, and even a 157nm wavelength with the use of a Fluorine laser. Moreover, EUV lithography is capable of producing wavelengths within a range of 20-5nm by using a synchrotron or by hitting a material (either solid or a plasma) with high energy electrons in order to produce photons within this range.

[0254] While the concepts disclosed herein may be used for imaging on a substrate such as a silicon wafer, it shall be understood that the disclosed concepts may be used with any type of lithographic imaging systems, e.g., those used for imaging on substrates other than silicon wafers.

[0255] As used herein, unless specifically stated otherwise, the term “or” encompasses all possible combinations, except where infeasible. For example, if it is stated that a database can include A or B, then, unless specifically stated otherwise or infeasible, the database can include A, or B, or A and B. As a second example, if it is stated that a database can include A, B, or C, then, unless specifically stated otherwise or infeasible, the database can include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0256] The descriptions above are intended to be illustrative, not limiting. Thus, it will be apparent to one skilled in the art that modifications may be made as described without departing from the scope of the claims set out below.

Claims

CLAIMS 1. A method for defect inspection, comprising: converting a two-dimensional image to a three-dimensional image via a trained convolutional neural network (CNN); calculating a defect strength at a location on the three-dimensional image; generating a binary map by aggregating the calculated defect strength; filtering the binary map to remove noise; and utilizing the binary map to guide inspection of a sample using an inspection tool.

2. The method of claim 1, wherein the CNN is trained via simulated two-dimensional images.

3. The method of claim 1, wherein the two-dimensional image comprises an experimentally obtained image of a sample.

4. The method of claim 1, wherein the trained CNN converts a first reference image to a first three-dimensional reference image and a second reference image to a second three-dimensional reference image.

5. The method of claim 4, wherein the first reference image and a second reference image are coupled with metrology information of a sample.

6. The method of claim 4 or 5, wherein the first reference image and the second reference image comprise design data of a sample or comprise previously collected images of a sample.

7. The method of claim 1, wherein calculating a defect strength is based on a mathematical operation that relates a defect strength signal to a signal over noise metric.

8. The method of claim 7, wherein calculating the defect strength is based on a threshold value of the signal over noise metric.

9. The method of claim 1, wherein the location comprises a pixel on the three- dimensional image.

10. The method of claim 1, wherein the generated binary map predicts a defect dimensionon a sample.

11. The method of claim 1, wherein filtering comprises applying a morphological operator to the generated binary map or applying a Gaussian blur to the binary map.

12. The method of claim 11, wherein the morphological operator applies a structural object shape or size to a predicted defect in the binary map.

13. The method of claim 1, wherein the defect predicted by the binary map comprises a toploss, scumming, or a footing defect.

14. The method of claim 1, wherein the two-dimensional image, the first reference image, the second reference image comprise inspection images of a sample.

15. The method of claim 1, wherein the three-dimensional image, the first reference three-dimensional image, and the second reference three-dimensional image comprise depth maps.

16. A system, comprising: an image inspection apparatus configured to scan a sample and generate an image; and one or more processors configured to perform instructions to cause the system to perform operations comprising: converting a two-dimensional image to a three-dimensional image via a trained convolutional neural network (CNN); calculating a defect strength at a location on the three-dimensional image; generating a binary map by aggregating the calculated defect strength; filtering the binary map to remove noise; and utilizing the binary map to guide inspection of a sample using an inspection tool.

17. A non-transitory computer readable medium comprising a set of instructions that is executable by one or more processors of a computing device to cause the computing device to perform operations for defect inspection, the operations comprising: converting a two-dimensional image to a three-dimensional image via a trained convolutional neural network (CNN); calculating a defect strength at a location on the three-dimensional image; generating a binary map by aggregating the calculated defect strength; filtering the binary map to remove noise; and utilizing the binary map to guide inspection of a sample using an inspection tool.

Images