Method and system for alignment based on deep generative models

Abstract

Methods and systems for deep learning alignment for semiconductor applications are provided. A method includes converting first actual information of an alignment target on a sample from design data to a sample image or converting a sample image to design data by inputting the first actual information into a deep generative model (e.g., GAN). The method also includes aligning the converted first actual information with second actual information of the alignment target, the second actual information having the same information type as the converted first actual information. The method further includes determining an offset between the converted first actual information and the second actual information based on a result of the alignment and storing the determined offset as an alignment-to-design offset for use in a process performed on the sample.

Description

Technical Field

[0001] The present invention generally relates to deep learning alignment for semiconductor applications. Certain embodiments relate to methods and systems for determining offsets in a process performed on a sample. Background Art

[0002] The following description and examples are not admitted to be prior art by virtue of their inclusion in this section.

[0003] Integrated circuit (IC) designs may be developed using methods or systems such as electronic design automation (EDA), computer-aided design (CAD), and other IC design software. Such methods and systems may be used to generate a circuit pattern database from an IC design. The circuit pattern database includes data representing a plurality of layouts of various layers of the IC. The data in the circuit pattern database may be used to determine the layouts of a plurality of reticles. The layout of a reticle generally includes a plurality of polygons defining features in a pattern on the reticle. Each reticle is used to fabricate one of the various layers of the IC. The layers of the IC may include, for example, junction patterns in a semiconductor substrate, gate dielectric patterns, gate electrode patterns, contact patterns in an interlayer dielectric, and interconnect patterns on metal layers.

[0004] The manufacture of semiconductor devices, such as logic and memory devices, typically includes processing a substrate, such as a semiconductor wafer, using a number of semiconductor manufacturing processes to form various features and multiple levels of the semiconductor device. For example, photolithography is a semiconductor manufacturing process that involves transferring a pattern from a reticule to a photoresist arranged on a semiconductor wafer. Additional examples of semiconductor manufacturing processes include, but are not limited to, chemical mechanical polishing (CMP), etching, deposition, and ion implantation. Multiple semiconductor devices may be manufactured in an arrangement on a single semiconductor wafer and then separated into individual semiconductor devices.

[0005] During the semiconductor manufacturing process, inspection processes are used at various steps to detect defects on the wafer to promote higher yields and therefore higher profits in the manufacturing process. Inspection has always been an important part of manufacturing semiconductor devices (such as ICs). However, as design rules shrink, the operation of semiconductor manufacturing processes can be closer to the limits of process performance capabilities. In addition, as design rules shrink, smaller defects can have an impact on the electrical parameters of the device, which drives more sensitive inspection. Therefore, as design rules shrink, the number of potential yield-related defects detected by inspection increases dramatically, and the number of harmful defects detected by inspection also increases dramatically.

[0006] Inspection systems and methods are increasingly designed to focus on the relationship between defects and design, since their impact on the sample design will determine whether and how important the defect is. For example, some methods have been developed for aligning inspection and design coordinates. One such method relies on the accuracy of the inspection system coordinate registration of the design. Another such method involves post-processing alignment of the inspection image patch and the associated design clip.

[0007] Some methods currently used perform patch-to-design alignment (PDA) training based on a setup die on a setup wafer and use a solid-based model to render images, as shown in FIG. 10 . For example, as shown in step 1000 of FIG. 10 , the methods currently used may scan a sample to find alignment targets. As shown in step 1002 , the method may acquire a design for each target. Using the images generated during the scan and the designs acquired for each target, the methods currently used may learn image rendering parameters from the example targets, as shown in step 1004 . Then, the methods currently used may use the learned image rendering parameters at each target to render an image from the design, as shown in step 1006 . As shown in step 1008 , the methods currently used may align the rendered image and the sample image at each target. Then, the methods currently used may determine the design-to-image offset for each target, as shown in step 1010 , and save the targets and offsets to a database for run-time verification, as shown in step 1012 . The run-time PDA process may then be performed in any suitable manner.

[0008] Although the currently used methods for setting up and executing PDAs have proven to be useful in many applications, there are many disadvantages in such methods and systems. For example, it is necessary to find good targets in order to create a solid-based model and render an image. Such methods and systems are also based on many assumptions, including that the design files that can be used for PDA settings contain all required layers (that is, all layers required to render an image that is sufficiently similar to the real image) and that good alignment targets can be found. However, due to many reasons, the design data may not be used to simulate all the layers required for a good image. In addition, sometimes there are no or not enough such good alignment targets. In either or both cases, it is difficult to create a solid-based model to find the best rendering parameters.

[0009] Accordingly, it would be advantageous to develop systems and methods for determining offsets in a process performed on a sample that do not have one or more of the disadvantages described above. Summary of the invention

[0010] The following description of various embodiments is not to be construed in any way as limiting the objectives of the appended claims.

[0011] One embodiment relates to a system configured to determine an offset in a process performed on a sample. The system includes one or more computer subsystems and one or more components executed by the one or more computer subsystems. The one or more components include a deep generative model. The one or more computer subsystems are configured to convert first actual information of an alignment target on a sample from a first type of information to a second type of information by inputting the first actual information into the deep generative model. If the first type of information is design data, then the second type of information is a sample image. If the first type of information is a sample image, then the second type of information is design data. The one or more computer subsystems are also configured to align the converted first actual information with second actual information of the alignment target, and the second actual information is the second type of information. The one or more computer subsystems are further configured to determine an offset between the converted first actual information and the second actual information based on the result of the alignment and store the determined offset as an alignment-to-design offset for use in a process performed on the sample with an output acquisition subsystem. The system may be further configured as described herein.

[0012] Another embodiment relates to a computer-implemented method for determining an offset in a process for performing on a sample. The method includes the converting, aligning, determining, and storing steps described above performed by one or more computer systems. Each of the steps of the method may be performed as further described herein. Additionally, the method described above may include any other steps of any other method described herein. Furthermore, the method may be performed by any of the systems described herein.

[0013] Additional embodiments relate to a non-transitory computer-readable medium storing program instructions executable on one or more computer systems for performing a computer-implemented method for determining an offset in a process performed on a sample. The computer-implemented method includes the steps of the method described above. The computer-readable medium may be further configured as described herein. The steps of the computer-implemented method may be further performed as described herein. In addition, the computer-implemented method (for which the program instructions may be executed) may include any other steps of any other method described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Further advantages of the present invention will become apparent to those skilled in the art with the benefit of the following detailed description of the preferred embodiments and upon reference to the accompanying drawings, in which:

[0015] Figure 1 and 2is a schematic diagram illustrating a side view of an embodiment of a system configured as described herein;

[0016] Figures 3 to 4 is a flow chart illustrating steps that may be performed by the embodiments described herein;

[0017] Figure 5 is a schematic diagram illustrating one example of a generator that may be included in an embodiment of a generative adversarial network (GAN);

[0018] Figure 6 is a schematic diagram illustrating an example of a discriminator that may be included in an embodiment of a GAN;

[0019] Figures 7 to 8 is a flow chart illustrating steps that may be performed by the embodiments described herein;

[0020] Fig. 9 is a block diagram illustrating one embodiment of a non-transitory computer-readable medium storing program instructions for causing a computer system to perform the computer-implemented methods described herein; and

[0021] FIG. 10 is a flow chart illustrating steps that may be performed by the presently used systems and methods.

[0022] Although the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown in the drawings by way of example and described in detail herein. The drawings may not be drawn to scale. However, it should be understood that the drawings and detailed description thereof are not intended to limit the invention to the particular forms disclosed, but on the contrary, the invention encompasses all modifications, equivalents and alternatives that fall within the spirit and scope of the invention as defined by the appended claims. DETAILED DESCRIPTION

[0023] The terms "design", "design data", and "design information" used interchangeably herein generally refer to the physical design (layout) of an IC or other semiconductor device and data derived from the physical design by complex simulations or simple geometric and Boolean operations. The design may include any other design data or design data proxies described in co-owned U.S. Pat. No. 7,570,796 issued to Zafar et al. on Aug. 4, 2009 and U.S. Pat. No. 7,676,077 issued to Kulkarni et al. on Mar. 9, 2010, both of which are incorporated by reference as if fully set forth herein. In addition, the design data may be standard cell library data, integrated layout data, one or more layers of design data, derivatives of the design data, and all or part of the chip design data. Furthermore, the "design", "design data", and "design information" described herein refer to information and data generated by a semiconductor device designer during the design process, and thus may be used in the embodiments described herein before the design is printed on any physical samples (e.g., reticles and wafers).

[0024] Turning now to the drawings, it should be noted that the drawings are not drawn to scale. In particular, the proportions of some elements of the drawings are greatly exaggerated to emphasize the characteristics of the elements. It should also be noted that the drawings are not drawn to the same scale. Elements shown in more than one figure that may be similarly configured have been indicated using the same reference numerals. Unless otherwise specified herein, any of the elements described and shown may include any suitable commercially available elements.

[0025] One embodiment is directed to a system configured to determine offsets for a process performed on a sample. The offsets determined by the embodiments described herein can be used for applications such as IC design through deep generative models to optical (or other) image alignment. An embodiment of this system is shown in Figure 1 The system includes one or more computer subsystems (eg, computer subsystems 36 and 102) and one or more components 100 executed by the one or more computer subsystems. The one or more components include a deep generative model 104, which is configured as further described herein.

[0026] In some embodiments, the sample is a wafer. The wafer may include any wafer known in semiconductor technology. Although some embodiments may be described herein with respect to one or more wafers, the embodiments are not limited to the samples with which they may be used. For example, the embodiments described herein may be used with samples such as reticles, flat panels, personal computer (PC) boards, and other semiconductor samples.

[0027] In some embodiments, the system includes an output acquisition subsystem including at least an energy source and a detector. The energy source is configured to generate energy directed to the sample. The detector is configured to detect energy from the sample and generate an output in response to the detected energy.

[0028] In one embodiment, the output acquisition subsystem is a light-based output acquisition subsystem. Figure 1 In the embodiment of the system shown in FIG. 1 , the output acquisition subsystem 10 includes an illumination subsystem configured to direct light to the sample 14. The illumination subsystem includes at least one light source. For example, Figure 1 , the illumination subsystem includes a light source 16. In one embodiment, the illumination subsystem is configured to direct light to the sample at one or more incident angles, which may include one or more oblique angles and / or one or more normal angles. Figure 1 , light from a light source 16 is directed through an optical element 18 and then a lens 20 to a beam splitter 21, which directs the light at a normal angle of incidence to a sample 14. The angle of incidence may include any suitable angle of incidence, which may vary depending on, for example, the characteristics of the sample, defects to be detected on the sample, measurements to be performed on the sample, etc.

[0029] The illumination subsystem may be configured to direct light to the sample at different incident angles at different times. For example, the output acquisition subsystem may be configured to change one or more characteristics of one or more elements of the illumination subsystem so that the light may be directed differently to the sample. Figure 1 In this one example, the output acquisition subsystem can be configured to move the light source 16, the optical element 18, and the lens 20 so that the light is directed to the sample at different incident angles.

[0030] In some examples, the output acquisition subsystem can be configured to direct light to the sample at more than one incident angle simultaneously. For example, the output acquisition subsystem can include more than one illumination channel, wherein one of the illumination channels can include, for example, Figure 1 1 and the other of the illumination channels (not shown) may include similar elements that may be configured differently or identically, or may include at least one light source and, possibly, one or more other components such as those described further herein. If this light is directed to the sample simultaneously with other light, one or more characteristics (e.g., wavelength, polarization, etc.) of the light directed to the sample at different angles of incidence may be different, so that light resulting from illuminating the sample at different angles of incidence may be distinguished from each other at a detector.

[0031] In another example, the illumination subsystem may include only one light source (eg, Figure 1) and the light from the light source may be separated into different optical paths (e.g., based on wavelength, polarization, etc.) by one or more optical elements (not shown) of the illumination subsystem. The light in each of the different optical paths may then be directed to the sample. Multiple illumination channels may be configured to direct light to the sample simultaneously or at different times (e.g., when the sample is sequentially illuminated using different illumination channels). In another example, the same illumination channel may be configured to direct light to samples with different characteristics at different times. For example, in some examples, the optical element 18 may be configured as a spectral filter and the properties of the spectral filter may be changed in a variety of different ways (e.g., by replacing the spectral filter) so that light of different wavelengths may be directed to the sample at different times. The illumination subsystem may have any other suitable configuration known in the art for directing light with different or the same characteristics to the sample sequentially or simultaneously at different or the same angles of incidence.

[0032] In one embodiment, the light source 16 may include a broadband plasma (BBP) light source. In this manner, the light generated by the light source and directed to the sample may include broadband light. However, the light source may include any other suitable light source, such as a laser, which may be any suitable laser known in the art and may be configured to generate light of any suitable wavelength known in the art. Additionally, the laser may be configured to generate monochromatic or nearly monochromatic light. In this manner, the laser may be a narrow band laser. The light source may also include a polychromatic light source that generates light of multiple discrete wavelengths or wavelength bands.

[0033] Light from optical element 18 may be focused by lens 20 to beam splitter 21. Although lens 20 is Figure 1 2 is shown as a single refractive optical element, but in reality lens 20 may include several refractive and / or reflective optical elements that combine to focus light from the optical element to the sample. Figure 1 The illumination subsystem shown in and described herein may include any other suitable optical elements (not shown). Examples of such optical elements include, but are not limited to, polarizing components, spectral filters, spatial filters, reflective optical elements, apodizers, beam splitters, apertures, and the like, which may include any such suitable optical elements known in the art. Additionally, the system may be configured to alter one or more elements of the illumination subsystem based on the type of illumination to be used for inspection, metrology, etc.

[0034] The output acquisition subsystem may also include a scanning subsystem configured to scan light over the sample. For example, the output acquisition subsystem may include a stage 22 on which the sample 14 is placed during inspection, measurement, etc. The scanning subsystem may include any suitable mechanical and / or robotic assembly (including the stage 22) that may be configured to move the sample so that light can be scanned over the sample. Additionally or alternatively, the output acquisition subsystem may be configured so that one or more optical elements of the output acquisition subsystem perform some scanning of light over the sample. The light may be scanned over the sample in any suitable manner.

[0035] The output acquisition subsystem further includes one or more detection channels. At least one of the one or more detection channels includes a detector configured to detect light from the sample as a result of illumination of the sample by the output acquisition subsystem and to generate an output in response to the detected light. For example, Figure 1 The output acquisition subsystem shown in FIG. 1 includes two detection channels, one detection channel formed by collector 24, element 26 and detector 28 and the other detection channel formed by collector 30, element 32 and detector 34. Figure 1 , two detection channels are configured to collect and detect light at different collection angles. In some examples, one detection channel is configured to detect specularly reflected light and the other detection channel is configured to detect light that is not specularly reflected (e.g., scattered, diffracted, etc.) from the sample. However, one or more of the detection channels may be configured to detect the same type of light (e.g., specularly reflected light) from the sample. Figure 1 An embodiment of an output acquisition subsystem including two detection channels is shown, but the output acquisition subsystem may include a different number of detection channels (e.g., only one detection channel or two or more detection channels). Figure 1 , but each of the collectors may include one or more refractive optical elements and / or one or more reflective optical elements.

[0036] One or more detection channels may include any suitable detector known in the art. For example, the detector may include a photomultiplier tube (PMT) or any other suitable non-imaging detector known in the art. If the detector is a non-imaging detector, each of the detectors may be configured to detect a particular characteristic of the scattered light (e.g., intensity), but may not be configured to detect such characteristic as a function of position in the imaging plane. Thus, the output generated by each of the detectors included in each of the detection channels may be a signal or data, but not an image signal or image data. In such examples, a computer subsystem (e.g., computer subsystem 36 of the system) may be configured to generate an image of the sample from the non-imaging output of the detector.

[0037] It should be noted that this article provides Figure 1to generally illustrate configurations of output acquisition subsystems that may be included in embodiments of the systems described herein. Obviously, as is typically performed when designing commercial inspection, metrology, and the like systems, the output acquisition subsystem configurations described herein may be altered to optimize the performance of the system. Additionally, the systems described herein may be implemented using existing inspection or metrology systems (e.g., by adding the functionality described herein to an existing inspection or metrology system) (e.g., the 29xx and 39xx series of tools, the SpectraShape series of tools, and the Archer series of tools commercially available from KLA Corporation of Milpitas, California). For some such systems, the implementations described herein may be provided as optical functionality of the system (e.g., in addition to other functionality of the system). Alternatively, the output acquisition subsystem described herein may be designed "from the ground up" to provide an entirely new system.

[0038] The computer subsystem 36 of the system may be coupled to the detector of the output acquisition subsystem in any suitable manner (e.g., via one or more transmission media, which may include "wired" and / or "wireless" transmission media) so that during scanning of the sample, the computer subsystem may receive the output generated by the detector. The computer subsystem 36 may be configured to perform a variety of functions using the output of the detector as described herein and any other functions further described herein. This computer subsystem may be further configured as described herein.

[0039] This computer subsystem (and other computer subsystems described herein) may also be referred to herein as a computer system. Each of the computer subsystems or systems described herein may take various forms, including a personal computer system, an image computer, a mainframe computer system, a workstation, a network appliance, an Internet appliance, or other devices. In general, the term "computer system" may be broadly defined to encompass any device having one or more processors that execute instructions from a memory medium. The computer subsystem or system may also include any suitable processor known in the art (e.g., a parallel processor). In addition, the computer subsystem or system may include a computer platform with high-speed processing and software, as a stand-alone or network tool.

[0040] If the system includes more than one computer subsystem, the different computer subsystems may be coupled to each other so that images, data, information, instructions, etc. may be transmitted between the computer subsystems, as further described herein. For example, computer subsystem 36 may be coupled to computer subsystem 102 by any suitable transmission medium (e.g., by Figure 1 The transmission medium may include any suitable wired and / or wireless transmission medium known in the art. Two or more of such computer subsystems may also be effectively coupled by a shared computer-readable storage medium (not shown).

[0041] Although the output acquisition subsystem is described above as an optical or light-based subsystem, the output acquisition subsystem can be an electron beam-based subsystem. For example, in one embodiment, the output acquisition subsystem is an electron beam output acquisition subsystem. In this manner, the energy source can be an electron beam source. Figure 2 In this one embodiment shown in , the output acquisition subsystem includes an electronic column 122 that is coupled to a computer subsystem 124.

[0042] Likewise Figure 2 , the electron column includes an electron beam source 126 configured to produce electrons that are focused by one or more elements 130 to a sample 128. The electron beam source may include, for example, a cathode source or emitter tip, and the one or more elements 130 may include, for example, a gun sight, an anode, a beam limiting aperture, a gate valve, a beam current selecting aperture, an objective lens, and a scanning subsystem, all of which may include any such suitable elements known in the art.

[0043] Electrons returning from the sample (eg, secondary electrons) may be focused by one or more elements 132 onto a detector 134. One or more elements 132 may include, for example, a scanning subsystem, which may be the same scanning subsystem included in element 130.

[0044] The electron column may include any other suitable elements known in the art. In addition, the electron column may be further configured as described in U.S. Pat. No. 8,664,594 issued to Jiang et al. on April 4, 2014, U.S. Pat. No. 8,692,204 issued to Kojima et al. on April 8, 2014, U.S. Pat. No. 8,698,093 issued to Gubbens et al. on April 15, 2014, and U.S. Pat. No. 8,716,662 issued to MacDonald et al. on May 6, 2014, which are incorporated by reference as if fully set forth herein.

[0045] Although the electron column Figure 2 The electron beam subsystem is shown as being configured so that electrons are directed to the sample at an oblique angle of incidence and scattered from the sample at another oblique angle, but it should be understood that the electron beam can be directed to the sample and scattered from the sample at any suitable angle. In addition, the electron beam subsystem can be configured to use multiple modes to generate an image of the sample (e.g., with different illumination angles, collection angles, etc.). The multiple modes of the electron beam subsystem can differ in any image generation parameter of the subsystem.

[0046] The computer subsystem 124 may be coupled to a detector 134, as described above. The detector may detect electrons returning from the surface of the sample, thereby generating an output used by the computer subsystem to form an electron beam image of the sample, which may include any suitable electron beam image. The computer subsystem 124 may be configured to use the output of the detector and / or the electron beam image to perform any of the functions described herein. The computer subsystem 124 may be configured to perform any additional steps described herein. Figure 2 The system of output acquisition subsystem shown in may be further configured as described herein.

[0047] It should be noted that this article provides Figure 2 to generally illustrate the configuration of an electron beam-based output acquisition subsystem that may be included in the embodiments described herein. As with the optical subsystem described above, the electron beam subsystem configuration described herein may be varied to optimize the performance of the subsystem, as is typically performed when designing a commercial inspection or metrology system. In addition, the systems described herein may be implemented using existing inspection, metrology, or other systems (e.g., by adding the functionality described herein to the existing system), such as tools commercially available from KLA. For some such systems, the embodiments described herein may be provided as optional functionality of the system (e.g., in addition to other functionality of the system). Alternatively, the systems described herein may be designed "from the ground up" to provide an entirely new system.

[0048] Although the output acquisition subsystem is described above as a light-based or electron-beam-based subsystem, the output acquisition subsystem may be an ion-beam-based subsystem. Figure 2 The configuration is shown in FIG. 1 , except that the electron beam source may be replaced with any suitable ion beam source known in the art. Thus, in one embodiment, the energy directed to the sample includes ions. Additionally, the output acquisition subsystem may be any other suitable ion beam-based output acquisition subsystem, such as those included in commercially available focused ion beam (FIB) systems, helium ion microscope (HIM) systems, and secondary ion mass spectrometry (SIMS) systems.

[0049] The output acquisition subsystem described herein may be configured to generate output for a sample in a variety of modes. In general, a "mode" is defined by the parameter values ​​of the output acquisition subsystem used to generate an image of the sample. Thus, the value of at least one of the parameters of the output acquisition subsystem may be different (except for the location on the sample where the output is generated). For example, in the optical subsystem, different modes may use different wavelengths of light for illumination. As further described herein (e.g., by using different light sources, different spectral filters, etc.), for different modes, the modes may differ in illumination wavelength. In another embodiment, different modes may use different illumination channels of the optical subsystem. For example, as mentioned above, the optical subsystem may include more than one illumination channel. Thus, different illumination channels may be used for different modes. The modes may differ in any one or more variable parameters of the output acquisition subsystem (e.g., illumination polarization, angle, wavelength, etc., detection polarization, angle, wavelength, etc.).

[0050] In a similar manner, the output generated by the electron beam subsystem may include outputs generated by the electron beam subsystem having two or more different values ​​of a parameter of the electron beam subsystem. The multiple modes of the electron beam subsystem may be defined by the values ​​of the parameters of the electron beam subsystem used to generate the output of the sample. Thus, the values ​​of different modes may be different for at least one of the electron beam parameters of the electron beam subsystem. For example, in one embodiment of the electron beam subsystem, different modes may use different incident angles for illumination.

[0051] The output acquisition subsystem embodiments described herein may be configured for use in inspection, metrology, defect review, or other quality control related processes performed on samples. Figure 1 and 2 Embodiments of the output acquisition subsystem in can be modified in one or more parameters to provide different output generation capabilities depending on the application in which it will be used. In this example, if the output acquisition subsystem shown in Figure 1 If the output acquisition subsystem in is used for defect review or metrology rather than inspection, it can be configured to have a higher resolution. Figure 1 and 2 The embodiments of the output acquisition subsystem in the present invention describe some common and various configurations of the output acquisition subsystem, which can be customized in a variety of ways that will be apparent to technicians in the field to produce output acquisition subsystems with different output generation capabilities that are more or less suitable for different applications.

[0052] As mentioned above, the optical, electron, and ion beam subsystems are configured to scan energy (e.g., light, electrons, etc.) over a physical version of the sample to produce an output for the physical version of the sample. In this manner, the optical, electron, and ion beam subsystems may be configured as "real" subsystems rather than "virtual" subsystems. However, the Figure 1The storage medium (not shown) and computer subsystem 102 in the embodiment may be configured as a "virtual" system. In particular, the storage medium and computer subsystem may be configured as a "virtual" inspection system, as described in commonly assigned U.S. Pat. No. 8,126,255 issued on February 28, 2012 to Bhaskar et al. and U.S. Pat. No. 9,222,895 issued on December 29, 2015 to Duffy et al., both of which are incorporated by reference in their entirety. The embodiments described herein may be further configured as described in these patents.

[0053] As mentioned above, one or more components executed by one or more computer subsystems include a deep generative model, e.g. Figure 1 104 is shown in FIG. 104. In general, "deep learning" (DL) (also called deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning (ML) based on a set of algorithms that attempt to model high-level abstractions in data. In a simple case, there may be two groups of neurons: one group that receives input signals and one group that sends output signals. When an input layer receives input, it passes a modified version of the input to the next layer. In DL-based models, there are many layers between the input and output (and the layers are not composed of neurons, but it can help to think of it that way), allowing the algorithm to use multiple processing layers, composed of multiple linear and nonlinear transformations.

[0054] DL is part of a broader family of ML methods based on learning representations of data. An observation (e.g., an image) can be represented in a variety of ways (e.g., a vector of intensity values ​​for each pixel), or in a more abstract way as a set of edges, regions of a particular shape, etc. Some representations simplify learning tasks (e.g., face recognition or facial expression recognition) better than others. One of the promises of DL is to replace hand-crafted features with efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction.

[0055] A "generative" model can generally be defined as a model that is probabilistic in nature. In other words, a "generative" model is not a model that performs forward simulation or rule-based methods, and thus, a physical model of the process involved in generating the actual image is not necessary. Instead, as further described herein, a generative model can be learned based on a suitable training data set (because its parameters can be learned). A generative model can be configured to have a DL architecture that can include multiple layers that perform a variety of algorithms or transformations. The number of layers included in the generative model can depend on the use case. For practical purposes, suitable layers range from 2 layers to dozens of layers. A deep generative model that learns the joint probability distribution (mean and variance) between a sample image (e.g., an image of an actual wafer) and a design (e.g., a CAD or vector representation of an expected layout) can be configured as further described herein.

[0056] In one embodiment, the deep generative model is a generative adversarial network (GAN). In another embodiment, the deep generative model is a conditional generative adversarial network (cGAN). The deep generative model may also include any other suitable type of GAN known in the art.

[0057] In general, GAN consists of two adversarial models, the generative model G, which captures the data distribution, and the discriminative model D, which estimates the probability that a given sample comes from the training data rather than G. G and D can be multi-layer perceptrons, i.e., nonlinear mapping functions. The generator constructs a priori noise distribution p z (z) to the data space G(z; θ g ) in order to learn the generator distribution p on the data x g , where G is a function with parameters θ g The generator is trained to produce simulated outputs that are indistinguishable from the real data distribution or the ground truth data distribution. The adversarially trained discriminator is trained to detect fake messages created by the generator. Both the generator and the discriminator are trained as well as possible so that the generator produces excellent "fake" outputs.

[0058] When GANs are extended to conditional models (cGANs), both the discriminator and the generator are conditioned on some additional information y, which is fed into the discriminator and the generator as an additional input layer.

[0059] GAN is inspired by game theory, where the generator G and the critic (i.e., the discriminator D) compete with each other to make each other stronger. The goal is to make the generated data distribution p g The true sample distribution p evaluated with the Jensen-Shannon divergence r As similar as possible:

[0060]

[0061] Where D KL is the Kullback-Leibler divergence, defined as follows:

[0062] and

[0063]

[0064] The generator outputs synthetic samples given a random noise variable input z. Over time, the generator is trained to capture the true data distribution by causing the discriminator to reject images that it considers to be bad fakes.

[0065] For conditional GANs, the generator and discriminator are conditioned on some additional information y, which will be the design clip or sample image in the embodiments described herein. The following minimax game with target L(D,G) describes the setup:

[0066]

[0067] This means that G is trained to increase the chance that D produces a high probability of being a fake instance, thus minimizing It is also necessary to maximize the encoding Given a fake sample G(z) to ensure that the discriminator's decision on real data is accurate, the discriminator expects to maximize To output a probability D(G(z)) close to zero.

[0068] The one or more computer subsystems are configured to convert the first actual information for the alignment target on the sample from a first type of information to a second type of information by inputting the first actual information into the deep generative model. The one or more computer subsystems may input the first actual information into the deep generative model in any suitable manner.

[0069] As used herein, the term "actual information" refers to information that has not been simulated by a DL or ML model (including those described herein), but rather is information obtained from sources external to the models described herein. Possible sources of actual information in the embodiments described herein include design databases, files, storage media, etc., or sample imaging tools (such as those described herein).

[0070] The terms "first" and "second" as used herein do not have any meaning other than to indicate different things. As further described herein, the first and second types of information are different types of information, where typically the first or second type of information is design data and the other of the first and second types of information is a sample image. In this manner, the different types of information as used herein are not intended to mean the same type of information having different characteristics. For example, the different types of information as used herein are not intended to mean the same type of sample images (e.g., optical images) having different resolutions or other different image characteristics. Instead, the different types of information described herein are located in different domains, such as a design data space and a sample image space.

[0071] Many semiconductor quality control type applications currently used in the art provide improved results by using design data to perform specific functions such as defect detection and classification. In order for the design data to be useful for any of these functions, it must be properly aligned with the results produced by the quality control tool for the sample. For example, unless the sample image is properly aligned with the design data, it is not possible to accurately determine which parts of the image correspond to which parts of the design. One difficulty in performing this alignment is that the sample image often does not look much like its corresponding design, which can be due to several factors, including how the structures in the design are formed on the sample and the ability of the tool to image the structures. In other words, the sample image and the corresponding design data can have substantially different image characteristics, which can make the alignment of the corresponding sample image and design data particularly difficult. In most cases, it is not possible to change the characteristics of the sample image to make image-to-design alignment easier. Therefore, creative and better methods of making alignment possible are constantly sought. The embodiments described herein advantageously provide better alignment for samples with design-to-image mismatches (such as described above).

[0072] If the first type of information is design data, then the second type of information is a sample image. Figure 3 cGAN training is shown by providing the generator network with designed images. Specifically, Figure 3 As shown in FIG. 3 , condition information Y 300 (additional data, such as design image 302 or condition image) is input to generator network 306 along with latent space vector z 304. Thus, generator network 306 outputs a generated image X fake 308, which is a simulated sample image, such as simulated sample image 310. In this case, the real image X real 312 is an actual optical image 314 of the sample at a location on the sample where a portion of the design shown in the design image 302 is formed. In this way, the condition information Y 300 (e.g., the design image) can be a training input in the training set, and the real image X real 312 may be the corresponding image of the sample designated as the training output in the training set.

[0073] The latent space vector may have any suitable configuration known in the art. In general, the latent space vector represents an initial image consisting only of noise, i.e., pixels with randomly assigned grayscale values. For computational reasons, this image is stored as a vector by concatenating each row in the image next to each other, so that a 2D image (e.g., 32 pixels x 32 pixels) becomes a vector of a corresponding size, e.g., 1x1024. Some information (e.g., color) may be added to this randomness to guide the deep generative model into a specific direction.

[0074] The resulting image Xfake 308 and condition information Y 300 are combined as the first input of the discriminator network 320. real 312 and conditional information Y 316 (e.g., design image 318) are combined as a second input to a discriminator network 320. The conditional information Y 300 and 316 in this case are different instances of the same portion of the same design. The discriminator network may produce an output that is input to a loss function (not shown), which may produce an output P(real) 322, which is the probability that the resulting stitched image is a real or good "fake" image.

[0075] Each of the designed, real, and generated images shown in the figures is not meant to illustrate any particular sample or its characteristics with which the embodiments described herein may be used. In a similar manner, each of the real and generated images shown in the figures is not meant to illustrate any particular actual or simulated image that may be generated for the sample. Instead, the designed, real, and generated images shown in the figures are intended only to facilitate understanding of the embodiments described herein. The images actually input and output from the generator will differ depending on the sample and its characteristics, which are related to its design and the configuration of the imaging system that generates the actual image for the sample, which is used to train the GAN, thereby affecting the simulated image generated by the GAN.

[0076] The loss function may be any suitable loss function known in the art, such as the loss function disclosed in "Image-to-Image Translation with Conditional Adversarial Networks" by Isola et al., IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 1125-1134, which is incorporated by reference as if fully set forth herein. The embodiments described herein may be further configured as described in such reference. Figure 3 The generator and discriminator networks shown in can be further configured as described herein.

[0077] If the first type of information is a sample image, then the second type of information is design data. Figure 4 We show the training of a cGAN by providing the generator with real sample images. Specifically, Figure 4 , condition information Y 400 (eg, sample image 402 or condition image) is input to a generator network 406 along with a latent space vector z 404, which may be configured as above. Thus, the generator network 406 outputs a generated image X fake 408, in this case it may be a fake design clip 410. In this case, the real image Xreal 412 is the actual design clip 414 of the sample located at the location where the sample image 402 was generated. In this way, the condition information Y 400 (e.g., the actual sample image) can be the training input in the training set, and the real image X real 412 may be its corresponding design clip designated as the training output in the training set. The generated image X fake 408 and condition information Y 400 are combined as the first input of the discriminator network 420. real 412 and conditional information Y 416 (e.g., sample image 418) are combined as a second input to the discriminator network 420. The conditional information Y 400 and 416 in this case are different instances of the same sample image. The discriminator network can produce an output that is input to a loss function (not shown), which can produce an output P(real) 422, which is the probability that the generated design clip is a real or good "fake" design clip. The loss function can be configured as described above. Figure 4 The generator and discriminator networks shown in can be further configured as described herein.

[0078] An example of a generator that may be included in an embodiment of a GAN configured as described herein is shown in Figure 5 In general, a GAN may include an encoder-decoder network that progressively downsamples until the bottleneck layer and then the process is reversed. Figure 5 , the generator may include an encoder 500 and a decoder 502. Each of the boxes 506, 508, and 510 shown in the encoder 500 represents an example of the final output layer size after repeated convolutions, batch normalization, and rectified linear units (ReLU) are enabled and maximum pooling is applied at the end of each segment. Figure 5 506, whose output may be input to box 508, etc. The encoder may generate feature layer 512.

[0079] The decoder may also include multiple blocks that perform different functions on the feature layer 512 input to the decoder. Each of the blocks 514, 516, and 518 in the decoder represents an example of the final output layer size after repeated upsampling (transposed convolution) and ReLU activation. Figure 5514, its output may be input to block 516, etc. The output 520 of the decoder may be any of the converted first actual information described herein.

[0080] In some examples, the GAN may include jumps 522 between corresponding blocks in the encoder and decoder, such as between blocks 506 and 518, between blocks 508 and 516, and between blocks 510 and 514. Jumps may be made to transfer low-level information learned between blocks. Jumps may have any suitable configuration determined in any suitable manner known in the art. Figure 5 The numbers below the inputs and outputs in the encoder indicate the size of the output of the box, and the numbers below the boxes in the decoder indicate the size of the input of each of the boxes.

[0081] An example of a discriminator that may be included in an embodiment of a GAN configured as described herein is shown in Figure 6 The input 600 of the discriminator may include two images (not shown in Figure 6 ), one generated by the GAN and the generator of the original image. The discriminator may include several layers, including layers 602, 604, 606, 608, and 610, each of which may include some combination of convolution, ReLU, and max aggregation layers. The convolution, ReLU, and max aggregation layers may have any suitable configuration known in the art. The output 612 of the discriminator may be P(x), the probability of how well the simulated image (i.e., the converted first real information) matches the original image (i.e., the second real information) or the probability that the simulated image is a good "fake" image or a bad "fake" image. Figure 6 The numbers below the input in indicate the size of the input. The numbers below the layers in the discriminator indicate the size of the layer’s output. Figure 6 , but the discriminator included in the GAN of the embodiments described herein may have any suitable number of layers determined in any suitable manner known in the art.

[0082] Additional descriptions of the general architecture and configuration of GANs and cGANs can be found in “Generative Adversarial Nets”, Goodfellow et al., arXiv:1406.2661, 10 June 2014, 9 pages, “Semi-supervised Learning with Deep Generative Models”, Kingma et al., NIPS 2014, 31 October 2014, 1–9 pages, “Conditional Generative Adversarial Nets”, Mirza et al., arXiv:1411.1784, 6 November 2014, 7 pages, and “Adversarial Autoencoders”. The present invention relates to a novel method for the translation of image-to-image complexes using state-of-the-art autoencoders, Makhzani et al., arXiv:1511.05644v2, May 25, 2016, p. 16, and “Image-to-Image Translation with Conditional Adversarial Networks” Isola et al., arXiv:1611.07004v2, November 22, 2017, p. 17, which are incorporated by reference as if fully set forth herein. The embodiments described herein may be further configured as described in these references.

[0083] The deep generative models described herein may or may not be trained by one of the one or more computer subsystems and / or components executed by a computer subsystem. For example, another method or system may train a deep generative model, which may then be stored for use as a component executed by a computer subsystem. The deep generative model may also be trained or constructed at any time prior to run time. In this one example, the deep generative model may be constructed during PDA training or setup. The deep generative models described herein may also be updated, retrained, or modified at any time after the training model is set up and / or deployed.

[0084] In this instance, Figure 7 Flowchart showing steps that may be performed by embodiments described herein when they are configured for converting a design into a sample image using a deep generative model. Steps 700, 702, 704, 706, 708, and 710 are performed during setup, and steps 712, 714, 716, and 718 are performed during runtime. Conversely, Figure 8Flowchart showing steps that may be performed by embodiments described herein when they are configured for converting a sample image into a design using a deep generative model. Steps 800, 802, 804, 806, 808, and 810 are performed during setup, and steps 812, 814, 816, and 818 are performed during runtime.

[0085] In one embodiment, the computer subsystem is configured to select alignment targets from a set image of a sample generated by the output acquisition subsystem. For example, regardless of whether the deep generative model is configured to convert a sample image into design data or convert design data into a sample image, the computer subsystem may select alignment targets for training and runtime. The alignment targets selected for training and runtime may or may not be the same. For example, the same alignment targets may be used to perform training and runtime, but there are fewer examples of alignment targets for training relative to runtime. The opposite is also possible. The alignment targets selected by the computer subsystem may also include different alignment targets with different characteristics, not just different locations on the sample or in the design. The alignment targets may be selected as further described herein or in any other suitable manner known in the art.

[0086] Examples of design-based alignment are described in U.S. Pat. Nos. 9,830,421, issued to Bhattacharyya et al. on Nov. 28, 2017, 10,620,135, issued to Brauer on Apr. 14, 2020, and 10,698,325, issued to Brauer on Jun. 30, 2020, which are incorporated by reference as if fully set forth herein. The embodiments described herein may be configured to select an alignment target as described in these patents and may be further configured as described in these patents.

[0087] like Figure 7 Step 700 and Figure 8 As shown in step 800 of , the computer subsystem may scan the entire die to find 2D unique targets that are evenly distributed across the die. Scanning the entire die on the sample or another suitable area on the sample may be performed as further described herein. The unique targets may be unique in any suitable manner known in the art that makes the targets suitable for alignment purposes. The 2D unique targets may be found in any suitable manner, such as by searching within a predetermined image window for a pattern that is different from any other pattern in the window. For example, the unique targets may be patterns that have a unique shape compared to other patterns (e.g., image frames or jobs) within a predetermined search window, patterns that have a unique spatial relationship relative to each other within a predetermined search window, etc.

[0088] The alignment targets are preferably 2D, since they can be used to perform alignment in both the x and y directions, although this is not required. For example, alignment targets may be selected so that one or more are used for alignment only in the x direction and one or more others are used for alignment only in the y direction.

[0089] Additionally, although it may be preferred to find and use alignment targets that are evenly distributed across the die or other areas on the sample, this is not required. Figure 7 and 8 Although the alignment target is selected as shown in steps 700 and 800 of , the embodiments described herein may be used with any suitable alignment target known in the art selected in any suitable manner known in the art. Although it may be practical to select multiple unique targets for use in the embodiments described herein, in general, any one or more unique targets may be selected. Each of the unique targets may be different from each other in any unique manner. In addition, the unique targets may include more than one instance of the same unique target.

[0090] The one or more computer subsystems may receive a design for each of the unique objects found in the sample image. The one or more computer subsystems may receive the design in any suitable manner, such as by searching for a design of the sample based on information of the unique objects determined from the image generated by scanning, by requesting a portion of the design (e.g., a design clip) at the location of the unique objects from a storage medium or computer system containing the design, etc. The design received by the one or more computer subsystems may include any of the designs, design data, or design information further described herein.

[0091] In another embodiment, the computer subsystem is configured to generate a training set, the training set including training images of samples generated by the output acquisition subsystem and corresponding design data of the samples, and the deep generative model is trained with the training set, and the training images are generated at positions independent of the position of the calibration target or any other calibration target on the sample. For example, the deep generative model described herein can learn from a structure, even if the structure used for learning is not a good alignment site for traditional PDA technology. In other words, although the alignment target is necessary for the other steps described herein, the training of the deep generative model does not have to be performed using only the alignment target image and design or even any alignment target image and design. It is advantageous to be able to train the deep generative model with substantially any sample image that can be associated with the design data, especially when there are not many suitable alignment targets on the sample. In other words, there may be examples where the number of feasible alignment targets on the sample makes it difficult or even impossible to generate alignment targets for only the training set. In such (and any other) cases, the training set may include unaligned target images and design data to ensure that the deep generative model can be trained correctly.

[0092] In an additional embodiment, the computer subsystem is configured to generate a training set, the training set including training images of samples generated by the output acquisition subsystem and corresponding design data of the samples, and train the deep generative model with the training set, and the corresponding design data included in the training set includes design data of less than all layers on the sample that affect the training image. For example, the embodiments described herein provide highly accurate alignment without requiring complete knowledge of all layer information. It is advantageous to be able to generate sample images from less than all design data that have an impact on the sample image (or vice versa) because in many examples, the entire design file is not available in cases such as those described herein. In other cases, it may be difficult to know a priori which design layers have an impact on the sample image and are therefore applied to forward type simulations. In contrast, as long as the design data used to train the deep generative model corresponds at least to some extent to the sample image used for training, the deep generative model described herein will be able to generate simulated sample images from the design data (or vice versa). In other words, due to the nature of the deep generative model itself, even if the design data used for training lacks some pattern features that appear in the sample image or have an impact on the sample image, the embodiments described herein should be able to successfully generate a trained deep generative model.

[0093] In any of the embodiments described herein, generating a training set may include storing training images and corresponding design data in any suitable manner and in any suitable format such that the training set can be used to train a deep generative model. The computer subsystem may store information for the training set in any of the computer-readable storage media described herein. The training set may be used by the computer subsystem in any suitable manner known in the art, including being divided into subsets for training and validation, being updated, modified, or replaced over time or when processes change, and the like.

[0094] The deep generative model may then be trained based on a training set comprising sample patch images with corresponding design clips. The sample images are training inputs and the corresponding design clips are training outputs or vice versa depending on how the deep generative model will convert the first actual information and the type of information of the first actual information and the second actual information. Training may include inputting the training input into the deep generative model and changing one or more parameters of the deep generative model until the output produced by the deep generative model matches (or substantially matches) the training output. Training may include changing any one or more trainable parameters of the deep generative model. For example, one or more parameters of the deep generative model trained by the embodiments described herein may include one or more weights for any layer of the deep generative model having trainable weights. In this one example, the weights may include weights of a convolutional layer, but not weights of an aggregation layer.

[0095] Although some embodiments are described herein with respect to deep generative models, the embodiments described herein may include or use multiple deep generative models. In this one example, different deep generative models may be trained and used for different modes of the output acquisition subsystem as described herein. For example, an alignment performed for one mode is not necessarily transferable to another mode. In other words, by performing alignment for one mode, not all modes may be aligned with the design (or at least not aligned with the design with sufficient accuracy). In most cases, different modes of inspection or other imaging tools will produce images and / or outputs that differ from each other in one of several possible ways, such as noise level, contrast, resolution, image type (e.g., DF vs. BF, optical vs. electron beam, etc.), and the like. Therefore, if a deep generative model is trained for one mode of the tool, it may not be appropriate to be trained for another mode of the tool to convert information. Thus, multiple deep generative models may be trained separately and independently, one for each mode of interest, and image-to-design alignment may be performed separately and independently for different modes used in a single process performed on a sample. In this way, when different modes are used to generate images of a sample, images generated in more than one mode may be aligned with the design of the sample. The same pre-trained deep generative network can be used for each mode, but this is not required. Each trained deep generative model can then be used to perform mode-specific transformations.

[0096] When the first actual information is design data, the first actual information input to each different trained deep generative model can be the same design clip or different parts of the same design, because the design of the sample will not change due to the mode. When the first actual information is a sample image, then the input of each different trained deep generative model will be different.

[0097] When the first type of information is design data and the second type of information is sample images, the generator is trained to generate sample images that are indistinguishable from real sample images. When the first type of information is sample images and the second type of information is design data, the generator is trained to generate design clips that are indistinguishable from real design clips. In either case, the adversarially trained discriminator is trained to classify between fake and real tuples during training.

[0098] Once the deep generative model is trained, it can be used to transform the first actual information of the target from one type of information to another type of information. Figure 7 As shown in Figure 7In an embodiment where the first type of information is design data and the second type of information is a sample image, the one or more computer subsystems may obtain the design for each target, as shown in step 702, which may be performed as further described herein. As shown in step 704, the one or more computer subsystems may then create a sample patch image from the design at each target through the trained GAN. In this way, the GAN or another deep generative model described herein may be used to generate a fake sample image for any given design clip, which will then be used to align the sample image to calculate the offset, as further described herein. In other words, after training, the generator is used to create simulated sample images for pattern to design alignment (PDA) so that the actual sample image can be aligned with the simulated sample image and thereby aligned with its corresponding design clip.

[0099] In this way, a novel feature of the embodiments described herein is that they can be configured to use GAN or other deep generative models described herein for learning how artificial patch images should look based on design files. In addition, another novel feature of the embodiments described herein is that they can be configured to generate artificial patch images for PDAs through GAN or other deep generative models described herein.

[0100] To do the reverse, the design clip is replaced by the sample image and vice versa. Figure 8 In the display, Figure 8 In an embodiment where the first type of information is a sample image and the second type of information is design data, the one or more computer subsystems may obtain a design for each target, as shown in step 802, which may be performed as further described herein. Then, the one or more computer subsystems may create a design clip from the optical (or other) patch image at each target via a trained GAN (or other deep generative model described herein), as shown in step 804. In this manner, the GAN (or other deep generative) model is used to generate a fake design clip for any given patch image, which will then be used to align the fake design clip with the actual design clip to calculate the offset therebetween, which may be performed as further described herein.

[0101] In this way, a novel feature of the embodiments described herein is that they can be configured to generate design clips from sample images through GAN or other deep generative model-based image reconstruction. In addition, another novel feature of the embodiments described herein is that they can be configured to use design clips instead of sample images for alignment.

[0102] In one embodiment, the sample image is a low-resolution sample image. For example, for a process such as inspection, a sample image generated during the process may be "low-resolution" because it has a lower resolution than other types of information used by the embodiments described herein. In other words, when one type of information in the embodiments described herein is design information and the other type of information is a sample image, the sample image may have a resolution lower than (and even much lower than) the design data, even though the design data is generally considered to have no resolution. For example, unlike design data in which all features of the sample are perfectly resolved, in the sample images described herein, not all (or even none) of the features on the sample may be resolved. In this manner, the conversions described herein are not merely changing the format of the data or image (e.g., from binary to grayscale) or applying one or more filters (e.g., like a low-pass filter or a corner rounding filter) to the data or image, but involve converting one type of data or image into another completely different type of data or image. In this manner, the embodiments described herein may advantageously provide better alignment for samples (e.g., wafers) having a design-to-optical mismatch (e.g., appearance difference) that makes it impractical or even impossible to align the design directly to the sample image.

[0103] In another embodiment, when the second type of information is design data, the conversion includes binarizing the output of the deep generative model so that the converted first actual information includes binarized design data. For example, the output of the deep generative model may include a grayscale simulated design data image. In such embodiments, one or more computer subsystems may be configured to generate binarized design data (or simulated binary design data images) from the grayscale simulated design data images. Thus, the conversion step may include inputting the first actual information into the deep generative model and then binarizing the output of the deep generative model. Although a particularly suitable manner for generating a simulated binary design data image is further described herein, the computer subsystem may generate the simulated binary design data image in any other suitable manner known in the art.

[0104] In one embodiment, generating a simulated binary design data image includes thresholding the grayscale simulated design data image to binarize the grayscale simulated design data image and matching the nominal size of patterned features in the grayscale simulated design data image with the nominal size of patterned features in the design data of the sample. For example, typically the ground truth design image is binary or ternary (depending on whether the design image is for only one layer on the sample or two layers on the sample), and the rendered grayscale design image generated by the deep generative model may have a substantially large histogram peak corresponding to the ground truth image value. In order to perform alignment using the rendered design image generated by the deep generative model, the computer subsystem may threshold the rendered design image into binary or ternary values ​​corresponding to the ground truth. In some such examples, the computer subsystem may generate a histogram of grayscale values ​​in the grayscale simulated design data image and then apply a threshold to the histogram to thereby binarize the image. The threshold is selected to match the pattern width (or other size) of the features in the design. In this manner, the computer subsystem may threshold the simulated database image to binarize and match the nominal real design pattern width (or other dimension), and then, as further described herein, align it with the real design pattern. In other words, in order to use the simulated design data image for alignment, the computer subsystem may threshold the simulated design clip to match the high resolution image design rule (DR) width.

[0105] The one or more computer subsystems are further configured to align the converted first actual information with the second actual information of the alignment target. The second actual information is information of the second type. For example, in the case where the first type of information is design data and the second type of information is a sample image, such as Figure 7 As shown in FIG. 7 , one or more computer subsystems align the rendering image (i.e., the converted first actual information) and the sample image at each target (i.e., the second actual information) as shown in step 706. In contrast, in the case where the first type of information is the sample image and the second type of information is the design data, as shown in FIG. Figure 8 As shown in step 806 of , one or more computer subsystems align the IC design (ie, the second reality information) with the design clips generated at each target (ie, the converted first reality information).

[0106] In some embodiments, aligning the transformed first actual information with the second actual information is performed based on a normalized sum of squared differences (NSSD). Figure 7 Step 706 and Figure 8 Step 806 of may be performed using NSSD of the converted first actual information and second actual information to calculate the alignment offset. NSSD may be performed using any suitable method, algorithm, function, etc. known in the art. However, these alignment steps may be performed using any other suitable alignment method or algorithm known in the art.

[0107] The one or more computer subsystems are further configured to determine an offset between the converted first actual information and the second actual information based on the result of the alignment. This offset can be determined in any suitable manner and can be represented in any suitable manner (e.g., as a Cartesian offset, as a two-dimensional function, etc.). This offset can be determined in any suitable manner known in the art and can have any format known in the art.

[0108] In some embodiments, the conversion and alignment steps are performed for additional alignment targets on the sample, and determining the offset includes determining different offsets for different frames of output generated by the output acquisition subsystem for the sample during the process. Figure 7 Step 708 and Figure 8 As shown in step 808 of , one or more computer subsystems may determine a design-to-image offset for each inspection frame and the target.

[0109] The one or more computer subsystems are also configured to store the determined offset as an alignment-to-design offset for use in a process performed on a sample with an output acquisition subsystem. The computer subsystem may store the alignment-to-design offset in any suitable computer-readable storage medium. The alignment-to-design offset may be stored with any of the results described herein and may be stored in any manner known in the art. The storage medium may include any storage medium described herein or any other suitable storage medium known in the art. After the alignment-to-design offset has been stored, the alignment-to-design offset may be accessed in the storage medium and used by any of the method or system embodiments described herein, formatted for display to a user, used by another software module, method or system, etc.

[0110] In some embodiments, the conversion and alignment steps are performed for additional alignment targets on the sample, determining the offsets includes respectively determining different offsets for different frames of output generated by the output acquisition subsystem for the sample during the process, and storing the determined offsets includes storing the different offsets. In another embodiment, the one or more computer subsystems are configured to store second actual information of the alignment target for use in setting the alignment target image during a process performed on the sample with the output acquisition subsystem. For example, Figure 7 Step 710 and Figure 8 As shown in step 810 of , one or more computer subsystems may save the target and offset to a database for run-time verification (or another process described herein). These steps may be performed as further described herein.

[0111] In one embodiment, storing the alignment-to-design offset includes storing the alignment-to-design offset for use in a process performed on the sample and in a process performed on at least one other sample of the same type as the sample. For example, embodiments described herein may set the alignment of a sample for a process recipe, which may be performed as part of setting up, creating, calibrating, or updating a recipe. The recipe may then be stored and used by embodiments described herein (and / or another system or method) to perform a process on the sample and / or other samples to thereby generate information (e.g., defect information) for the sample and / or other samples. In this manner, an alignment-to-design offset may be generated and stored once per sample layer, and the alignment-to-design offset may be used for processes performed on multiple samples of the same layer. Of course, as with any procedure, if the process performed on the sample layer is updated, calibrated, modified, retrained, etc., then the alignment-to-design offset may also be updated, calibrated, modified, retrained, etc. in the same manner as any other process parameter. Updating, calibrating, modifying, retraining, etc. the alignment-to-design offset may be performed in the same manner as described herein for determining the offset in a process performed on the sample. In this manner, embodiments described herein may be configured to repeat the steps described herein at different times to modify a previously set up process.

[0112] The embodiments described herein may also perform the process after the run-time alignment sample as described above. In one embodiment, the process is an inspection process. However, the process may include any of the processes described herein, such as inspection, defect review, metrology, and the like. The computer subsystem may be configured to store information of the sample generated by executing the process, such as information of detected defects in any suitable computer-readable storage medium. The information may be stored with any of the results described herein and may be stored in any manner known in the art. The storage medium may include any storage medium described herein or any other suitable storage medium known in the art. After the information has been stored, the information may be accessed in the storage medium and used by any of the method or system embodiments described herein, formatted for display to a user, used by another software module, method or system, etc.

[0113] Thus, as described herein, embodiments may be used to set up a new process or recipe. Embodiments may also be used to modify an existing process or recipe, whether it is a process or recipe for a sample or created for one sample and being adapted for another sample. In addition, the embodiments described herein are not limited to inspection process creation or modification. For example, the embodiments described herein may also be used to set up or modify processes for metrology, defect review, etc. in a similar manner. In particular, determining offsets in the process and performing sample-to-design alignment as described herein may be performed regardless of the process being set up or modified. Thus, the embodiments described herein may be used to establish or modify not only an inspection process, but also any quality control type process performed on the samples described herein.

[0114] In one embodiment, one or more computer subsystems are configured to perform the process on the sample with the output acquisition subsystem, and the process includes aligning a runtime alignment target image generated for the sample in the process with a setup alignment target image, the runtime alignment target image being the second type of information, determining a runtime-to-setup offset based on a result of aligning the runtime alignment target image with the setup alignment target image, and determining a runtime-to-design offset based on the runtime-to-setup offset and the alignment-to-design offset. For example, Figure 7 As shown in step 712 of , during the process, one or more computer subsystems may align the setup image with the runtime image at each target. Thus, in this embodiment, the setup image is the rendered sample image and the runtime image is the actual sample image. In contrast, Figure 8 As shown in step 812 of , during the process, one or more computer subsystems may align the setup design clips with the runtime design clips at each target. Thus, in this embodiment, the setup image is the actual design data, and the runtime image is the rendered design data generated from the runtime sample image. Figure 7 Step 714 and Figure 8 As further shown in step 814 of , one or more computer subsystems may determine the offset between the setup and run time of each inspection frame. Figure 7 Step 716 and Figure 8 As shown in step 816 of , one or more computer subsystems may determine the offset between the design and run time of each inspection frame. All of these steps may be performed as further described herein.

[0115] Determining the runtime-to-design offset based on the runtime-to-setup offset and the alignment-to-design offset may include modifying the alignment-to-design offset by or using the runtime-to-setup offset in any suitable manner. This step may be performed to correct the alignment-to-design offset for any differences in the alignment of the setup samples and the runtime samples with the output acquisition subsystem and any other differences between the setup samples and the runtime samples, such as differences in the placement of alignment targets on different samples.

[0116] In some such embodiments, the computer subsystem is configured to generate a runtime alignment target image by inputting a sample image generated at the location of the alignment target on the sample during the process into a deep generative model, by converting the sample image generated at the location of the alignment target on the sample during the process into runtime alignment target design data. For example, at runtime, a GAN or other deep generative model described herein generates a forged design clip from the sample image, which is then aligned with the PDA settings database image.

[0117] In another such embodiment, the process includes determining a region of interest in output generated by an output acquisition subsystem during a process performed on a sample based on a run-time versus design shift. Figure 7 Step 718 and Figure 8 As shown in step 818 of , one or more computer subsystems may place regions of interest based on the offset correction. A "region of interest," as commonly referred to in the art, is an area on a sample that is of concern for inspection purposes. Sometimes, regions of interest are used to distinguish between areas on a sample that have been inspected and areas on the sample that are not inspected during the inspection process. Additionally, regions of interest are sometimes used to distinguish between areas on a sample that will be inspected using one or more different parameters. For example, if a first area of ​​a sample is more critical than a second area on the sample, then the first area may be inspected at a higher sensitivity than the second area, such that defects are inspected at a higher sensitivity in the first area. Other parameters of the inspection process may be changed from regions of interest to regions of interest in a similar manner.

[0118] In these embodiments, one or more computer subsystems may use zero region of interest boundaries in the x and y directions. For example, because the embodiments described herein may align the sample image with the design with substantially high accuracy, the region of interest may be positioned in the sample image with substantially high accuracy. Thus, boundaries that are typically used to artificially increase the region of interest to account for any errors in the placement of the region of interest may be effectively eliminated by the embodiments described herein. Placing the region of interest with such high accuracy and eliminating the region of interest boundary factor is advantageous for a number of reasons, including significantly reducing the detection of nuisances on the sample and improving the detection of defects of interest (DOI) on the sample.

[0119] In one embodiment, the computer subsystem is configured to store the determined offset as an alignment to design offset for use in processes performed on additional samples with the output acquisition subsystem, and when the second type of information is design data, if the second type of information is a sample image, the converted first actual information is less susceptible to variations in the one or more processes performed on the additional samples. For example, when the deep generative model is configured to convert the sample image into design data, the design image generated from the sample image can potentially be less susceptible to process variations than when the deep generative model is configured to generate the sample image from the design data.

[0120] In another embodiment, the computer subsystem is configured to store the determined offset as an alignment-to-design offset for use in a process performed on additional samples with the output acquisition subsystem, and when the second type of information is design data, the alignment performed in the process using the alignment-to-design offset is less susceptible to variations in one or more processes performed on the additional samples than when the second type of information is a sample image. For example, performing PDA on a binarized design rendition with a real design clip may be less susceptible to process variations than when performing PDA using a sample image.

[0121] The results and information generated by performing the processes described herein on a sample or other samples of the same type may be used by the embodiments and / or other systems and methods described herein in a variety of ways. Such functions include, but are not limited to, changing a process, such as a manufacturing process or step, that has been or will be performed on a sample or another sample in a feedback or feedforward manner. For example, the computer subsystem may be configured to determine one or more changes to a process that has been or will be performed on a sample inspected as described herein based on detected defects. Changes to the process may include any suitable changes to one or more parameters of the process. The computer subsystem preferably determines those changes so that defects on other samples on which the modified process is performed can be reduced or prevented, defects on the sample can be corrected or eliminated in another process performed on the sample, defects can be compensated for in another process performed on the sample, etc. The computer subsystem may determine such changes in any suitable manner known in the art.

[0122] Those changes may then be sent to a semiconductor manufacturing system (not shown) or storage media (not shown) accessible to the computer subsystem and semiconductor manufacturing system. The semiconductor manufacturing system may or may not be part of the system embodiments described herein. For example, the computer subsystem and / or output acquisition subsystem described herein may be coupled to the semiconductor manufacturing system, for example, via one or more common elements, such as a housing, a power supply, a sample handling device or mechanism, etc. The semiconductor manufacturing system may include any semiconductor manufacturing system known in the art, such as a lithography tool, an etching tool, a chemical mechanical polishing (CMP) tool, a deposition tool, and the like.

[0123] Each of the embodiments of each of the systems described above may be combined together into a single embodiment.

[0124] Another embodiment relates to a computer-implemented method for determining an offset in a process performed on a sample. The method includes the steps of converting, aligning, determining, and storing as further described herein. These steps are performed by one or more computer systems. One or more components are executed by one or more computer systems, and one or more components include a deep generative model.

[0125] Each of the steps of the method may be performed as further described herein. The method may also include any other steps that may be performed by the system, computer system, and / or deep generative model described herein. The computer system may be configured according to any of the embodiments described herein, such as the computer subsystem 102. The deep generative model may be configured according to any of the embodiments described herein. In addition, the method described above may be performed by any of the system embodiments described herein.

[0126] Additional embodiments are directed to a non-transitory computer-readable medium storing program instructions executable on one or more computer systems for performing a computer-implemented method for determining a deviation in a process performed on a sample. This one embodiment is shown in Fig. 9 In particular, Fig. 9 , non-transitory computer readable medium 900 includes program instructions 902 executable on a computer system 904. The computer-implemented method may include any of the steps of any of the methods described herein.

[0127] Program instructions 902 implementing methods such as those described herein may be stored on a computer readable medium 900. The computer readable medium may be a storage medium such as a magnetic or optical disk, tape, or any other suitable non-transitory computer readable medium known in the art.

[0128] The program instructions may be implemented in any of a variety of ways, including (among others) process-based techniques, component-based techniques, and / or object-oriented techniques. For example, the program instructions may be implemented using ActiveX controls, C++ objects, JavaBeans, Microsoft Foundation Classes ("MFC"), SSE (Streaming SIMD Extensions), or other techniques or methods as desired.

[0129] Computer system 904 may be configured according to any of the embodiments described herein.

[0130] In view of this description, further modifications and alternative embodiments of various aspects of the present invention will be apparent to those skilled in the art. For example, methods and systems for determining offsets in a process for performing on a sample are provided. Accordingly, this description is interpreted as illustrative only and for the purpose of teaching those skilled in the art to implement the general method of the present invention. It should be understood that the forms of the present invention shown and described herein should be considered as currently preferred embodiments. Elements and materials may replace the elements and materials illustrated and described herein, parts and processes may be reversed, and specific features of the present invention may be utilized independently, and those skilled in the art should understand all of the above after obtaining the advantages of this description of the present invention. Changes may be made in the elements described herein without departing from the spirit and scope of the present invention described in the appended claims.

Claims

1. A system configured to determine a drift in a process performed on a sample, comprising: one or more computer subsystems; and one or more components executed by the one or more computer subsystems, wherein the one or more components include a deep generative model; and wherein the one or more computer subsystems are configured to: converting first actual information for an alignment target on a sample from a first type of information to a second type of information by inputting the first actual information into the deep generative model; aligning the converted first actual information with second actual information of the alignment target, wherein the second actual information is information of the second type, wherein the first actual information and the second actual information are information acquired from a source and have not been simulated, and wherein one of the first type of information and the second type of information is design data and the other is a sample image; determining an offset between the converted first actual information and the second actual information based on a result of the alignment; and The determined offset is stored as an alignment-to-design offset for use in a process performed on the sample using an output acquisition subsystem.

2. The system of claim 1, wherein the one or more computer subsystems are further configured to select the alignment target from a setup image of the sample generated with the output acquisition subsystem.

3. A system according to claim 1, wherein the one or more computer subsystems are further configured to generate a training set including training images of the sample generated by the output acquisition subsystem and corresponding design data of the sample and use the training set to train the deep generative model, and wherein the training images are generated at a location independent of the location of the alignment target or any other alignment targets on the sample.

4. The system of claim 1 , wherein the one or more computer subsystems are further configured to generate a training set including a training image of the sample generated by the output acquisition subsystem and corresponding design data of the sample and use the training set to train the deep generative model, and the corresponding design data included in the training set includes design data for less than all layers on the sample that affect the training image. The system of claim 1 , wherein the sample image is a low-resolution sample image.

6. The system of claim 1 , wherein the one or more computer subsystems are further configured to store the determined offset as the alignment-to-design offset for use in the process performed on additional samples using the output acquisition subsystem, wherein the second type of information is design data, and wherein the converted first actual information is less affected by variations in the one or more processes performed on the additional samples than if the second type of information was a sample image.

7. The system of claim 1 , wherein the one or more computer subsystems are further configured to store the determined offset as the alignment-to-design offset for use in the process performed on additional samples using the output acquisition subsystem, wherein the second type of information is design data, and wherein alignment performed in the process using the alignment-to-design offset is less affected by variations in the one or more processes performed on the additional samples than if the second type of information was a sample image.

8. The system of claim 1, wherein the second type of information is design data, and wherein the converting further comprises binarizing an output of the deep generative model such that the converted first actual information comprises binarized design data.

9. A system according to claim 1, wherein the conversion and alignment are performed for additional alignment targets on the sample, wherein the determining the offset comprises respectively determining different offsets for different frames of output generated by the output acquisition subsystem for the sample during the process, and wherein storing the determined offsets comprises storing the different offsets.

10. The system of claim 1, wherein the one or more computer subsystems are further configured to store the second actual information of the alignment target for use in setting an alignment target image during the process performed on the sample using the output acquisition subsystem.

11. The system of claim 10, wherein the one or more computer subsystems are further configured to perform the process on the sample with the output acquisition subsystem, and wherein the process includes aligning a runtime alignment target image generated for the sample in the process with the setup alignment target image, wherein the runtime alignment target image is the second type of information, determining a runtime-to-setup offset based on a result of aligning the runtime alignment target image with the setup alignment target image, and determining a runtime-to-design offset based on the runtime-to-setup offset and the alignment-to-design offset.

12. A system according to claim 11, wherein the one or more computer subsystems are further configured to generate the runtime alignment target image by converting the sample image generated at the location of the alignment target on the sample during the process into runtime alignment target design data, and converting the sample image into the runtime alignment target design data is performed by inputting the sample image generated at the location of the alignment target on the sample during the process into the deep generative model.

13. The system of claim 11, wherein the process further comprises determining, based on the runtime-to-design drift, an area of ​​interest in output produced by the output acquisition subsystem during execution of the process on the sample.

14. The system of claim 1, wherein the deep generative model is a generative adversarial network.

15. The system of claim 1, wherein the deep generative model is a conditional generative adversarial network.

16. The system of claim 1, wherein the process is an inspection process.

17. The system of claim 1, wherein the output acquisition subsystem is a light-based output acquisition subsystem.

18. The system of claim 1, wherein the output acquisition subsystem is an electron beam output acquisition subsystem.

19. The system of claim 1, wherein the first type of information is design data and the second type of information is sample images.

20. The system of claim 1, wherein the first type of information is a sample image and the second type of information is design data.

21. A non-transitory computer readable medium storing program instructions executable on one or more computer systems for performing a computer implemented method for determining a drift in a process performed on a sample, wherein the computer implemented method comprises: converting first actual information of an alignment target on a sample from a first type of information to a second type of information by inputting the first actual information into a deep generative model, wherein one or more components are executed by the one or more computer systems, and wherein the one or more components include the deep generative model; aligning the converted first actual information with second actual information of the alignment target, wherein the second actual information is information of the second type, wherein the first actual information and the second actual information are information acquired from a source and have not been simulated, and wherein one of the first type of information and the second type of information is design data and the other is a sample image; determining an offset between the converted first actual information and the second actual information based on a result of the alignment; and The determined offsets are stored as aligned-to-design offsets for use in a process performed on the samples with an output acquisition subsystem, wherein the inputting, aligning, determining, and storing are performed by the one or more computer systems.

22. A computer-implemented method for determining a drift in a process performed on a sample, comprising: converting first actual information of an alignment target on a sample from a first type of information to a second type of information by inputting the first actual information into a deep generative model, wherein one or more components are executed by one or more computer systems, and wherein the one or more components include the deep generative model; aligning the converted first actual information with second actual information of the alignment target, wherein the second actual information is information of the second type, wherein the first actual information and the second actual information are information acquired from a source and have not been simulated, and wherein one of the first type of information and the second type of information is design data and the other is a sample image; determining an offset between the converted first actual information and the second actual information based on a result of the alignment; and The determined offsets are stored as aligned-to-design offsets for use in a process performed on the samples with an output acquisition subsystem, wherein the inputting, aligning, determining, and storing are performed by the one or more computer systems.

Images