Deep learning model-based alignment for semiconductor applications

Abstract

Methods and systems are provided for deep learning alignment for semiconductor applications. A method includes transforming design information about an alignment target on a sample into a predicted image of the alignment target by inputting the design information into a deep learning model; and aligning the predicted image to an image of the alignment target on the sample generated by an imaging subsystem. The method also includes determining an offset between the predicted image and the image produced by the imaging subsystem based on a result of the alignment; and storing the determined offset as an alignment-to-design offset used in a process performed on the sample using the imaging subsystem.

Description

Technical Field

[0001] The present invention generally relates to deep learning alignment for semiconductor applications. Certain embodiments relate to methods and systems for determining an offset used in a process performed on a sample. Background Art

[0002] The following description and examples are not admitted to be prior art by virtue of their inclusion in this section.

[0003] Integrated circuit (IC) designs may be developed using methods or systems such as electronic design automation (EDA), computer-aided design (CAD), and other IC design software. Such methods and systems may be used to generate a circuit pattern database from an IC design. The circuit pattern database includes data representing a plurality of layouts of various layers of the IC. The data in the circuit pattern database may be used to determine the layouts of a plurality of reticles. The layout of a reticle typically includes a plurality of polygons that define features in a pattern on the reticle. Each reticle is used to fabricate one of the various layers of the IC. The layers of the IC may include, for example, junction patterns in a semiconductor substrate, gate dielectric patterns, gate electrode patterns, contact patterns in an interlayer dielectric, and interconnect patterns on a metallization layer.

[0004] The manufacture of semiconductor devices such as logic and memory devices typically includes processing a substrate (e.g., a semiconductor wafer) using a number of semiconductor manufacturing processes to form various features and multiple levels of the semiconductor device. For example, photolithography is a semiconductor manufacturing process that involves transferring a pattern from a reticle to a resist arranged on a semiconductor wafer. Additional examples of semiconductor manufacturing processes include, but are not limited to, chemical mechanical polishing (CMP), etching, deposition, and ion implantation. Multiple semiconductor devices may be manufactured in an arrangement on a single semiconductor wafer and then separated into individual semiconductor devices.

[0005] Inspection processes are used at various steps during the semiconductor manufacturing process to detect defects on the wafer to promote higher yields and therefore higher profits in the manufacturing process. Inspection has always been an important part of manufacturing semiconductor devices (such as ICs). However, as design rules shrink, semiconductor manufacturing processes can operate closer to the limits of the performance capabilities of the process. In addition, as design rules shrink, smaller defects can have an impact on the electrical parameters of the device, which drives more sensitive inspection. Therefore, as design rules shrink, the population of potential yield-related defects detected by inspection grows dramatically, and the population of disturbing point defects detected by inspection also increases dramatically.

[0006] Inspection systems and methods are increasingly designed to focus on the relationship between defects and design, because the impact on the design of the sample will determine whether and how important the defect is. For example, some methods have been developed for aligning inspection and design coordinates. One such method depends on the accuracy of the inspection system coordinate registration with the design. Another such method involves performing post-processing alignment of the inspection image tiles and associated design clips.

[0007] Some currently used methods are based on performing pixel-to-design alignment (PDA) training on a setup die on a setup wafer, and a physics-based model is used to render the image from the design aligned to a sample image. For example, image rendering based on a partially coherent physical model (PCM) has been used to perform PDA on a sample tool. Although such currently used methods for setting up and performing PDA have proven useful in several applications, such methods and systems have several disadvantages. For example, when the patterned features in the design become relatively dense, PCM-based methods may encounter difficulties in rendering the optical image from the design. In addition, on some sample layers, PCM-based methods may produce relatively low accuracy PDA results.

[0008] Therefore, it would be advantageous to develop systems and methods for determining an offset for use in a process performed on a sample without one or more of the disadvantages described above. Summary of the invention

[0009] The following description of various embodiments should in no way be construed as limiting the subject matter of the appended claims.

[0010] One embodiment relates to a system configured to determine an offset used in a process performed on a sample. The system includes one or more computer subsystems and one or more components executed by the one or more computer subsystems. The one or more components include a deep learning (DL) model. The one or more computer subsystems are configured to transform the design information about an alignment target on a sample into a predicted image of the alignment target by inputting the design information into the DL model. The computer subsystem is also configured to align the predicted image to an image of the alignment target on the sample generated by an imaging subsystem. In addition, the computer subsystem is configured to determine an offset between the predicted image and the image generated by the imaging subsystem based on the result of the alignment. The computer subsystem is further configured to store the determined offset as an alignment to design offset used in a process performed on the sample using the imaging subsystem. The system may be further configured as described herein.

[0011] Another embodiment relates to a computer-implemented method for determining an offset used in a process performed on a sample. The method includes the transforming, aligning, determining, and storing steps described above performed by one or more computer systems. Each step of the method may be performed as further described herein. In addition, the method described above may include any other steps of any other method described herein. Furthermore, the method may be performed by any system described herein.

[0012] Additional embodiments relate to a non-transitory computer-readable medium storing program instructions that can be executed on one or more computer systems to perform a computer-implemented method for determining an offset used in a process performed on a sample. The computer-implemented method includes the steps of the method described above. The computer-readable medium can be further configured as described herein. The steps of the computer-implemented method can be performed as further described herein. In addition, the computer-implemented method for which the program instructions can be executed can include any other steps of any other method described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Further advantages of the present invention will become apparent to those skilled in the art with the benefit of the following detailed description of the preferred embodiments and after reference to the accompanying drawings, in which:

[0014] Figure 1 and 2 is a schematic diagram illustrating a side view of an embodiment of a system configured as described herein;

[0015] Figure 3 and 5 is a flow chart illustrating an embodiment of steps that may be performed by the embodiments described herein;

[0016] Figure 4 is a block diagram illustrating one embodiment of a deep learning (DL) model that may be included in the embodiments described herein; and

[0017] Figure 6 is a block diagram illustrating one embodiment of a non-transitory computer-readable medium storing program instructions for causing a computer system to perform the computer-implemented methods described herein.

[0018] Although the invention is susceptible to various modifications and alternative forms, specific embodiments of the invention are shown by way of example in the drawings and described in detail herein. The drawings may not be to scale. However, it should be understood that the drawings and detailed description thereof are not intended to limit the invention to the specific forms disclosed, but on the contrary, are intended to cover all modifications, equivalents and alternatives falling within the spirit and scope of the invention as defined by the appended claims. DETAILED DESCRIPTION

[0019] The terms "design", "design data", and "design information" as used interchangeably herein generally refer to the physical design (layout) of an IC or other semiconductor device and data derived from the physical design by complex simulations or simple geometric and Boolean operations. The design may include any other design data or design data proxies described in the following commonly owned patents: U.S. Pat. No. 7,570,796, issued to Zafar et al. on August 4, 2009, and U.S. Pat. No. 7,676,077, issued to Kulkarni et al. on March 9, 2010, both of which are incorporated herein by reference as if set forth in their entirety. In addition, the design data may be standard cell library data, integrated layout data, design data for one or more layers, derivatives of the design data, and full or partial chip design data. Furthermore, the "design", "design data", and "design information" described herein refer to information and data generated by a semiconductor device designer during the design process and thus can be used in the embodiments described herein well before the design is printed on any physical samples (such as reticles and wafers).

[0020] Referring now to the drawings, it should be noted that the drawings are not drawn to scale. In particular, the proportions of some elements of the drawings are greatly exaggerated to emphasize the characteristics of the elements. It should also be noted that the drawings are not drawn to the same scale. The same reference symbols have been used to indicate similarly configurable elements shown in more than one figure. Unless otherwise mentioned herein, any element described and shown may include any suitable commercially available element.

[0021] One embodiment relates to a system configured to determine an offset used in a process performed on a sample. The offset determined by the embodiments described herein can be used for applications such as IC design to optical (or other) image alignment through deep learning (DL) models. Figure 1 An embodiment of such a system is shown in . The system includes one or more computer subsystems (such as computer subsystems 36 and 102) and one or more components 100 executed by the one or more computer subsystems. The one or more components include a DL model 104 configured as further described herein.

[0022] In some embodiments, the sample is a wafer. The wafer may include any wafer known in the semiconductor field. Although some embodiments are described herein with respect to a wafer or a plurality of wafers, the embodiments are not limited to using samples thereof. For example, the embodiments described herein may be used for samples such as reticles, flat panels, personal computer (PC) boards, and other semiconductor samples.

[0023] In some embodiments, the system includes an imaging subsystem, the imaging subsystem including at least an energy source and a detector. The energy source is configured to generate energy directed to the sample. The detector is configured to detect energy from the sample and generate an output in response to the detected energy.

[0024] In one embodiment, the imaging subsystem is a light-based imaging subsystem. Figure 1 In one embodiment shown in FIG. 1 , the imaging subsystem 10 includes an illumination subsystem configured to direct light to the sample 14. The illumination subsystem includes at least one light source (e.g., Figure 1 16 shown in FIG. 1 ). The illumination subsystem is configured to direct light to the sample at one or more incident angles, which may include one or more oblique angles and / or one or more normal angles. Figure 1 , light from a light source 16 is directed through an optical element 18 and then through a lens 20 to a beam splitter 21, which directs the light at a normal angle of incidence to a sample 14. The angle of incidence may include any suitable angle of incidence, which may vary depending on, for example, the characteristics of the sample, defects being detected on the sample, measurements being performed on the sample, etc.

[0025] The illumination subsystem may be configured to direct light to the sample at different incident angles at different times. For example, the imaging subsystem may be configured to modify one or more characteristics of one or more elements of the illumination subsystem so that light can be directed to the sample at different incident angles. Figure 1 The incident angle of the incident angle shown in FIG. 1 directs light to the sample. In one such example, the imaging subsystem can be configured to move the light source 16, the optical element 18, and the lens 20 so that the light is directed to the sample at different incident angles.

[0026] The imaging subsystem may be configured to direct light to the sample at more than one incident angle at the same time. For example, the imaging subsystem may include more than one illumination channel, one of which may include, for example, Figure 1 , and the other of the illumination channels (not shown) may include similar elements that may be configured differently or identically or may include at least one light source and possibly one or more other components (such as those described further herein). If this light is directed to the sample simultaneously with other light, one or more characteristics (such as wavelength, polarization, etc.) of the light directed to the sample at different angles of incidence may be different, so that light originating from illuminating the sample at different angles of incidence can be distinguished from one another at the detector.

[0027] The lighting subsystem may include only one light source (e.g. Figure 1) and the light from the light source may be separated into different optical paths by one or more optical elements (not shown) of the illumination subsystem (e.g., based on wavelength, polarization, etc.). The light in each of the different optical paths may then be directed to the sample. Multiple illumination channels may be configured to direct light to the sample at the same time or at different times (e.g., when different illumination channels are used to sequentially illuminate the sample). In another example, the same illumination channel may be configured to direct light with different characteristics to the sample at different times. For example, the optical element 18 may be configured as a spectral filter and the properties of the spectral filter may be changed in a variety of different ways (e.g., by swapping out the spectral filter) so that light of different wavelengths may be directed to the sample at different times. The illumination subsystem may have any other suitable configuration known in the art for directing light with different or the same characteristics to the sample sequentially or simultaneously at different or the same angles of incidence.

[0028] The light source 16 may include a broadband plasma (BBP) light source. In this manner, the light generated by the light source and directed to the sample may include broadband light. However, the light source may include any other suitable light source, such as a laser, which may be any suitable laser known in the art, and may be configured to generate light of any suitable wavelength known in the art. The laser may be configured to generate monochromatic or near-monochromatic light. In this manner, the laser may be a narrow-band laser. The light source may also include a polychromatic light source that generates light of multiple discrete wavelengths or bands.

[0029] Light from the optical element 18 can be focused by the lens 20 onto the beam splitter 21. Figure 1 Lens 20 is shown in FIG. 2 as a single refractive optical element, but in reality, lens 20 may include several refractive and / or reflective optical elements that combine to focus light from the optical elements onto the sample. Figure 1 The illumination subsystem shown in and described herein may include any other suitable optical elements (not shown). Examples of such optical elements include, but are not limited to, polarizing components, spectral filters, spatial filters, reflective optical elements, apodizers, beam splitters, apertures, and the like, which may include any such suitable optical elements known in the art. Additionally, the system may be configured to alter one or more elements of the illumination subsystem based on the type of illumination used for inspection, metrology, and the like.

[0030] The imaging subsystem may also include a scanning subsystem configured to cause light to be scanned across the sample. For example, the imaging subsystem may include a stage 22 on which the sample 14 is placed during inspection, measurement, etc. The scanning subsystem may include any suitable mechanical and / or robotic assembly (including the stage 22) that may be configured to move the sample so that light can be scanned across the sample. Additionally or alternatively, the imaging subsystem may be configured so that one or more optical elements of the imaging subsystem perform a certain scan of light across the sample. The light may be scanned across the sample in any suitable manner.

[0031] The imaging subsystem further includes one or more detection channels. At least one of the one or more detection channels includes a detector configured to detect light from the sample as a result of illumination of the sample by the imaging subsystem and to generate an output in response to the detected light. For example, Figure 1 The imaging subsystem shown in FIG. 1 includes two detection channels, one detection channel formed by light collector 24, element 26 and detector 28 and the other detection channel formed by light collector 30, element 32 and detector 34. Figure 1 , two detection channels are configured to collect and detect light at different collection angles. In some examples, one detection channel is configured to detect specularly reflected light, and the other detection channel is configured to detect light that is not specularly reflected (e.g., scattered, diffracted, etc.) from the sample. However, two or more detection channels may be configured to detect the same type of light (e.g., specularly reflected light) from the sample. Although Figure 1 An embodiment of an imaging subsystem including two detection channels is shown, but the imaging subsystem may include a different number of detection channels (e.g., only one detection channel or two or more detection channels). Figure 1 Each light collector is shown as a single refractive optical element in , but each light collector may include one or more refractive optical elements and / or one or more reflective optical elements.

[0032] One or more detection channels may include any suitable detector known in the art. For example, the detector may include a photomultiplier tube (PMT) or any other suitable non-imaging detector known in the art. If the detector is a non-imaging detector, each detector may be configured to detect certain characteristics of the scattered light (e.g., intensity) but may not be configured to detect such characteristics that vary depending on the position in the imaging plane. Thus, the output generated by each detector included in each detection channel may be a signal or data rather than an image signal or image data. In such examples, a computer subsystem (e.g., computer subsystem 36 of the system) may be configured to generate an image of the sample from the non-imaging output of the detector.

[0033] Provided in this article Figure 1to generally illustrate the configuration of an imaging subsystem that may be included in an embodiment of the system described herein. Obviously, the imaging subsystem arrangement described herein may be modified to optimize the performance of the system as is typically performed when designing a commercial inspection, metrology, or the like system. In addition, the system described herein may be implemented using an existing inspection or metrology system (e.g., by adding the functionality described herein to the existing inspection or metrology system), such as the 29xx and 39xx series of tools, the SpectraShape series of tools, and the Archer series of tools commercially available from KLA, Milpitas, California. For some such systems, the embodiments described herein may be provided as optional functionality of the system (e.g., in addition to other functionality of the system). Alternatively, the imaging subsystem described herein may be designed "from scratch" to provide an entirely new system.

[0034] The computer subsystem 36 of the system may be coupled to the detector of the imaging subsystem in any suitable manner (e.g., via one or more transmission media, which may include "wired" and / or "wireless" transmission media) so that the computer subsystem may receive the output generated by the detector during scanning of the sample. The computer subsystem 36 may be configured to use the output of the detector to perform several functions as described herein and any other functions further described herein. This computer subsystem may be further configured as described herein.

[0035] This computer subsystem (and other computer subsystems described herein) may also be referred to herein as a computer system. Each of the computer subsystems or systems described herein may take various forms, including a personal computer system, an image computer, a mainframe computer system, a workstation, a network appliance, an Internet appliance, or other devices. In general, the term "computer system" may be broadly defined to encompass any device having one or more processors that execute instructions from a memory medium. The computer subsystem or system may also include any suitable processor known in the art (e.g., a parallel processor). In addition, the computer subsystem or the system may include a computer platform (as a stand-alone tool or a network-linked tool) with high-speed processing and software.

[0036] If the system includes more than one computer subsystem, the different computer subsystems may be coupled to each other so that images, data, information, instructions, etc. may be sent between the computer subsystems, as further described herein. For example, computer subsystem 36 may be coupled to computer subsystem 102 (e.g., by any suitable transmission medium, which may include any suitable wired and / or wireless transmission medium known in the art) via any suitable transmission medium. Figure 1 Two or more such computer subsystems may also be effectively coupled via a shared computer-readable storage medium (not shown).

[0037] In another embodiment, the imaging subsystem is an electron beam imaging subsystem. In this manner, the energy source may be an electron beam source. Figure 2 In one such embodiment shown in , the imaging subsystem includes an electron column 122 coupled to a computer subsystem 124 .

[0038] Also like Figure 2 , the electron column includes an electron beam source 126 configured to produce electrons that are focused by one or more elements 130 to a sample 128. The electron beam source may include, for example, a cathode source or emitter tip, and the one or more elements 130 may include, for example, a gun lens, an anode, a beam limiting aperture, a gate valve, a beam current selection aperture, an objective lens, and a scanning subsystem, all of which may include any such suitable elements known in the art.

[0039] Electrons returning from the sample (eg, secondary electrons) may be focused by one or more elements 132 onto a detector 134. One or more elements 132 may include, for example, a scanning subsystem, which may be the same scanning subsystem included in element 130.

[0040] The electron column may include any other suitable elements known in the art. In addition, the electron column may be further configured as described in the following patents: U.S. Pat. No. 8,664,594 issued to Jiang et al. on April 4, 2014, U.S. Pat. No. 8,692,204 issued to Kojima et al. on April 8, 2014, U.S. Pat. No. 8,698,093 issued to Gubbens et al. on April 15, 2014, and U.S. Pat. No. 8,716,662 issued to MacDonald et al. on May 6, 2014, which are incorporated herein by reference as if fully set forth.

[0041] Despite Figure 2 , the electron beam is shown as being configured so that electrons are directed to the sample at an oblique angle of incidence and scattered from the sample at another oblique angle, but it should be understood that the electron beam can be directed to the sample and scattered from the sample at any suitable angle. In addition, the electron beam subsystem can be configured to use multiple modes to generate an image of the sample (e.g., with different illumination angles, collection angles, etc.). The multiple modes of the electron beam subsystem can be different in any image generation parameter of the subsystem.

[0042] The computer subsystem 124 may be coupled to a detector 134, as described above. The detector may detect electrons returning from the surface of the sample, thereby generating an output used by the computer subsystem to form an electron beam image of the sample (which may include any suitable electron beam image). The computer subsystem 124 may be configured to use the output of the detector and / or the electron beam image to perform any of the functions described herein. The computer subsystem 124 may be configured to perform any additional steps described herein. The computer subsystem 124 may be further configured as described herein to include Figure 2 The imaging subsystem of the system is shown in FIG.

[0043] Provided in this article Figure 2 to generally illustrate the configuration of an electron beam-based imaging subsystem that may be included in the embodiments described herein. As with the optical subsystem described above, the electron beam subsystem arrangement described herein may be modified to optimize the performance of the subsystem as is typically performed when designing a commercial inspection or metrology system. In addition, the systems described herein may be implemented using existing inspection, metrology, or other systems such as tools commercially available from KLA (e.g., by adding the functionality described herein to the existing system). For some such systems, the embodiments described herein may be provided as optional functionality of the system (e.g., in addition to other functionality of the system). Alternatively, the systems described herein may be designed "from scratch" to provide an entirely new system.

[0044] The imaging subsystem may alternatively be an ion beam based subsystem. Figure 2 , except that any suitable ion beam source known in the art may be used in place of the electron beam source. Thus, in one embodiment, the energy directed to the sample includes ions. Additionally, the imaging subsystem may be any other suitable ion beam-based imaging subsystem, such as those included in commercially available focused ion beam (FIB) systems, helium ion microscope (HIM) systems, and secondary ion mass spectrometer (SIMS) systems.

[0045] The imaging subsystem described herein may be configured to generate an image of a sample using multiple modes. In general, a "mode" is defined by the value of a parameter of the imaging subsystem used to generate an image of the sample. Thus, (in addition to the location on the sample where the output is generated) the mode may differ in the value of at least one of the parameters of the imaging subsystem. For example, in the optical subsystem, different modes may use different wavelengths of light for illumination. The modes may be different for different modes in terms of illumination wavelength, as further described herein (e.g., by using different light sources, different spectral filters, etc.). In another embodiment, different modes may use different illumination channels of the optical subsystem. For example, as mentioned above, the optical subsystem may include more than one illumination channel. Thus, different illumination channels may be used for different modes. The modes may differ in any one or more modifiable parameters of the imaging subsystem (e.g., illumination polarization, angle, wavelength, etc., detection polarization, angle, wavelength, etc.).

[0046] In a similar manner, the output generated by the electron beam subsystem may include an image generated by the electron beam subsystem using two or more different values ​​of a parameter of the electron beam subsystem. Multiple modes of the electron beam subsystem may be defined by the values ​​of the parameters of the electron beam subsystem used to generate an image of the sample. Thus, different modes may differ in the value of at least one of the electron beam parameters of the electron beam subsystem. For example, in one embodiment of the electron beam subsystem, different modes may be illuminated using different angles of incidence.

[0047] The imaging subsystem embodiments described herein may be configured for inspection, metrology, defect detection, or another quality control related process performed on a sample. For example, the imaging subsystem embodiments described herein and described in the examples may be modified in terms of one or more parameters. Figure 1 and 2 The embodiments of the imaging subsystem shown in FIG. 1 provide different imaging capabilities depending on the application in which it is to be used. In one such example, Figure 1 The imaging subsystem shown in FIG. 1 can be configured to have higher resolution in the case where it is to be used for defect inspection or metrology rather than for inspection. In other words, Figure 1 and 2 The embodiments of the imaging subsystem shown in describe some general and various configurations of the imaging subsystem, which can be customized in a number of ways that will be apparent to those skilled in the art to produce imaging subsystems with different imaging capabilities that are more or less suitable for different applications.

[0048] The imaging subsystem described above is configured to scan energy (e.g., light, electrons, etc.) across a physical version of a sample, thereby generating an image for the physical version of the sample. In this way, the imaging subsystem may be configured as a "real" subsystem rather than a "virtual" subsystem. However, Figure 1The storage medium (not shown) and computer subsystem 102 shown in the embodiment of the present invention may be configured as a "virtual" system. In particular, the storage medium and computer subsystem may be configured as a "virtual" inspection system as described in the commonly assigned U.S. Patent No. 8,126,255 issued on February 28, 2012 to Bhaskar et al. and U.S. Patent No. 9,222,895 issued on December 29, 2015 to Duffy et al., both of which are incorporated herein by reference as if fully set forth. The embodiments described herein may be further configured as described in these patents.

[0049] As mentioned above, one or more components executed by one or more computer subsystems include a DL model, e.g. Figure 1 104 is shown in FIG. 104. In general, "deep learning" (also called deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning (ML) based on a set of algorithms that attempt to model high-level abstractions of data. In DL-based models, there are usually many layers between the input and output, allowing the algorithm to use multiple processing layers consisting of (for example) multiple linear and nonlinear transformations. DL methods are based on learning representations of data. Observations (such as images) can be represented in many ways (such as a vector of intensity values ​​per pixel) or in more abstract ways (such as a set of edges, regions of a specific shape, etc.).

[0050] DL models can also be generally defined as models that are probabilistic in nature. In other words, DL models are not models that perform forward simulation or rule-based methods, and thus, a physics model of the processes involved in generating actual images is not necessary. Instead, as further described herein, a DL model (wherein its parameters can be learned) can be learned based on a suitable training data set.

[0051] Many semiconductor quality control type applications currently used in the art provide improved results by using design data to perform certain functions such as defect detection and classification, etc. In order for the design data to be useful for any of these functions, it must be properly aligned to the results generated by the quality control tool for the sample. For example, unless the sample image is properly aligned to the design data, it is impossible to accurately determine which parts of the image correspond to which parts of the design. One difficulty in performing this alignment is that the sample image often does not look much like its corresponding design, which can be due to several factors, including how the structures in the design are formed on the sample and the ability of the tool to image the structures. In other words, the sample image can have image characteristics that are substantially different from the corresponding design data, which can make it particularly difficult to align the corresponding sample image with the design data. In most cases, it is not possible to change the characteristics of the sample image to make image-to-design alignment easier. Therefore, creative and better ways to make such alignment possible continue to be developed. The embodiments described herein advantageously provide better alignment for samples with design-to-image mismatches, such as the design-to-image mismatch described above.

[0052] The one or more computer subsystems are configured to transform the design information of the alignment target on the sample into a predicted image of the alignment target by inputting the design information into the DL model, such as in Figure 3 The one or more computer subsystems may input the design information into the DL model in any suitable manner. The DL model then transforms the input design information into a predicted image of the alignment target that approximates an image of the alignment target formed on the sample that will be produced by an imaging subsystem (such as the imaging subsystem described herein). Thus, the DL model transforms the design information from the design space to the sample image space. In this way, the input and output of the DL model described herein are in different domains, i.e., design data space versus sample image space.

[0053] Thus, the input design information may differ significantly from the output predicted image. In particular, the input and output of the transformation process may have significantly different image characteristics. However, through proper training of the DL model, which may be performed as further described herein, the predicted image of the alignment target produced by the DL model will have substantially the same image characteristics as the actual image of the alignment target formed on the sample produced by the imaging subsystem. In other words, when the DL model is properly trained, the predicted image will be indistinguishable (or nearly indistinguishable) from the corresponding real image of the sample produced by the imaging subsystem.

[0054] For ease of understanding, images of a sample produced by an imaging subsystem may be referred to herein simply as "real images" or "actual images," meaning that they are produced by a system that includes imaging hardware rather than merely by modeling as predicted images.

[0055] In one embodiment, the DL model is configured as a variational autoencoder (VAE). In this way, the embodiments described herein provide a DL VAE predicted optical image from a design and its use in pixel-to-design alignment (PDA). The embodiments described herein also provide a complete method / process for performing PDA on a tool (such as the tool further described herein) using a DL-based method. Although certain embodiments are described herein with respect to a VAE (which may be used interchangeably with the term "VAE model"), the embodiments are not limited to VAE models. DL models other than VAEs may be used in the embodiments described herein. In general, a VAE may have a configuration such as the configuration further described herein or any suitable configuration known in the art. Additional information about VAEs may be found in "Tutorial on Variational Autoencoders", Doersch, arXiv:1606.05908, August 13, 2016, page 23, which is incorporated herein by reference as if fully set forth. The embodiments described herein may be further configured as described in this reference.

[0056] In one embodiment, one or more computer subsystems are configured to convert a binary design image of an alignment target into a grayscale design image, such as Figure 3 The computer subsystem may rasterize the binary design image to convert the design polygons into a grayscale image. The rasterized grayscale image may then be input into a DL model as further described herein to generate a predicted image. The original binary design data image need not then be input into the DL model, but in some cases, it may be useful to input the original binary design data image into the DL model with the rasterized grayscale image as another channel of the input.

[0057] Using a rasterized grayscale image instead of a binary design image as a DL model input is an important new feature of the embodiments described herein. In the case of an optical imaging subsystem, the real optical image has a finite pixel size (e.g., 30nm). In contrast, the design polygon has a sub-pixel resolution. A binary design image with the same pixel size as the optical image has a + / -0.5 pixel uncertainty. The rasterized image is a more accurate representation of the design polygon. For example, the rasterized grayscale image may have the same pixel size as the real sample image. Internally, the computer subsystem may perform database (DB) rasterization using a 1 / 32 pixel size to represent the polygon, and then apply appropriate low-pass filtering to produce a rasterized image. If the pixel is "inside the polygon", the pixel may be assigned a grayscale value of 255. In contrast, if the pixel is "outside the polygon", the pixel may be assigned a grayscale value of 0. If the pixel is relatively close to the edge of the polygon, for example, within 1 pixel or half a pixel, the intensity may be a linear function of the distance to the edge of the polygon (e.g., edge speed). Of course, any suitable method for producing a rasterized design image from a binary design image may be used to produce input to the DL models described herein.

[0058] Figure 4 An embodiment of a VAE architecture that can be used in the embodiments described herein is shown. The VAE architecture may include one encoder layer and more than one decoder layer. A computer subsystem may generate a rasterized design input 400 that is input to an encoder 402. The encoder may have any suitable configuration known in the art and may include any suitable number (e.g., 4) of layers and any suitable type of layers known in the art (e.g., convolution, pooling, etc.). The output of the encoder may be input to both a design decoder 404 and an image decoder 410. The encoder extracts features with reduced size from the rasterized design input, which preserves maximum information when encoding. For example, the original input image size may be 192x 192. The size of the encoder output may be 12x 12.

[0059] The decoder may have any suitable configuration known in the art and may include any suitable number (e.g., 4) of layers and any suitable type of layers known in the art (e.g., convolutional, pooling, etc.). Image decoder 410 produces predicted images 412 from the output of the encoder. In this way, the VAE predicts images from the design.

[0060] In some embodiments, the predicted image 412 and the aligned real image 414 are input to the error minimization step 416. Both the rasterized design input and the aligned real image may be generated prior to the training of the VAE model. The aligned real image may be a sample image generated by the imaging subsystem that has been aligned to the design. For example, the initial aligned (optical or other) image may be generated using the offset measured by the Normalized Cross Correlation (NCC) and / or the currently used PDA offset based on the Partial Coherence Model (PCM). In particular, generating the aligned real image to the design may be performed by shifting the real sample image using the offset between the rasterized design and the sample image measured by the NCC and / or by shifting the real sample image using the offset between the image generated from the design via the PCM measured by the NCC and the real sample image.

[0061] The error minimization step may minimize the mean squared error (or another error function) between the predicted image and the aligned true image. In particular, step 416 may minimize the mean squared error for each difference pixel between the true and predicted images. This minimization step may be used only during training of the VAE model, where one or more parameters of the model are modified until there is no error (or an error below some predetermined threshold) between the predicted image and the aligned true image. However, this step may also be used after training to monitor the performance of the VAE model (e.g., to determine whether an update, modification, or retraining of the VAE model has become necessary). For example, if the difference between the aligned true image and the predicted image is detected by the error minimization step and is greater than some predetermined criteria, the VAE model may be updated, modified, or retrained, which may be performed as further described herein.

[0062] In another embodiment, the DL model is also configured to transform the grayscale design image into a predicted grayscale design image. For example, the VAE may generate a predicted sample image and a predicted rasterized design from a rasterized design input (which may include multiple layers of a design as further described herein). Figure 4 4, the VAE may include a design decoder 404 that may generate a predicted rasterized design 406 from the output of the encoder 402. The predicted rasterized design 406 and the rasterized design input 400 may be input to an error minimization step 408. This minimization step may include a cross-entropy error (or another suitable type of error function) between the predicted rasterized design and the rasterized design input. In other words, step 408 may include minimizing the cross-entropy reconstruction error between the input and the generated design, which may be performed in any suitable manner known in the art.

[0063] The input and output rasterization designs are not necessarily the same. In particular, the rasterization design input is generated directly from the design polygons without using any type of DL modeling, such as the DL modeling further described herein. The predicted rasterization design is the output from the VAE DL model. By minimizing the cross entropy (or other) error between the input rasterization design and the predicted rasterization design, the encoder layer can be adjusted, which will help produce better and / or more stable predicted images. In this way, a predicted rasterization design can be generated and output to control the behavior of the encoder. Unlike the error minimization 416, which can be used only during training but can optionally also be used during runtime, the error minimization step 408 can be performed during both training and runtime.

[0064] In some embodiments, the design information includes information of multiple layers of the design of the sample. In this way, the embodiments described herein may be configured for VAE-based image generation using multiple layers of the design. For example, the VAE model described herein supports multiple layers of the design. The DL model may be configured to handle multiple layers of the design in the same way that it can handle multiple other information streams (such as multiple color channels). For example, the input to the VAE model may be changed from something like (width, height, 1) to (width, height, nLayers). The term multiple layers of the design as used herein generally refers to layers formed on top of each other or one after another on the sample. Therefore, different layers of the design may include information of different polygons formed on the sample in different steps of the manufacturing process. The multiple layers used in the prediction described herein may not include all layers of the design formed to the point where the sample is imaged. For example, the layers of the design input to the DL model may only include the layers that are expected to have at least some effect on the image of the sample generated by the imaging subsystem. Generating or predicting an image from multiple layers of the design may result in a predicted image that is better similar to the real sample image.

[0065] In one such embodiment, the DL model is configured to have different weights for at least two of the multiple layers that are determined separately during training of the DL model. For example, some design layers will most likely contribute more to generating / predicting a sample image than other design layers. The first and last VAE convolutional layers may have pattern filter coefficients for training. These coefficients may control the relative weight of each design layer. The weights applied to each layer may be learned during VAE model training, which may be performed as further described herein.

[0066] The DL models described herein may or may not be trained by one of the one or more computer subsystems and / or components executed by a computer subsystem. For example, another method or system may train a DL model, which may then be stored for use as a component executed by a computer subsystem. The DL model may also be trained or established at any time prior to run time, such as during PDA training or setup. The DL models described herein may also be updated, retrained, or revised at any time after setup and / or deployment of the trained model. Once the VAE model has been trained, it may be used by the embodiments described herein to generate predicted sample images from rasterized design images at all selected locations.

[0067] The computer subsystem may select alignment targets for use in training and / or run time from a set image of a sample generated using the imaging subsystem. The alignment targets selected for training and run time may or may not be the same. For example, the same alignment target may be used, but training and run time may be performed using fewer examples of alignment targets for training to run time. The opposite is still feasible. The selected alignment target may also include different alignment targets with different characteristics rather than just different positions on the sample or in the design. The alignment target may be selected in any other suitable manner known in the art. Alternatively, another system or method may select the alignment target, and the computer subsystem may obtain information of the selected alignment target in any suitable manner. The embodiments described herein may also be used with any suitable alignment target selected in any suitable manner known in the art as known in the art.

[0068] The one or more computer subsystems may receive design data for each of the alignment targets in any suitable manner, such as by searching for a design of the sample based on the information of the alignment targets, by requesting a portion of the design (e.g., a design clip) at the location of the alignment targets from a storage medium or computer system containing the design data, etc. The design data received by the one or more computer subsystems may include any of the designs, design data, or design information described further herein.

[0069] The VAE models described herein may be trained using one or more relatively small training datasets, which may include the VAE models described above with respect to Figure 4 The training set can be used by the computer subsystem in any suitable manner known in the art (including being divided into subsets for training and validation, updated, modified or replaced over time or when the process changes, etc.).

[0070] The DL model may then be trained based on a training set comprising a rasterized design image and a corresponding aligned real sample image. Both the sample image and the rasterized design image are training inputs. Training may include inputting the training input into the DL model and changing one or more parameters of the DL model until the output produced by the DL model matches (or substantially matches) the training input. Training may include changing any one or more trainable parameters of the DL model. For example, one or more parameters of the DL model trained by the embodiments described herein may include one or more weights of any layer of the DL model having trainable weights. In one such example, the weights may include weights of a convolutional layer rather than a pooling layer.

[0071] Although some embodiments are described herein with respect to DL models, the embodiments described herein may include or use multiple DL models. In one such example, different DL models may be trained and used for different modes of the imaging subsystem as described herein. For example, an alignment performed for one mode may not necessarily be transferable to another mode. In other words, by performing alignment for one mode, not all modes may be aligned to the design (or at least not aligned to the design with sufficient accuracy). In most cases, different modes of the imaging tool will produce images that differ from each other in one of several possible ways, such as noise level, contrast, resolution, image type (e.g., DF vs. BF, optical vs. electron beam, etc.), etc. Therefore, if a DL model is trained for one mode of the tool, it may be improperly trained to transform information for another mode of the tool. Thus, multiple DL models (one for each mode of interest) may be trained separately and independently, and image-to-design alignment may be performed separately and independently for different modes used in a single process performed on a sample. In this way, when different modes are used to generate images of a sample, images generated in more than one mode may be aligned to the design of the sample. The same pre-trained DL model can be used for each mode, but this is not necessary. Each trained DL model can then be used to perform mode-specific transformations. Since the design of the sample will not change from mode to mode, the design information input to each different trained DL model can be the same design information. Although different rasterized images can be input to different DL models when appropriate.

[0072] The DL model is trained to produce predicted images that are indistinguishable from real sample images. Experiments performed by the inventors have shown that images produced by VAE are better for alignment to the design than images predicted by existing methods. For example, experiments performed by the inventors showed that images produced by VAE are closer to real sample images than images produced by PCM. Specifically, the inventors generated X / Y projections of PCM and VAE-generated images, which showed a closer similarity between the VAE-generated images and the real sample images. In this way, the DL-based PDA described herein can generate / predict images that have a closer similarity to real sample images and can be used to implement a more accurate PDA. In addition, shrinking design rules can present difficulties in predicting images by PCM, and other currently used methods can encounter difficulties that are not a problem for the DL model described herein.

[0073] Once the DL model is trained, it can be used to transform the design information of the alignment target into a predicted image. For example, one or more computer subsystems may obtain the design data or rasterize the design image for each target, which can be performed as further described herein. One or more computer subsystems can then generate a predicted tile image from the design at each target through the trained DL model. In this way, the DL model described herein can be used to generate a predicted sample image for any given design clip, which will then be used to align the sample image to calculate the offset, as further described herein. In other words, after training, the DL model can be used to generate a predicted sample image for the PDA so that the actual sample image can be aligned to the predicted sample image and thereby its corresponding design clip.

[0074] The one or more computer subsystems are also configured to align the predicted image to an image of an alignment target on the sample generated by the imaging subsystem, such as Figure 3 304 of . For example, one or more computer subsystems may align the predicted image with the sample image at one or more (or each) target. Aligning the predicted image to the real image may be performed based on a normalized sum of squared differences (NSSD). For example, the NSSD of the predicted image and the sample image may be used to calculate the alignment offset. The NSSD may be performed using any suitable method, algorithm, function, etc. known in the art. However, the alignment step may be performed using any other suitable alignment method or algorithm known in the art.

[0075] The one or more computer subsystems are further configured to determine an offset between the predicted image and the image generated by the imaging subsystem based on the results of the alignment, such as Figure 3306 of . In this way, embodiments described herein may use the predicted images produced by the VAE to measure the PDA shift. This shift may be determined in any suitable manner and may be expressed in any suitable manner (e.g., as a Cartesian shift, as a two-dimensional function, etc.). For example, the computer subsystem may identify the PDA shift between the VAE predicted image and the true sample image for all selected locations by using the NCC for measuring the individual shifts.

[0076] The transformation and alignment steps may be performed for additional alignment targets on the sample, and determining the offset may include determining different offsets for different frames, jobs, sub-swaths, swaths, etc. of images generated by the imaging subsystem for the sample during the process. For example, one or more computer subsystems may determine the design-to-image offset for each inspection frame from the target. Compared to currently used methods and systems for PDAs, the VAE model described herein may advantageously generate more predicted images for selected wafer die locations, which may then be used to align the sample image to the design. In this way, the VAE model may be used to generate predicted images from all selected locations of the die using a rasterized design.

[0077] Being able to use more selected positions for alignment means a better PDA that can follow image jitter or other alignment error sources across the entire die or wafer. The inventors performed experiments to compare the NCC peak and alignment offset of a PCM-based PDA and a VAE-based PDA. Specifically, the alignment offset X value, alignment offset Y value, and NCC peak are determined and plotted as a function of the test index on the X-axis. The test index on the X-axis corresponds to a real image / design selected from a neighboring proximity position of the die position along X. The distance between the neighboring index values ​​along X is close enough to capture image jitter changes. The NCC between the real sample image and the image predicted by VAE is used to measure the VAE alignment offset X and Y values. The NCC between the real sample image and the image generated by PCM is used to measure the PCM alignment offset on X and Y. At almost every data point examined by the inventors, the VAE-based PDA exhibits a higher NCC peak than the PCM-based PDA, corresponding to a more accurate offset measurement. In other words, the experiments performed by the inventors show that the VAE-based PDA can produce a more accurate alignment offset than the currently used PDA method. Additionally, the VAE-based PDA results show significantly smoother shifts in both X and Y across all tested index values, indicating a more accurate alignment because image jitter is gradually changed and can be measured at nearby locations.

[0078] Additional experiments performed by the inventors included a comparison of the offset measured by VAE from one sub-scan on the inspection tool to the wafer with the currently used PDA offset. The pixel offset between optical to design on both X and Y was plotted as a function of the relative position of the die on X. The optical to design alignment offset is primarily due to image jitter which should vary smoothly across the entire die for the entire sub-scan. The plotted results show that the VAE offset between the VAE offset and the dispersed currently used PDA offset is smoother on Y, indicating that the VAE offset is more accurate than the currently used PDA offset.

[0079] The inventors also generated a similar graph comparing the offset measured by VAE from one scan band on the inspection tool to another wafer with the currently used PDA offset. In the same manner described above, the optical to design alignment offset is primarily due to image jitter which should vary smoothly across the entire die across the scan band. The plotted results show that the VAE offset is smoother than the currently used PDA offset, indicating that the VAE offset is more accurate than the currently used PDA offset. Similar results are shown in a similar graph generated from the VAE offset across the scan band from the die.

[0080] The computer subsystem may be configured to determine the PDA offset of the VAE predicted image to the real sample image for all selected positions and cluster the PDA offset to produce a robust offset. In this way, the PDA offset determined and used by the embodiments described herein may be a clustered or individual offset. Adding clustering to the VAE-based PDA may produce even better results than using individual offsets. In one such embodiment, the alignment target is one of a plurality of alignment targets positioned in the scan band of the image generated by the imaging subsystem, and one or more computer subsystems are configured to perform transformation, alignment, determination, and storage steps for a plurality of alignment targets, cluster the offsets determined for a plurality of alignment targets to produce a clustered offset, and use the clustered offset to replace one or more of the offsets determined for a plurality of alignment targets. In this way, clustering may be used to process the offsets of each scan band on the scan band level. In other words, clustering may be performed separately for each scan band of the image generated for the sample. However, clustering may be performed on a different frequency basis, such as for less than all scan bands, for a plurality of scan bands, for the entire sample, and the like.

[0081] After clustering, incorrect individual offsets may be replaced with the clustered offsets. For example, if an offset is determined for one of the alignment targets that clearly falls outside of the clustered offsets, the offset may be replaced with the clustered offsets. In this way, the robustness of the offsets may be improved through clustering. The robust offsets after clustering may then be used as described herein, for example, to substantially accurately place a design on top of an optical image during runtime of verification.

[0082] The embodiments described herein may also be configured for bandpass filtering the predicted and real images prior to alignment, offset determination, etc. In this manner, the PDA offset may not be measured directly from the rendered and real images, but rather after bandpass filtering of these images, which may be performed in any suitable manner known in the art. Both rendered and real images may be processed using bandpass filtering to improve alignment offset measurement accuracy. Any other suitable image filtering may also or alternatively be performed on the predicted and real images prior to alignment and offset determination to improve alignment offset measurement accuracy. Whether the offset is measured directly from the predicted and real images or after the predicted and real images have been bandpass (or other) filtered, the offset may be measured using NCC or another suitable method.

[0083] The one or more computer subsystems are also configured to store the determined offset as an alignment-to-design offset for use in a process performed on a sample using the imaging subsystem, such as in Figure 3 The computer subsystem may store the aligned-to-design offset in any suitable computer-readable storage medium. The aligned-to-design offset may be stored with any results described herein and may be stored in any manner known in the art. The storage medium may include any storage medium described herein or any other suitable storage medium known in the art. After the aligned-to-design offset has been stored, the aligned-to-design offset may be accessed in the storage medium and used by any method or system embodiment described herein, formatted for display to a user, used by another software module, method or system, etc. The determined offset may also be stored indefinitely, such as with the results of the process using it, or temporarily, such as only when it is needed in the process of using it to execute the PDA.

[0084] When the transform and alignment steps are performed for additional alignment targets on the sample, determining the offsets may include determining different offsets for different frames of images generated by the imaging subsystem for the sample during the process, etc., and storing the determined offsets includes storing the different offsets. In another embodiment, the one or more computer subsystems are configured to store predicted images of the alignment targets, such as Figure 3 310 of . For example, one or more computer subsystems may save the predicted images and corresponding offsets to a database for run-time verification (or another process described herein). These steps may be performed as further described herein.

[0085] Embodiments described herein may set up alignment of samples for a process recipe, which may be performed as part of setting up, generating, calibrating, or updating a recipe. The recipe may then be stored and used by embodiments described herein (and / or another system or method) to perform a process on a sample to thereby generate information (e.g., defect information) for the sample. In this manner, alignment to design offsets may be generated and stored once per sample layer and / or on a per sample basis. In this manner, predicted images and determined offsets generated by embodiments described herein may be used for only a single sample.

[0086] Thus, as described herein, embodiments may be used to set up new processes or recipes. Embodiments may also be used to modify existing processes or recipes, whether or not they are processes or recipes for a sample or generated for one sample and adapted for another sample. In addition, the embodiments described herein are not limited to inspection process generation or modification. For example, the embodiments described herein may also be used to set up or modify processes for metrology, defect inspection, etc. in a similar manner. In particular, determining the offsets used in the process and performing sample-to-design alignment as described herein may be performed without regard to the set up or modified process. Thus, the embodiments described herein may be used not only to set up or modify an inspection process, but may also be used to set up or modify any quality control type process performed on the samples described herein.

[0087] The embodiments described herein may also perform a process after aligning the sample at run time as described herein. In one embodiment, the process is an inspection process. However, the process may include any of the processes described herein, such as inspection, defect inspection, metrology, etc. The computer subsystem may be configured to store information of the sample generated by executing the process (e.g., information of detected defects in any suitable computer-readable storage medium). The information may be stored with any results described herein and may be stored in any manner known in the art. The storage medium may include any storage medium described herein or any other suitable storage medium known in the art. After the information has been stored, the information may be accessed in the storage medium and used by any method or system embodiment described herein, formatted for display to a user, used by another software module, method or system, etc.

[0088] In one such embodiment, one or more computer subsystems are configured to perform a process on a sample using an imaging subsystem, and during the process, input, alignment, determination, and storage are performed, and the offset is a runtime to design offset. For example, during the process, one or more computer subsystems may generate a predicted image for each target and align the predicted image to the runtime image at each target. One or more computer subsystems may determine the offset between predicted and runtime for each inspection frame, each scan band, each subscan band, etc. All of these steps may be performed as further described herein.

[0089] Unlike some currently used methods for aligning setup and runtime, the embodiments described herein do not require additional steps like each of: generating a setup alignment target image; determining a runtime-to-setup offset; modifying the alignment-to-design offset using the runtime-to-setup offset to obtain a runtime-to-design offset, etc. In other words, since a predicted image can be generated from design information of an alignment target by the embodiments described herein during runtime, the embodiments described herein can generate a predicted image only during runtime and then align the predicted image to a runtime image to obtain a runtime-to-design offset.

[0090] Thus, because image prediction can be done so quickly by the VAE model described herein, embodiments can reduce the complexity of the alignment process by eliminating several steps currently performed for image alignment, which can also substantially increase the accuracy of the offsets determined by the embodiments described herein. In this way, once a VAE model has been trained as described herein, the trained VAE model can be used during runtime to predict images for alignment until any retraining of the VAE model is needed or desired. Thus, while predicted images may be saved by one or more computer subsystems for any number of reasons, unlike currently used methods in which predicted images are saved during setup and then reused during runtime, the embodiments described herein do not require such image storage and reuse. For example, in one embodiment, the one or more computer subsystems are configured for training a DL model and setup process, and the training and setup do not include storing setup predicted images of alignment targets used in processes performed on samples using the imaging subsystem.

[0091] Next, to reiterate some important differences between VAE-based PDA and existing PDA, the existing PDA process in training may include generating PCM rendering / prediction images and measuring the offsets A between the optical images from training and the PCM rendering images. These offsets A are between the optical images from training and the design. During training, the existing PDA saves a PDA database that includes the offsets A and optical images from each alignment target / alignment target position. At runtime, the existing PDA identifies the offsets B between the optical images in runtime and the optical images in training. The offsets A and B are then added together to produce the final offset from runtime optical to design.

[0092] In contrast, a VAE PDA process performed by an embodiment described herein may include, during training, generating an initial VAE model and saving alignment target position information. During runtime, an embodiment may run the VAE model to predict a VAE-generated sample image for each target position. An embodiment measures the offsets between the runtime sample images and the predicted images generated by the VAE. These offsets are runtime sample image to design offsets. For example, if the process variation is relatively severe between the initial training samples or samples used for previous model training / update and the current samples, the VAE model may be updated in one or more ways during runtime, as further described herein.

[0093] In another embodiment of this, the process includes accurately placing the design area of ​​interest on top of the inspection image by shifting the design area of ​​interest or the inspection image using a run-time to design offset. For example, one or more computer subsystems may place the area of ​​interest based on the offset correction. As commonly referred to in the art, a "area of ​​interest" is an area on a sample that is of interest for the purpose of inspection from design. Sometimes, the area of ​​interest is used to distinguish between inspected areas on a sample and areas on the sample that were not inspected during the inspection process. In addition, the area of ​​interest is sometimes used to distinguish areas on a sample that are to be inspected using one or more different parameters. For example, if a first area of ​​a sample is more critical than a second area on the sample, then the first area can be inspected using a higher sensitivity than the second area so that defects are detected in the first area using a higher sensitivity. Other parameters of the inspection process can be changed in a similar manner, area by area of ​​interest.

[0094] In these embodiments, one or more computer subsystems may use zero care region boundaries in the x and y directions. For example, because the embodiments described herein can align the sample image to the design with substantially high accuracy, the care region can be positioned in the sample image with substantially high accuracy. Therefore, the boundaries that are typically used to artificially increase the care region to account for any errors in the placement of the care region can be effectively eliminated by the embodiments described herein. Placing the care region with such high accuracy and eliminating the care region boundaries is advantageous for several reasons, including that the detection of disturbing points on the sample can be significantly reduced and the detection of defects of interest (DOIs) on the sample can be improved.

[0095] In one embodiment, one or more computer subsystems are configured to perform input, alignment, determination, and storage during the runtime of a process performed on a sample and during the runtime of a process performed on one or more other samples. For example, the DL-based PDA described herein can be effectively implemented on graphics processing unit (GPU) hardware and can be faster than a PCM-based PDA. In this way, the steps described herein can be performed in runtime for each scan of each sample. In particular, since the DL-based method described herein can be easily implemented on a GPU with faster processing power, the embodiments described herein can be used in a runtime PDA on a tool such as the tool described herein. As further described herein, being able to perform the steps described herein in runtime means that the embodiments have fewer steps and are simpler than the currently used PDA method, which can also cause the embodiments described herein to have better alignment accuracy.

[0096] In some embodiments, one or more computer subsystems are configured to retrain the DL model by performing iterative training of the DL model for different samples, such as in Figure 3 312 of . For example, iterative VAE model training may be performed when there are process variations on new samples. Process variations between samples may produce differences in the images of the samples produced by the imaging subsystem, such as color variations, blurred images due to out-of-focus conditions, etc. These process variations may or may not be defects on the sample and may cause problems with the PDA set up with one sample and used for one or more other samples. In this way, the VAE model may be retrained using iterative VAE model training.

[0097] Iterative VAE model training may be performed for each additional sample that is scanned using the PDA setup and aligned with another sample. Alternatively, iterative VAE model training may be performed only after certain time intervals, which may be based on time, number of samples, etc. Iterative VAE model training may also be performed when it is suspected or known that some process variation has occurred between samples. The computer subsystem may also determine whether iterative VAE model training should be performed based on the actual sample image generated by the imaging subsystem, for example by comparing it with the VAE predicted image to determine whether it is different enough to warrant VAE retraining, updating, modification, calibration, etc., or when the alignment result such as the NCC value drops below a predetermined threshold.

[0098] The computer subsystem may repeatedly train the VAE model to produce predicted images that are better similar to real sample images. VAE model training may be performed using a relatively small number of real sample images that have been aligned to the design to initially train the VAE model, which may be performed as further described herein. The initially trained VAE model may then be used to generate more predicted images from the rasterized design. The computer subsystem may then perform NCC between the VAE predicted image and the real sample image to measure the offset and then shift the real sample image by the offset to align to the design, which may be performed as further described herein. These steps may generate more real sample images aligned to the design that may then be used for additional training of the VAE model. In other words, iterative VAE model training may include using an existing VAE model to generate predicted images, identifying a PDA offset between the predicted image and the real image, using the offset to generate an aligned sample image, and using the aligned image and the rasterized design image to retrain the VAE model. In this way, the computer subsystem may repeatedly use more aligned real sample images to better train the VAE model.

[0099] In one such embodiment, one or more computer subsystems are configured to perform retraining during runtime of the process performed on different samples. In this way, due to the substantially high processing power advantage of the embodiments described herein, the DL model can be updated in runtime to overcome process variation changes. In particular, due to the fast processing power of the embodiments described herein compared to currently used PDA methods, the embodiments can perform all of the steps described herein, including iterative training, in runtime of the process performed on any sample.

[0100] In another embodiment, one or more computer subsystems are configured to perform input, alignment, determination, and storage for a first portion of the sample and retrain the DL model by performing iterative training of the DL model for a second portion of the sample. For example, in one proposed process for a VAE-based PDA, an initial VAE model may be trained during PDA training or setup. A relatively small number of PCM renderings / generated images from a rasterization design at a good target position may be used to generate an aligned image. A "good target position" in this context refers to a position where a PCM rendering image can be used to measure a substantially accurate offset between the PCM rendering image and the true sample image. The computer subsystem may then repeatedly train the VAE model from scanning the first few scans (e.g., 5 scans) or the entire sample on the sample by the imaging subsystem. The imaging subsystem may then optionally be used to rescan the sample and the computer subsystem may use the VAE model to infer images from the rasterization design for all target positions to perform PDA alignment, as further described herein. In this way, a VAE model may be generated for a first sample. An initial VAE model may be trained / generated from a first scan of an image. The VAE model can then be retrained using iterative VAE model training from subsequent scans of the image to produce a better VAE model.

[0101] In one such embodiment, one or more computer subsystems are configured to perform input, alignment, determination, and storage and retraining for a first portion of the sample during runtime of a process performed on the sample. For example, an embodiment may be configured for subsequent VAE model updates during PDA runtime. The computer subsystem may use an existing VAE model to render / generate an image from a rasterized design at a selected target location to produce an aligned image. The computer subsystem may repeatedly update / train the VAE model from several scans before the current scan. The imaging subsystem may then rescan the sample, and the computer subsystem may use the updated VAE model to infer an image from a rasterized design for all target locations to perform PDA alignment, as further described herein.

[0102] In another embodiment, the computer subsystem is configured to perform a process on the sample using the imaging subsystem, and the process includes: aligning a target image of an inspection area on the sample to a design of the sample based on an align-to-design offset, such as Figure 5 The design information of the inspection area is transformed into a predicted target image of the inspection area by inputting the design information of the inspection area into the DL model, such as Figure 5 The predicted target image is subtracted from the aligned target image, as shown in step 502 of Figure 5 and applying the defect detection method to the subtracted result, such as Figure 5506 of . For example, in addition to performing PDA and attention area placement, the embodiments described herein may also generate predicted images that can be used as reference images for inspection. In other words, once the VAE model described herein is trained to generate predicted images of an alignment target, the same trained VAE model may be able to generate predicted images of other portions of the sample that closely resemble true images of those portions of the sample.

[0103] In this way, the inspection process may include setting up a PDA process as described herein or at least performing PDA using a small number of predicted images of one alignment target on the sample on which the inspection process is performed. The results of the alignment process may then be used to align other images to a design including any real sample image generated at any target or inspection area on the sample (which may include, for example, only the area of ​​interest or any area to be inspected). As described herein, the trained VAE model may be used prior to the process or during runtime to generate predicted images of the target or inspection area.

[0104] Since the predicted image is generated from design information (e.g., a rasterized design image), the predicted image is essentially a database (DB) reference image that can be subtracted from the real image after the predicted image and the real image have been aligned with each other, as further described herein. The subtraction can be performed in any suitable manner known in the art. The result of the subtraction can be a difference image showing any differences between the predicted image and the real image.

[0105] The inspection may include applying a defect detection method to the difference image to determine whether any of the differences indicate a defect or a defect candidate. For example, the defect detection method may include: applying a threshold to the difference image; determining that any pixel having a value above the threshold is a defect or a potential defect; and determining that all pixels having a value below the threshold are not defects or potential defects. Of course, this may be the simplest defect detection method that can be used in the inspection process, and any other suitable defect detection method may be used with the embodiments described herein. In other words, the predicted image generated by the VAE model for the inspection area on the sample can be used in the same manner as any other reference image in any defect detection method known in the art. In this way, the predicted inspection area image described herein is not specific to any particular defect detection method.

[0106] In addition to performing PDA, care area placement, and inspection, the embodiments described herein may also be useful in any system or method that performs alignment of different types of images to each other and / or alignment of information and / or images in different coordinate spaces. One example of this useful application is for performing image alignment to produce golden grid images, such as those described in U.S. Patent Application Publication No. 2022 / 0067898 published by Chen et al. on March 3, 2022, which is incorporated herein by reference as if fully set forth. The embodiments described herein may be further configured as described in such a publication.

[0107] The results and information generated by performing the processes described herein on a sample or other samples of the same type can be used in various ways through the embodiments and / or other systems and methods described herein. Such functions include, but are not limited to, changing a process, such as a manufacturing process or step that is or will be performed on a sample or another sample in a feedback or feedforward manner. For example, the computer subsystem may be configured to determine one or more changes to the process that is or will be performed on a sample inspected as described herein based on detected defects. The change in the process may include any suitable change in one or more parameters of the process. The computer subsystem preferably determines the change so that defects on other samples on which the revised process is performed can be reduced or prevented, defects on the sample can be corrected or eliminated in another process performed on the sample, defects can be compensated for in another process performed on the sample, etc. The computer subsystem can determine such changes in any suitable manner known in the art.

[0108] The changes may then be sent to a semiconductor manufacturing system (not shown) or a storage medium (not shown) accessible to the computer subsystem and the semiconductor manufacturing system. The semiconductor manufacturing system may or may not be part of the system embodiments described herein. For example, the computer subsystem and / or imaging subsystem described herein may be coupled to the semiconductor manufacturing system, for example, via one or more common elements such as a housing, a power supply, a sample handling device or mechanism, etc. The semiconductor manufacturing system may include any semiconductor manufacturing system known in the art, such as a lithography tool, an etching tool, a chemical-mechanical polishing (CMP) tool, a deposition tool, and the like.

[0109] In addition to the advantages further described herein, the embodiments described herein have several advantages. For example, the DL model described herein can predict more accurate sample images by design compared to the currently used PDA method. In particular, the DL predicted sample images generated as described herein are closer to the sample images than other currently used methods. Experiments performed by the inventors show that the NCC peak between the DL predicted image and the true sample image is higher than the NCC peak between the PCM predicted image and the true sample image, as further described herein.

[0110] The DL predicted images generated as described herein can also be used to generate more accurate PDA results than currently used PDA systems and methods. In addition, the DL-based methods described herein also provide substantially accurate PDA performed on out-of-focus images. In other words, due to the adaptive nature of the embodiments described herein, the real-time sample image is out of focus, and the embodiments described herein can still be used to generate predicted images that are sufficiently similar to the sample image that PDA can be successfully performed.

[0111] The DL-based PDA described herein can also be effectively implemented on a GPU and can be faster than currently used methods. The high processing power of these methods enables it to be used for each sample scan in runtime. Therefore, the embodiments described herein can improve the PDA accuracy on the inspection and other tools described herein. Since the DL-based method described herein can be easily implemented on a GPU with faster processing power, the embodiments can be used in the runtime PDA on the inspection and other tools described herein.

[0112] Furthermore, the embodiments described herein can predict closer to the real sample image from multiple layers of the design. In addition to being beneficial for PDAs, this capability also makes the embodiments useful in other applications such as defect detection where real sample images are compared to DL predicted images.

[0113] The above advantages of the embodiments are achieved by several important new features described herein. These features include, but are not limited to, DL-based methods that can predict more accurate sample images (e.g., optical inspection tool images) than other currently used methods and systems for PDA. Another new feature is a DL-based method for predicting sample images from rasterized design images. An additional new feature is a DL method for predicting sample images from multiple layers of a design. A further new feature is repeated DL model training for PDA. Another important new feature is the entire process for producing better PDA results on inspection and other tools.

[0114] Each of the embodiments of each system described above may be combined together into a single embodiment.

[0115] Another embodiment relates to a computer-implemented method for determining an offset used in a process performed on a sample. The method comprises the transforming, aligning, determining, and storing steps further described herein. These steps are performed by one or more computer systems. One or more components are performed by one or more computer systems, and the one or more components comprise a DL model.

[0116] Each step of the method may be performed as further described herein. The method may also include any other steps that may be performed by the system, computer system, and / or DL ​​model described herein. The computer system may be configured according to any embodiment described herein (e.g., computer subsystem 102). The DL model may be configured according to any embodiment described herein. In addition, the method described above may be performed by any system embodiment described herein.

[0117] Additional embodiments relate to a non-transitory computer-readable medium storing program instructions executable on one or more computer systems to perform a computer-implemented method for determining an offset used in a process performed on a sample. Figure 6 In particular, as shown in Figure 6 , non-transitory computer readable medium 600 includes program instructions 602 executable on a computer system 604. The computer-implemented method may include any of the steps of any of the methods described herein.

[0118] Program instructions 602 implementing methods such as those described herein may be stored on computer readable medium 600. The computer readable medium may be a storage medium such as a magnetic or optical disk, tape, or any other suitable non-transitory computer readable medium known in the art.

[0119] The program instructions may be implemented in any of a variety of ways, including process-based techniques, component-based techniques, and / or object-oriented techniques, etc. For example, the program instructions may be implemented using ActiveX controls, C++ objects, Python, JavaBeans, Microsoft Foundation Classes ("MFC"), SSE (Streaming SIMD Extensions), TensorFlow, CustomOp, CUDA, or other techniques or methodologies as desired.

[0120] Computer system 604 may be configured according to any of the embodiments described herein.

[0121] In view of this description, further modifications and alternative embodiments of various aspects of the present invention will be apparent to those skilled in the art. For example, methods and systems for determining the offset used in the process of performing on a sample are provided. Therefore, this description should be interpreted as illustrative only and is for the purpose of teaching those skilled in the art to practice the general manner of the present invention. It should be understood that the form of the present invention shown and described herein will be considered as the current preferred embodiment. As will be understood by those skilled in the art after having benefited from this description of the present invention, elements and materials may replace the elements and materials illustrated and described herein, parts and processes may be reversed, and specific features of the present invention may be utilized independently. Changes may be made to the elements described herein without departing from the spirit and scope of the present invention as described in the appended claims.

Claims

1. A system configured to determine an offset used in a process performed on a sample, comprising: one or more computer subsystems; and one or more components executed by the one or more computer subsystems, wherein the one or more components include a deep learning model; and wherein the one or more computer subsystems are configured to: transforming design information about an alignment target on a sample into a predicted image of the alignment target by inputting the design information into the deep learning model; aligning the predicted image to an image of the alignment target on the sample produced by an imaging subsystem; determining an offset between the predicted image and the image produced by the imaging subsystem based on a result of the aligning; and The determined offset is stored as an alignment-to-design offset for use in a process performed on the sample using the imaging subsystem.

2. The system of claim 1, wherein the deep learning model is configured as a variational autoencoder.

3. The system of claim 1 , wherein the one or more computer subsystems are further configured to convert the binary design image of the alignment target into a grayscale design image, and wherein the design information includes the grayscale design image instead of the binary design image. 4 . The system of claim 3 , wherein the deep learning model is further configured to transform the grayscale design image into a predicted grayscale design image. The system according to claim 1 , wherein the design information includes information of a plurality of layers of the design of the sample.

6. The system of claim 5, wherein the deep learning model is configured to have different weights for at least two of the plurality of layers that are determined separately during training of the deep learning model.

7. The system of claim 1, wherein the one or more computer subsystems are further configured to retrain the deep learning model by performing iterative training of the deep learning model for different samples.

8. The system of claim 7, wherein the one or more computer subsystems are further configured to perform the retraining during runtime of a process performed on the different samples.

9. A system according to claim 1, wherein the one or more computer subsystems are further configured to perform the input, alignment, determination and storage for a first portion of the sample and retrain the deep learning model by performing repeated training of the deep learning model for a second portion of the sample.

10. A system according to claim 9, wherein the one or more computer subsystems are further configured to perform the inputting, aligning, determining and storing the first portion of the sample and the retraining during runtime of the process performed on the sample.

11. A system according to claim 1, wherein the one or more computer subsystems are further configured to perform the input, alignment, determination and storage during the runtime of the process performed on the sample and during the runtime of the process performed on one or more other samples.

12. The system of claim 1, wherein the one or more computer subsystems are further configured to perform the process on the sample using the imaging subsystem, and wherein the process comprises: aligning a target image of an inspection area on the sample to a design of the sample based on the align-to-design offset; transforming the design information of the inspection area into a predicted target image of the inspection area by inputting the design information of the inspection area into the deep learning model; subtracting the predicted target image from the aligned target image; and applying a defect detection method to the subtracted result.

13. A system according to claim 1, wherein the one or more computer subsystems are further configured for training the deep learning model and setting up the process, and wherein the training and setting up do not include storing a setup prediction image of the alignment target used in the process performed on the sample using the imaging subsystem.

14. The system of claim 13, wherein the one or more computer subsystems are further configured to perform the process on the sample using the imaging subsystem and to perform the inputting, aligning, determining, and storing during the process, and wherein the offset is a run-time to design offset.

15. The system of claim 14, wherein the process further comprises accurately placing a region of interest on an inspection image generated by the imaging subsystem based on the run-time to design offset during the process performed on the sample.

16. A system according to claim 1, wherein the alignment target is one of a plurality of alignment targets positioned in a scan band of an image generated by the imaging subsystem, and wherein the one or more computer subsystems are further configured to perform the transforming, aligning, determining and storing steps for the plurality of alignment targets, to group the offsets determined for the plurality of alignment targets to produce a grouped offset, and to use the grouped offset to replace one or more of the offsets determined for the plurality of alignment targets.

17. The system of claim 1, wherein the process is an inspection process.

18. The system of claim 1, wherein the imaging subsystem is a light-based imaging subsystem.

19. The system of claim 1, wherein the imaging subsystem is an electron beam imaging subsystem.

20. A non-transitory computer readable medium storing program instructions executable on one or more computer systems to perform a computer-implemented method for determining a bias used in a process performed on a sample, wherein the computer-implemented method comprises: transforming design information about an alignment target on a sample into a predicted image of the alignment target by inputting the design information into a deep learning model, wherein one or more components are executed by the one or more computer systems, and wherein the one or more components include the deep learning model; aligning the predicted image to an image of the alignment target on the sample produced by an imaging subsystem; determining an offset between the predicted image and the image produced by the imaging subsystem based on a result of the aligning; and The determined offset is stored as an alignment-to-design offset for use in a process performed on the sample using the imaging subsystem, wherein the inputting, aligning, determining, and storing are performed by the one or more computer systems.

21. A computer-implemented method for determining an offset for use in a procedure performed on a sample, comprising: transforming design information about an alignment target on a sample into a predicted image of the alignment target by inputting the design information into a deep learning model, wherein one or more components are executed by one or more computer systems, and wherein the one or more components include the deep learning model; aligning the predicted image to an image of the alignment target on the sample produced by an imaging subsystem; determining an offset between the predicted image and the image produced by the imaging subsystem based on a result of the aligning; and The determined offset is stored as an alignment-to-design offset for use in a process performed on the sample using the imaging subsystem, wherein the inputting, aligning, determining, and storing are performed by the one or more computer systems.

Images