Image Reconstruction via Manifold Learning
By using neural networks for image reconstruction during semiconductor manufacturing, the problems of low efficiency and poor quality in existing image reconstruction technologies are solved, achieving efficient and fast image reconstruction results, which are suitable for inspection, measurement and defect inspection in semiconductor manufacturing processes.
Patent Information
- Application Number
- CN202580027614.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2025-02-25
- Filing Date
- 2025-05-22
- Publication Date
- 2026-11-17
AI Technical Summary
Existing image reconstruction methods suffer from ill-posed problems in image enhancement and restoration tasks during semiconductor manufacturing. Reliance on prior knowledge or image models makes it difficult to implement stable algorithms, and iterative algorithms are slow, limiting their practical applications.
Image reconstruction is performed using neural networks (NNs). By projecting patch images onto a low-dimensional manifold space learned from training images, the manifold learning technique is used to improve image quality. The system includes an imaging subsystem, a computer system, and a neural network to achieve efficient image reconstruction.
It improves the efficiency and quality of image reconstruction, enabling rapid and accurate reconstruction of images during semiconductor manufacturing processes under limited time and resources, and is suitable for inspection, metrology, and defect inspection.
Smart Images

Figure CN123125616A_ABST
Abstract
Description
Technical Field
[0001] This invention generally relates to methods and systems for image reconstruction. Some embodiments relate to image reconstruction via manifold learning. Background Technology
[0002] The following descriptions and examples are not considered prior art simply because they are included in this section.
[0003] Semiconductor devices, such as logic and memory devices, typically involve processing a substrate, such as a semiconductor wafer, using numerous semiconductor manufacturing processes to form various features and multiple layers of the semiconductor device. For example, photolithography is a semiconductor manufacturing process involving transferring a pattern from a magnifying mask to a photoresist disposed on a semiconductor wafer. Additional examples of semiconductor manufacturing processes include, but are not limited to, chemical mechanical polishing (CMP), etching, deposition, and ion implantation. Multiple semiconductor devices can be fabricated in an arrangement on a single semiconductor wafer, and then separated into individual semiconductor devices.
[0004] Inspection processes are used at various stages of the semiconductor manufacturing process to detect defects in samples, driving higher throughput and thus higher profits. Inspection has always been a crucial part of manufacturing semiconductor devices. However, as the size of semiconductor devices shrinks, inspection becomes even more critical for successfully manufacturing acceptable semiconductor devices, because even small defects can lead to device failure.
[0005] Defect inspection typically involves re-inspecting defects already detected by the inspection process and using high-magnification optical systems or scanning electron microscopy (SEM) to generate additional information about the defects at higher resolution. Therefore, defect inspection is performed at discrete locations on the sample where defects were detected by inspection. The higher-resolution data of defects generated through defect inspection are better suited for determining defect attributes such as profile, roughness, and more accurate size information. Compared to inspection, defects can be more accurately classified into defect types based on the information determined through defect inspection.
[0006] Metrology processes are also used at various steps during semiconductor manufacturing to monitor and control the process. Metrology differs from inspection processes because, unlike inspection processes which detect defects on samples, metrology is used to measure one or more characteristics of a sample that cannot be determined using currently available inspection tools. For example, metrology is used to measure one or more characteristics of a sample (e.g., the dimensions of features formed on the sample during the process, such as linewidth, thickness, etc.)) so that the performance of the process can be determined from these characteristics. Furthermore, if one or more characteristics of the sample are unacceptable (e.g., outside a predetermined range), the measurement of these characteristics can be used to modify one or more parameters of the process so that additional samples manufactured through the process have acceptable characteristics.
[0007] The metrology process also differs from the defect inspection process because, unlike defect inspection where defects detected by inspection are revisited, the metrology process can be performed at a location where defects have not yet been detected. In other words, unlike defect inspection, the location where the metrology process is performed on the sample can be independent of the results of the inspection process performed on the sample. Specifically, the location where the metrology process is performed can be chosen independently of the inspection results. Furthermore, because the location where the metrology process is performed on the sample can be chosen independently of the inspection results, unlike defect inspection where the location for defect inspection on the sample cannot be determined until the inspection results of the sample have been generated and are available, the location where the metrology process is performed can be determined before the inspection process has been performed on the sample.
[0008] Methods and systems for performing the yield-related processes described above are typically developed by first identifying the optimal possible hardware configuration for generating images, data, measurements, signals, etc., of the sample. Once the hardware configuration is established, the optimal hardware parameters for the process are selected. The choice of hardware parameters can significantly influence the responsiveness of images, data, measurements, signals, etc., to the sample and the extent to which they can be used to determine information about the sample.
[0009] Sometimes, even the best possible hardware configuration and associated parameters cannot produce an output that is ideal (or even good enough) for determining the information about the sample. For example, in the case described above, the best possible hardware configuration and parameters may still produce a suboptimal image for detecting or re-detecting defects on the sample, determining information about defects, measuring the characteristics of patterned features on the sample, etc. Therefore, especially as the tool reaches its maximum performance capability, increasing effort is spent on trying to improve the image after it has been generated by such tools.
[0010] Several currently used methods and systems for image enhancement are designed to deblur images. One such method uses a blind deconvolution algorithm. This method is used when information about the distortion is unknown. It attempts to simultaneously restore the image and the point scatter function (PSF). Another method uses the Lucy-Richardson algorithm as an iterative process to restore an image blurred by a known spatially invariant PSF. Different methods for image deblurring include regularization filters. This method adds a regularization term to the deconvolution process to handle noise. Yet another method for image deblurring uses a Wiener filter. This method is a statistical approach that assumes knowledge of the spectral properties of both the original image and the noise.
[0011] However, current methods and systems for image deblurring suffer from several significant drawbacks. For example, image enhancement and restoration tasks are often ill-posed problems, meaning they involve restoring image information lost during degradation (image acquisition). The need to rely on prior knowledge or image models to regularize the solution makes implementing stable algorithms substantially difficult. Furthermore, iterative algorithms are inherently slow, limiting their practical applications.
[0012] Therefore, it would be advantageous to develop systems and methods for image reconstruction that do not have one or more of the disadvantages described above. Summary of the Invention
[0013] The following description of various embodiments should not be construed in any way as limiting the subject matter of the appended claims.
[0014] One embodiment relates to a system configured for image reconstruction. The system includes an imaging subsystem configured to illuminate a sample with an energy source and, in response, generate an image. The sample has a repeating pattern formed thereon based on a semiconductor device design, the semiconductor device being formed from the sample. The system also includes a computer system configured to separate the image into patch images smaller than the image itself. Additionally, the system includes a neural network (NN) executed by the computer system and configured to, when the computer system inputs one of the patch images into the neural network, project the patch image onto a manifold containing feature vectors learned from training images, the image quality of which meets or exceeds predetermined criteria. The neural network is further configured to reconstruct the patch image from the feature vectors aligned to the manifold, thereby producing a reconstructed patch image having one or more image qualities better than the input patch image. The system may be further configured as described herein.
[0015] Another embodiment relates to a computer-implemented method for image reconstruction. The method includes illuminating a sample with an energy source using an imaging subsystem, and generating an image in response. The sample is configured as described above. The method also includes the separation, projection, and reconstruction described above. The separation is performed via a computer system, and the projection and reconstruction are performed via a neural network executed using the computer system. Each of the steps in the method described above can be performed as further described herein. The method may include any other steps of any other method described herein. The method can be performed using any of the systems described herein.
[0016] Another embodiment relates to a non-transitory computer-readable medium storing program instructions executable on a computer system to perform a computer-implemented method for image reconstruction. The computer-implemented method includes the steps of the methods described above. The computer-readable medium may be further configured as described herein. The steps of the computer-implemented method may be performed as further described herein. Additionally, the computer-implemented method for which the program instructions are executable may include any other steps of any other method described herein. Attached Figure Description
[0017] Further advantages of the invention will become apparent to those skilled in the art upon the following detailed description and with reference to the accompanying drawings, wherein:
[0018] Figure 1 and 2 This is a schematic diagram of a side view illustrating an embodiment of a system configured as described herein;
[0019] Figure 3 This is a flowchart illustrating an embodiment of a neural network (NN) that may be included in the embodiments described herein;
[0020] Figure 4 This is a flowchart illustrating an embodiment of NN training that can be performed by the embodiments described herein;
[0021] Figure 5 This is a schematic diagram illustrating an example of a tile image input to a neural network configured as described in this document and a reconstructed tile image generated by the neural network;
[0022] Figure 6 This is a flowchart illustrating an embodiment of steps that may be performed during runtime by the embodiments described herein; and
[0023] Figure 7 This is a block diagram illustrating an embodiment of a non-transitory computer-readable medium storing program instructions for causing a computer system to perform the computer-implemented methods described herein.
[0024] While the invention is open to various modifications and alternatives, specific embodiments thereof are shown by way of example in the drawings and described in detail herein. The drawings may not be to scale. However, it should be understood that the drawings and their detailed description are not intended to limit the invention to the specific forms disclosed, but rather to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims. Detailed Implementation
[0025] Turning now to the diagrams, it should be noted that the diagrams are not drawn to scale. In particular, the scale of some components in the diagrams has been greatly exaggerated to emphasize the characteristics of the components. It should also be noted that the diagrams are not drawn to the same scale. The same reference numerals have been used to indicate components that can be similarly configured and are shown in more than one diagram. Unless otherwise mentioned herein, any of the components described and shown may include any commercially available components.
[0026] Generally, the embodiments described herein are configured for image reconstruction. The image reconstruction described herein is particularly useful for samples having repeating patterns formed thereon. For example, a sample on which image reconstruction is performed as described herein has repeating patterns formed thereon based on a semiconductor device design, the semiconductor device being formed with the sample (e.g., formed on a sample such as a wafer or with a sample such as a reduction mask). The semiconductor industry relies on computer-aided design (CAD) tools that emphasize the use of hierarchical and common cell representations. Therefore, the patterns being examined exhibit very high repeatability.
[0027] This repeatability suggests the existence of a dense, low-dimensional manifold space suitable for image reconstruction, as further described herein. For example, theoretically, an image can be represented as points in a substantially high-dimensional space. An image of size W x H is a set of W x H pixels, where each pixel is represented by a number between 0 and 1. This creates a huge space, R^(w x n), where most points correspond to nothing more than noise. If you happen to fall very accidentally on a point representing something or of interest, then all its neighbors are again just noise. This results in a substantially sparse representation of the image corresponding to the real object. In contrast, especially in semiconductor technology where structures are parameterized using CAD tools (e.g., lines, groups of predictable angles (e.g., 45 and / or 90), quantized sizes, etc.), real images have a substantially finite number of possible arrangements compared to other types of images. A relatively low-dimensional subspace (manifold representation) can be found where each point corresponds to a possible arrangement. If this space is constructed efficiently, then neighboring points should have special relationships (e.g., latent space, etc.). The embodiments described herein utilize the properties of the images described herein in conjunction with manifold learning to provide several improvements and advantages over other currently used methods for image reconstruction (restoration).
[0028] As used interchangeably herein, the terms “design,” “design data,” and “design information” generally refer to the physical design (layout) of an IC or other semiconductor device, and data derived from the physical design through complex simulations or simple geometric and Boolean operations. A design may include any other design data or design data proxy described in the following commonly owned patents: U.S. Patent No. 7,570,796, issued August 4, 2009, to Zafar et al., and U.S. Patent No. 7,676,077, issued March 9, 2010, to Kulkarni et al., both of which are incorporated herein by reference as if fully described herein. Additionally, design data may be standard cell link library data, integrated layout data, design data for one or more layers, derivatives of design data, and complete or partial chip design data. Furthermore, the terms “design,” “design data,” and “design information” as used herein refer to information and data generated by a semiconductor device designer during the design process and which are therefore well-suited for use in the embodiments described herein prior to printing the design onto any physical sample, such as a photomask or wafer.
[0029] In some embodiments, the sample is a wafer on which a semiconductor device is formed. The wafer may comprise any wafer known in semiconductor technology. Although some embodiments may be described herein with respect to one or more wafers, the embodiments are not limited to semiconductor-related samples on which they can be used. For example, the embodiments described herein can be used for samples such as photomasks, tablets, personal computer (PC) boards, and other semiconductor samples. In the case of photomasks, the sample has a repeating pattern formed thereon based on a design of a semiconductor device formed (printed) on another sample using a photomask.
[0030] Figure 1 An embodiment of a system configured for image reconstruction is illustrated herein. System 10 includes an imaging subsystem 100 configured to illuminate a sample with an energy source and, in response thereto, generate an image. The imaging subsystem may be configured as one of the types of imaging subsystems described herein, such as an inspection, metrology, or defect inspection subsystem, and may include and / or be coupled to computer system 36 and / or one or more computer systems 102.
[0031] The terms “imaging system” and “imaging subsystem” are used interchangeably herein and generally refer to the hardware configured to generate images of a sample. Generally, the imaging subsystem described herein includes at least one energy source and a detector. The energy source is configured to generate energy directed to the sample. The detector is configured to detect the energy from the sample and generate an output in response to the detected energy.
[0032] In one embodiment, the energy source is a light source. For example, in a light-based imaging subsystem, the energy directed to the sample contains light, and the energy detected from the sample contains light. In one such example, such as Figure 1 As shown, the imaging subsystem includes an illumination subsystem configured to direct light to sample 14. The illumination subsystem includes at least one light source, such as light source 16. The illumination subsystem is configured to direct light to the sample at one or more incident angles, said one or more incident angles may include one or more tilt angles and / or one or more normal angles. For example, as... Figure 1 As shown, light from light source 16 is guided at an angle of incidence through optical element 18 and then through lens 20 to sample 14. The angle of incidence can include any suitable angle of incidence, which can vary depending on, for example, the characteristics of the sample and the defects to be detected on the sample, the characteristics of the sample to be measured, etc.
[0033] The illumination subsystem can be configured to guide light to the sample at different incident angles. For example, the imaging subsystem can be configured to change one or more parameters of one or more elements of the illumination subsystem, so that the light can be directed to the sample at different incident angles. Figure 1The image shows different incident angles guiding light to the sample. In one example, the imaging subsystem can be configured to move the light source 16, optics 18, and lens 20 such that light is guided to the sample at different tilted incident angles or normal (or near-normal) incident angles. The illumination subsystem can have any other suitable configuration known in the art for guiding light to the sample sequentially or simultaneously at one or more incident angles.
[0034] The illumination subsystem can also be configured to direct light with different characteristics to the sample. For example, optical element 18 can be configured as a spectral filter, and the properties of the spectral filter can be changed in a variety of different ways (e.g., by replacing one spectral filter with another) so that light of different wavelengths can be directed to the sample at different times.
[0035] Light source 16 may comprise a broadband plasma (BBP) source. In this manner, the light generated by the source and directed to the sample may comprise broadband light. However, the source may comprise any other suitable source, such as any suitable laser known in the art configured to produce light at any suitable wavelength. Additionally, the laser may be configured to produce monochromatic or near-monochromatic light. In this manner, the laser may be a narrow-band laser. The source may also comprise a multicolor source that produces light at multiple discrete wavelengths or bands.
[0036] Light from optical element 18 can be focused onto sample 14 by lens 20. Although lens 20 is... Figure 1 While shown as a single refractive optical element, lens 20 may actually comprise a combination of refractive and / or reflective optical elements that focus light from the optical element onto the sample. Figure 1 The illumination subsystem shown and described herein may include any other suitable optical elements (not shown). Examples of such optical elements include, but are not limited to, polarizing components, spectral filters, spatial filters, reflective optics, apodizers, beam splitters, apertures, and the like, which may include any such suitable optical elements known in the art. Additionally, the system may be configured to change one or more elements of the illumination subsystem based on the type of illumination to be used for imaging.
[0037] The imaging subsystem may also include a scanning subsystem configured to change the position on the sample where light is directed and detected, and may cause the light to scan across the sample. For example, the imaging subsystem may include a stage 22 on which sample 14 is placed during imaging. The scanning subsystem may include any suitable mechanical and / or robotic assembly (which includes stage 22) configured to move the sample such that light can be directed to and detected from different positions on the sample. Alternatively, the imaging subsystem may be configured such that one or more optical elements of the imaging subsystem perform a certain scan of the light across the sample, such that light can be directed to and detected from different positions on the sample. The light may scan across the sample in any suitable manner (e.g., in a serpentine or helical path).
[0038] The imaging subsystem includes one or more detection channels. At least one of the detection channels includes a detector configured to detect light from the sample due to illumination of the sample by the system, and to generate an output in response to the detected light. Figure 1 The imaging subsystem shown includes two detection channels: one formed by a light collector 24, element 26, and detector 28, and the other formed by a light collector 30, element 32, and detector 34. The two detection channels are configured to collect and detect light at different collection angles. In some examples, the two detection channels are configured to detect scattered light, and the detection channels are configured to detect light scattered from the sample at different angles. However, one or more of the detection channels may be configured to detect another type of light from the sample (e.g., reflected light).
[0039] exist Figure 1 In this embodiment, two detection channels are shown positioned in the plane of the paper, and the illumination subsystem is also shown positioned in the plane of the paper. Therefore, in this embodiment, the two detection channels are positioned in the plane of incidence (e.g., centered on the plane of incidence). However, one or more of the detection channels may be positioned outside the plane of incidence. For example, the detection channel formed by the light collector 30, element 32, and detector 34 may be configured to collect and detect light scattered from the plane of incidence. Therefore, this detection channel may generally be referred to as a "side" channel, and this side channel may be centered on a plane substantially perpendicular to the plane of incidence.
[0040] although Figure 1An embodiment of an imaging subsystem comprising two detection channels is shown; however, the imaging subsystem may comprise a different number of detection channels (e.g., only one detection channel or two or more detection channels). The detection channel formed by the light collector 30, element 32, and detector 34 may form a side channel as described above, and the imaging subsystem may include an additional detection channel (not shown) formed as another side channel positioned on the opposite side of the incident plane. Therefore, the imaging subsystem may include a detection channel comprising the light collector 24, element 26, and detector 28 centered on the incident plane and configured to collect and detect light at a scattering angle normal to or near normal to the sample surface. Therefore, this detection channel may generally be referred to as a “top” channel, and the imaging subsystem may also include two or more side channels configured as described above. Thus, the imaging subsystem may include at least three channels (i.e., one top channel and two side channels), and each of the at least three channels is configured to collect light at a scattering angle different from that of each of the other light collectors.
[0041] As further described above, one or more of the detection channels can be configured to detect scattered light. Therefore, Figure 1 The imaging subsystem shown herein can be configured for dark-field (DF) imaging. However, the imaging subsystem may also, or alternatively, include a detection channel configured for bright-field (BF) imaging. Therefore, the imaging subsystem described herein can be configured for DF only, BF only, or both DF and BF imaging. Although each of the condensers... Figure 1 The optical element is shown as a single refractive optical element; however, each of the optical collectors may contain refractive optical elements and / or reflective optical elements.
[0042] One or more detection channels may contain any suitable detector known in the art, such as a photomultiplier tube (PMT), charge-coupled device (CCD), and time-delay integration (TDI) camera. The detector may also contain non-imaging detectors or imaging detectors. If the detector is a non-imaging detector, each of the detectors may be configured to detect specific properties of light (e.g., intensity), but may not be configured to detect such properties that vary depending on position within the imaging plane. Therefore, the output generated by each of the detectors in each of the detection channels may be a signal or data, rather than an image signal or image data. In such examples, a computer system may be configured to generate an image of the sample from the non-imaging output of the detector. However, in other examples, the detector may be configured as an imaging detector configured to generate an imaging signal or image data. Therefore, the imaging subsystem may be configured to generate an image in several ways.
[0043] Computer system 36 may be coupled to the detector of the imaging subsystem in any suitable manner (e.g., via one or more transmission media, which may include “wired” and / or “wireless” transmission media), such that the computer system can receive the output generated by the detector. Computer system 36 may be configured to perform certain functions using the detector’s output, as further described herein. Computer system 36 may be further configured as described herein.
[0044] Computer system 36 (and other computer systems described herein) may also be referred to herein as a computer subsystem. Each of the computer subsystems or systems described herein may take various forms, including personal computer systems, graphics computers, mainframe computer systems, workstations, network devices, Internet devices, or other devices. Generally, the term "computer system" may be broadly defined to encompass any device having one or more processors that execute instructions from memory media. The computer subsystem or system may also include any suitable processor known in the art, such as a parallel processor. Additionally, the computer subsystem or system may include a computer platform (as a single-level or networked tool) with high-speed processing and software.
[0045] If the system comprises more than one computer system, the different computer systems can be coupled to each other, enabling the transmission of images, data, information, instructions, etc., between the computer systems. For example, computer subsystem 36 can be coupled to computer system 102 via any suitable transmission medium, such as in... Figure 1 The transmission medium, shown by dashed lines, may include any suitable wired and / or wireless transmission medium known in the art. Two or more of these computer systems may also be effectively coupled by sharing a computer-readable storage medium (not shown).
[0046] In another embodiment, the energy source is an electron beam source. For example, in an electron beam imaging subsystem, the energy directed to the sample contains electrons, and the energy detected from the sample also contains electrons. Figure 2 In one embodiment shown, the imaging subsystem includes an electron column 122, and the system includes a computer system 124 coupled to the imaging subsystem. The computer system 124 may be configured as described above. Alternatively, this imaging subsystem may be identical to those described above and in... Figure 1 The method shown in the diagram is coupled to one or more other computer systems.
[0047] For example Figure 2As shown, the electron column includes an electron beam source 126 configured to generate electrons that are focused onto the sample 128 by one or more elements 130. The electron beam source may include, for example, a cathode source or an emitter tip, and the one or more elements 130 may include, for example, a gun lens, an anode, a beam-limiting aperture, a gate valve, a beam current selection aperture, an objective lens, and a scanning subsystem, all of which may include any such suitable elements known in the art.
[0048] Electrons returning from the sample (e.g., secondary electrons) can be focused onto detector 134 by one or more elements 132. One or more elements 132 may include, for example, a scanning subsystem, which may be the same scanning subsystem included in element 130.
[0049] The electronic column may include any other suitable element known in the art. Additionally, the electronic column may be further configured as described in the following patents: U.S. Patent No. 8,664,594, issued April 4, 2014, to Jiang et al.; U.S. Patent No. 8,692,204, issued April 8, 2014, to Kojima et al.; U.S. Patent No. 8,698,093, issued April 15, 2014, to Gubbens et al.; and U.S. Patent No. 8,716,662, issued May 6, 2014, all of which are incorporated herein by reference as if fully described herein.
[0050] Despite the electron column Figure 2 The electron beam is shown to be configured such that electrons are guided to the sample at an angle of incidence and scattered from the sample at another angle; however, the electron beam can be guided to and scattered from the sample at any suitable angle. Furthermore, the electron beam imaging subsystem can be configured to use multiple modes to produce sample outputs (e.g., using different illumination angles, collection angles, etc.), as further described herein. The multiple modes of the electron beam imaging subsystem can differ in terms of the parameters produced at any output of the imaging subsystem.
[0051] Computer system 124 may be coupled to detector 134, as described above. The detector detects electrons returning from the surface of the sample, thereby forming an electron beam image (or other output) of the sample. The electron beam image may contain anything suitable for an electron beam image. Computer system 124 may be configured to perform any of the steps described herein. Figure 2 The imaging subsystem shown in the document can be further configured as described herein.
[0052] This article provides Figure 1 and 2The configuration of the imaging subsystems that may be included in the system embodiments described herein is explained in general terms. It is evident that the imaging subsystem arrangements described herein can be modified to optimize the performance of the imaging subsystem as is typically performed when designing commercial imaging systems. Additionally, the systems described herein can be implemented using existing imaging subsystems, such as those commercially available from KLA Corporation in Milpitas, California (e.g., by adding the functionality described herein to an existing inspection system). For some such systems, the methods described herein may provide optional functionality for the imaging subsystem (e.g., in addition to other functionality of the imaging system). Alternatively, the imaging systems described herein can be designed "from scratch" to provide entirely new systems.
[0053] Although the imaging subsystem has been described above as a light or electron beam imaging subsystem, it can also be an ion beam imaging subsystem. This imaging subsystem can be as follows: Figure 2 The configuration shown is such that, except that the electron beam source can be replaced with any suitable ion beam source known in the field, the imaging subsystem may include any other suitable ion beam system, such as those found in commercially available focused ion beam (FIB) systems, helium ion microscopy (HIM) systems, and secondary ion mass spectrometry (SIMS) systems.
[0054] An imaging subsystem can be configured to produce outputs (e.g., images) of a sample using multiple modes. Generally, a "mode" is defined by parameter values of the imaging subsystem used to produce an image of the sample (or by the output used to produce an image of the sample). Therefore, modes can differ in the value of at least one of the parameters of the imaging subsystem (except for the lateral position on the sample where the output is produced). For example, modes can differ in any one or more modifiable parameters of the imaging subsystem (e.g., illumination polarization, illumination angle, illumination wavelength, etc.; detection polarization, detection angle, detection wavelength, etc.). The imaging subsystem can be configured to scan the sample using different modes in the same scan or different scans, for example, depending on the ability to scan the sample simultaneously using multiple modes.
[0055] Similarly, the electron beam subsystem can be configured to generate images using two or more modes, which can be defined by parameter values of the electron beam subsystem used to generate images of the sample. Therefore, the modes can differ in the value of at least one of the electron beam parameters of the electron beam subsystem. For example, different modes can use different incident angles for illumination.
[0056] The imaging subsystem described herein can be configured as an inspection system, a metrology system, and / or a defect inspection system. For example, Figure 1 and 2The embodiments of the imaging subsystem shown can be modified on one or more parameters to provide different imaging capabilities depending on the application to which it will be used. In one such example, if the imaging subsystem will be used for metrology rather than inspection, it can be configured to have a higher resolution. In other words, Figure 1 and 2 The embodiments of the imaging subsystem shown in the text describe some general and various configurations for the imaging subsystem, which can be customized in several ways that will be understood by those skilled in the art to produce systems with different imaging capabilities that are more or less suitable for different applications.
[0057] In this way, the imaging subsystem can be configured to generate outputs suitable for detecting or re-inspecting defects on a sample in an inspection system or defect inspection system, and for measuring one or more characteristics of the sample in a metrology system. In an inspection system, Figure 1 The computer system 36 shown herein can be configured to detect defects on sample 14 by applying a defect detection method or algorithm to the output generated by one or more of the detectors, or according to one or more embodiments described herein. In the defect inspection system, Figure 2 The computer system 124 shown can be configured to re-detect defects on sample 128 by applying a defect re-detection method to the output generated by detector 134, and may use the output generated by the detector to determine additional information about the re-detected defects. In a metrology system, Figure 1 The computer system 36 shown herein can be configured to determine one or more characteristics of sample 14 using the output generated by detectors 28 and / or 34. The system can be further configured to detect or re-detect defects on the sample, determine the characteristics of the sample, determine other information about the sample, etc., as further described herein.
[0058] As mentioned above, the imaging subsystem is configured to scan a physical version of the sample with energy (e.g., light, electrons, etc.), thereby producing an output of the physical version of the sample. In this way, the imaging subsystem can be configured as a "real" subsystem, rather than a "virtual" subsystem. However, storage media (not shown) and Figure 1 The computer system 102 shown herein can be configured as a “virtual” system. Specifically, the storage medium and the computer system can be configured as a “virtual” imaging system, as described in the following commonly assigned patents: U.S. Patent No. 8,126,255, issued February 28, 2012, to Bhaskar et al., and U.S. Patent No. 9,222,895, issued December 29, 2015, which are incorporated herein by reference as if fully described herein. The embodiments described herein can be further configured as described in those patents.
[0059] The system includes a computer system, which may include the computer subsystems described above or any configuration of any of the systems. The computer system is configured to transfer images ( Figure 6 The image 600 from the imaging subsystem shown herein is separated into patch images smaller than the image (as shown in step 602). For example, an image with relatively high resolution (e.g., a scanning electron microscope (SEM) image with a size of 1024 x 1024 pixels) may be divided into smaller patch images, each measured to be 128 x 128 pixels. However, the input and patch image sizes can vary depending on the imaging subsystem arrangement, how the images generated by the imaging subsystem arrangement are typically processed for inspection, defect inspection, and / or metrology, and the configuration of the embodiments described herein. Separating the image into patch images that are subsequently input into the NN as described herein is a crucial step in the embodiments described herein. For example, the idea that the sample images described herein should have a manifold is not obvious and appears to work only when the inventors divide the image into smaller patch images to match a low-dimensional representation.
[0060] The embodiments described herein have been found to be of particular interest within the context of SEM image restoration and automated tool calibration. However, the image input to the separation step may contain any of the outputs of the imaging subsystem further described herein, such as images, image data, signals, image signals, etc. The image input to the separation step may contain the raw detector output, meaning that the detector output has not been processed in any way prior to the separation step. However, the image input to the separation step may be an image that has been processed in some way, as it may be in a normal inspection (or other) process. This processing may include, for example, high-pass filtering, image alignment, and the like.
[0061] Each (or at least one) of the image tiles separated from the input image can be reconstructed (restored) individually, as further described herein. For example, only one of the image tiles at the location of the detected defect in the original image can be reconstructed, only one of the image tiles can be reconstructed and then used for alignment of the original image, or only one of the image tiles can be reconstructed and used for any other purpose. This limited tile image reconstruction can be performed when time and other resources are limited (even if the embodiments described herein are faster than other currently available image restoration methods and systems). However, the image reconstruction described herein can be performed for any or all of the image tiles from which the original image is separated. Reconstructing all image tiles can also be performed depending on the reason for image reconstruction. For example, not all image tiles may need to be reconstructed for image alignment, but all image tiles may be reconstructed when performing defect detection using reconstructed image tiles.
[0062] The system also includes Figure 1The neural network (NN) 104 shown is executed by a computer system and configured to be used when the computer system inputs one of the tile images into the NN (e.g., ...). Figure 6 In step 604 (as shown in step 606), one of the tile images is projected onto a manifold containing feature vectors learned from training images, the image quality of which meets or exceeds predetermined criteria. For example, the image tiles are transformed into feature vectors of a compact manifold with only 64 (1 x 64) dimensions, but retain the complete information needed to reconstruct the original 128 x 128 pixel tiles.
[0063] In one embodiment, a manifold is a low-dimensional representation of a training image that preserves the relationships between data points in the training image. Unlike autoencoders designed to simply compress data, manifold methods focus on discovering and utilizing the inherent patterns or geometric properties of higher-dimensional data stores. The goal of manifold learning is to find a low-dimensional representation of the data that preserves the important relationships between data points.
[0064] In another embodiment, the neural network (NN) is configured to learn feature vectors through unsupervised learning. In an additional embodiment, training images are generated by an imaging subsystem using the best known parameters of the imaging subsystem. The premise is that the manifold is learned unsupervised from a set of relatively high-quality images, preferably reflecting optimal imaging subsystem conditions such as focus, astigmatism, and high signal-to-noise ratio (SNR). Predetermined criteria satisfied or exceeded by the training images can be defined by any or any other one or more criteria suitable for image quality, which may vary depending on the reason for image generation. For example, predetermined criteria may include one or more of the following image qualities: a focused image, an image without astigmatism or with astigmatism below a certain measure, an image with an SNR above a certain threshold, an image with at least one predetermined qualitative or quantitative image sharpness, contrast, etc. In this way, training images can be high-quality images, where what is qualitatively or quantitatively high-quality can be determined by the user, the reason for image generation, the sample, the imaging subsystem, and any other such parameters of the methods and systems described herein.
[0065] The optimal imaging subsystem conditions may include one or more best known patterns (BKMs) of the process performed on the sample using images generated by the imaging subsystem, or be determined from said one or more BKMs. For example, a computer system may select one or more BKMs as the optimal imaging subsystem conditions for the process. The computer system may identify or determine the BKM in any suitable manner. For example, the BKM may be identified by searching for process recipes for the process and the sample in a storage medium (e.g., a wafer fabrication database or a storage medium accessible to the computer system) and identifying the patterns in the process recipes as optimal imaging subsystem conditions. In this way, the optimal imaging subsystem conditions may vary depending on the process (e.g., array inspection versus logic inspection, inspection versus metrology, etc.) and the sample type (etched wafer versus photolithographic wafer, wafer versus magnified mask, wafer with a front-end layer formed thereon versus wafer with a back-end layer formed thereon, etc.).
[0066] Instead of BKM, the computer system can select a default set of best known imaging parameters as the optimal imaging subsystem conditions, even if the imaging parameters may not be suitable for the process, for example, even if they are the best available imaging parameters on the tool, they may be too slow for the overall sample inspection process.
[0067] In one of these embodiments, such as Figure 4 As shown, a high-quality image 400 and an untrained (or untrained) neural network 402 can be input into an unsupervised learning step 404 to produce a result 406 comprising a trained neural network with a learned manifold. Generally, unsupervised training means that the high-quality images used for training are not labeled; however, unsupervised training can be performed in any suitable manner known in the field. A computer system can be configured to perform unsupervised training. Alternatively, another system or method can perform unsupervised training, and the computer system can retrieve or access the trained neural network from or on a computer-readable storage medium in which the other system or method has stored the trained neural network. In this way, one entity can establish and train the neural network, and another entity can use the neural network as described herein.
[0068] In some examples, a neural network trained on an image of one sample can be used to reconstruct images of different samples, which may or may not have the same design formed thereon. For example, high-quality images can be generated for each sample on a per-sample basis, making it possible to train a neural network for each sample. However, this training or retraining of the neural network for each sample may be unnecessary, and for samples whose design and images are sufficiently similar, the trained neural network can be reused without retraining.
[0069] The NN is also configured to reconstruct one of the tiles in the tile image from the feature vectors aligned to the manifold on the tile image (e.g., Figure 6 (As shown in step 608), thereby generating a reconstructed tile image (reconstructed tile image 610) with one or more image qualities better than the input tile image. When a degraded image (e.g., a blurred image or an image with a relatively low SNR) is projected onto the manifold, it is aligned with the feature vectors of a clean image. For example, when a degraded image is projected onto the manifold, it does not fall precisely on the manifold because it is degraded. However, due to the properties of the manifold (dense representation), if the computer system or NN moves from the projection point to the nearest point on the manifold and reconstructs the image, then the reconstructed image is guaranteed to have a “possible” pattern, which means sharp and clean (here it is assumed that only sharp and clean patterns are used to build the manifold). Specifically, the NN is the learning to transform the image (high dimension) into a lower dimension (manifold) and to extend the representation from the original representation.
[0070] Therefore, the embodiments described herein differ from other methods of image “reconstruction” in several important ways. For example, in general, it is impossible to solve in high-dimensional spaces using direct maximum a posteriori (MAP) estimation (or computationally cumbersome methods such as solving inverse problems). However, when the data can be represented by a low-dimensional manifold, then simple flows (e.g., normalized flows) can be used for computation, and the probability can be maximized through invertible transformations, normalized flows, gradient descent, etc. To keep things simple, if the problem can be decomposed into two steps, the first step is dimensionality reduction (manifold projection) through injective transformations, and the second step is MAP estimation in low dimensions, thereby providing an end-to-end solution to restore degraded high-resolution images.
[0071] Therefore, the reconstruction process produces an image with one or more image qualities that are better than the input tile image. For example, the reconstructed image may be sharp and / or noise-free. Thus, in one embodiment, the reconstructed tile image contains one or more image qualities with less blur than the input tile image. Figure 5 This section illustrates an example of manifold reconstruction for image deblurring. Specifically, image 500 is an example of a tile image that can be input to a neural network as described herein. Image 502 is an example of a reconstructed tile image produced by image restoration performed via manifold reconstruction as described herein. As readily apparent by comparing images 500 and 502, the reconstructed image has significantly less blur than the input image. In another embodiment, one or more image qualities of the reconstructed tile image contain less noise than the input tile image. In other words, the reconstructed tile image may be quieter than the input image and have a higher SNR for sample features and defects than the input image.
[0072] The neural networks included in the system can have any suitable network topology, optimization process, loss function, probability estimation, etc. known in their respective fields. Figure 3 This illustration describes an example of a portion of the neural network (NN) used in the embodiments described herein. In this example, a latent distribution 300 may be generated for an input patch image by one or more layers (not shown) contained in the NN. Arrows pointing to the right in the figure indicate the sample generation step, and arrows pointing to the left indicate the encoded data stream. For sample generation, the latent distribution is input to a p-jet stream 302, which projects the latent distribution onto a singlet map 304. The singlet map projection of the input latent distribution is input to a singlet generator 308 portion of NN layer 306, which projects the singlet map projection of the input latent distribution onto a manifold 310. The data 312 aligned to the singlet map projection may be input to an encoder 314 contained in layer 306, which encodes the data back into the singlet map 304 space. Next, the portion of the injective graph corresponding to data 312 can be input to a counterflow 302, which outputs a corresponding latent distribution, which can be input to other layers of the neural network (not shown) to generate a reconstructed patch image. Examples of suitable neural network configurations that can be used in the embodiments described herein can also be found in "TRUMPETS: Injective Flows for Inference and Inverse Problems" by Kothari et al. (arXiv:2102.10461v1, February 20, 2021, p. 16), which is incorporated herein by reference as if described in its entirety. The embodiments described herein can be further configured as described in this reference.
[0073] In one embodiment, the computer system is configured to align the reconstructed tile image to a design (e.g., as in...). Figure 6 (As illustrated in step 612 of the alignment to design process). For example, a particularly useful and important application of the embodiments described herein is image-to-design alignment, such as CAD to SEM alignment. Images captured during rapid acquisition processes (such as verification) often exhibit problems such as suboptimal focus and relatively low SNR, especially when only a single frame is collected. These deficiencies can significantly hinder the accuracy of SEM to CAD registration algorithms to mitigate this challenge. Manifold image restoration can be employed, as described herein, to enhance image quality before attempting CAD registration.
[0074] Once a reconstructed tile image has been aligned to the design, other tile images in the original image can be aligned to the design based on an aligned reconstructed tile image (e.g., using a simple coordinate transformation determined from the reconstructed tile image aligned to the design). Alternatively, each tile image that the original image has been separated into (or as many tile images as needed) can be reconstructed and then each individually aligned to the design.
[0075] The image-to-design alignment algorithm or method into which the reconstructed tile image is input can be any suitable alignment algorithm or method known in the art. In other words, the reconstructed image produced by the embodiments described herein can be used for image-to-design alignment in the same manner as any other image currently input to such algorithms and methods.
[0076] In another embodiment, the computer system is configured to segment the reconstructed tile image based on one or more characteristics of the reconstructed tile image (such as in...). Figure 6 (See image segmentation step 614 shown in the text). For example, another particularly useful and important application of the embodiments described herein is image segmentation. It is often difficult to locate objects and boundaries in relatively low-quality images. Assigning labels to each pixel in an image so that pixels with the same label share specific characteristics (noise, etc.) is crucial for defect detection (e.g., so that pixels sharing specific characteristics can be inspected with the same inspection sensitivity). Just as with image-to-design alignment, a reconstructed tile image can be segmented, and then the segmentation result can be used to segment other original tile images. However, if desired, each tile image can also be reconstructed, and then each reconstructed tile image can be segmented. The image segmentation algorithm or method into which the reconstructed tile image is input can be any suitable image segmentation algorithm or method known in the art. In other words, the reconstructed tile image produced by the embodiments described herein can be used for image segmentation in the same way as any other image currently input to such algorithms and methods.
[0077] The computer system can be further configured for imaging subsystem calibration, such as Figure 6 Step 616 is illustrated. For example, in an additional embodiment, the computer system is configured to automatically calibrate the imaging subsystem based on the difference between the reconstructed patch image and one of the inputs in the patch image. In yet another embodiment, the computer system is configured to automatically calibrate the imaging subsystem based on the difference between the reconstructed patch image and one of the inputs in the patch image when performing a process on a sample with the imaging subsystem. For example, an additional particularly useful and important application of the embodiments described herein is automated tool calibration, which may or may not be performed during the process runtime.
[0078] During relatively long inspection operations, the system (such as the column configuration in an electron beam system) must be kept perfectly calibrated. However, this calibration is a fundamentally challenging task, especially for relatively complex multi-beam systems, such as those containing up to 300 beams. Using the embodiments described herein (manifold image reconstruction), a side-by-side evaluation of the unprocessed image can be performed against an idealized version. The results of this evaluation allow for fine-tuning of calibration parameters to reduce discrepancies, ultimately leading to perfect calibration of the tool. This operation can be performed in situ (with the wafer held on the stage). The imaging subsystem calibration algorithm or method into which the reconstructed patch image is input can also be any suitable calibration algorithm or method known in the art. In other words, the reconstructed patch image generated by the embodiments described herein can be used for imaging subsystem calibration in the same manner as any other image currently input to such algorithms and methods.
[0079] A significant advantage of the embodiments described herein for applications such as the automated calibration described above, and other possible applications described herein, is the real-time (or near-real-time) performance of image reconstruction and the practicality of applications utilizing the reconstructed images. For example, an neural network can generate reconstructed (or idealized) images of patch images while generating patch images, and the computer system can continuously compare the generated images with their idealized images for improving flow robustness, flow rate, image consistency, image calibration, etc. While this image reconstruction and use may be particularly advantageous for some of the applications described herein, any of the image reconstruction and applications described herein can certainly be performed offline or tooling-based, for example, after all images of the sample have been generated and / or via a computer system not coupled to the imaging subsystem (e.g., Figure 1 The computer system 102 shown in the image.
[0080] The computer subsystem can also be configured for mode evaluation of the imaging subsystem, such as... Figure 6 Step 618 is shown. For example, another particularly useful and important application of the embodiments described herein is pattern evaluation, or simply determining information about the pattern of an imaging subsystem. In one such embodiment, training images are generated for one or more first slabs on a sample, projection and reconstruction are performed on additional patch images generated for second slabs on the sample, thereby generating corresponding reconstructed additional patch images, and a computer system is configured to determine information about the pattern of the imaging subsystem used to generate the additional patch images based on the differences between the additional patch images and their corresponding reconstructed additional patch images.
[0081] This embodiment can be useful because the imaging parameters or patterns preferably result in images unaffected by variations in the process or sample (e.g., color variations) that are not of interest. The embodiments described herein can be used to learn manifolds on specific slabs (e.g., one or more central slabs) on a sample, and to perform side-by-side evaluation of unprocessed images against an idealized version across other different slabs on the sample. The patterns can then be evaluated based on the degree of difference between the generated image and its idealized version, depending on the slab or other location on the sample. For example, a pattern exhibiting the same (or substantially the same) difference between the corresponding idealized image and the original image for different locations on the sample may advantageously be less sensitive to process or sample variations (such as color) than a pattern whose corresponding idealized and original images have differences varying according to their location on the sample.
[0082] The pattern evaluation algorithm or method into which the reconstructed tile image is input can also be any suitable pattern evaluation algorithm or method known in the art. In other words, the reconstructed image generated by the embodiments described herein can be used for pattern evaluation in the same manner as any other image currently input to such algorithms and methods. The pattern evaluation described above can also be performed as part of the pattern selection application described herein.
[0083] The embodiments described herein can be further configured for imaging subsystem mode selection, such as... Figure 6 Step 620 is shown. For example, an additional particularly useful and important application of the embodiments described herein is image mode selection, which can be particularly important for BBP-type inspection tools and SEM tools. In one such embodiment, training images are generated for one or more first slabs on a sample, and projection and reconstruction are performed on additional patch images generated for a second slab on the sample using multiple modes of the imaging subsystem, thereby generating corresponding reconstructed additional patch images for different combinations of the second slab and multiple modes, and the computer system is configured to select one or more of the multiple modes for the process performed on the sample by the imaging subsystem based on the difference between the additional patch images and their corresponding reconstructed additional patch images. This embodiment is suitable for situations where optimal imaging parameters (e.g., the selected mode) should result in images unaffected by unrelated processes or variations in the sample (e.g., color variations). The embodiments described herein can be used to learn manifolds on specific slabs (one or more central slabs) on a sample, and to perform side-by-side evaluation of unprocessed images for an idealized version across other different slabs on the sample.
[0084] In another mode selection possibility, the difference between the idealized image and the original image can be determined for different modes. In this way, it can be determined how far the image of the mode is from the ideal image. In other words, a comparison between the reconstructed image and the original image can be performed to determine the similarity between the images, which can be determined quantitatively (e.g., based on SNR) where possible, or qualitatively for image quality that is not easily quantifiable. Then, one or more modes that are most similar to the generated and reconstructed images can be selected for the process performed on the sample. In some cases, such as for testing processes, it may also be useful to select multiple modes that have significantly different capabilities for imaging the sample, and such modes can also be selected based on the comparison results described above.
[0085] The pattern selection algorithm or method that inputs the reconstructed tile image into it can also be any suitable pattern selection algorithm or method known in the art. In other words, the reconstructed image generated by the embodiments described herein can be used for pattern selection in the same way as any other image currently input into such algorithms and methods.
[0086] Once the mode has been selected for the process, the computer system or another method or system may select one or more other parameters for the program. Such parameters may include any image processing parameters, any parameters of the method or system used to determine information from the image generated using the selected mode (such as a defect detection algorithm), any parameters of the method or system used to determine additional information about the sample from the determined information, any parameters of the method or system used to produce the results of the process, etc. As just one example, the computer subsystem may be configured to determine a threshold for a defect detection method that will be applied to the image generated by the selected mode to detect defects in the image. Such parameters may be selected using any suitable method known in the art.
[0087] In an additional embodiment, training images are generated for one or more first dies on the sample, and projection and reconstruction are performed on additional patch images generated for second dies on the sample, thereby generating corresponding reconstructed additional patch images. The computer system is configured to determine cross-sample variations in sample characteristics (e.g., ...) based on the differences between the additional patch images and their corresponding reconstructed additional patch images. Figure 6 (As shown in step 622). For example, another particularly useful and important application of the embodiments described herein is cross-wafer process variation estimation.
[0088] Training images can be generated as described herein, and the first shrapnel can be, for example, one or more central shrapnels on the sample. Projection and reconstruction of the second shrapnel image can be performed as described herein, and the computer system can determine any differences in suitability properties of the sample based on the location on the sample, according to the differences between the original patch image and the reconstructed patch image of the second shrapnel.
[0089] The sample characteristics used in this application may be independent of discrete defects that may or may not be detected on samples as described herein. For example, sample characteristics may be color variations, film thickness, and the like, which may not necessarily be defects or defect characteristics of the sample, but can be detected by the comparisons described above and used to determine potentially important information about the sample or the process performed on it based on its location across the sample. Alternatively, the sample characteristics and their variations across samples may be determined in other ways using any suitable method or algorithm known in the art.
[0090] Another particularly useful and important application of the embodiments described herein is single-die inspection. For example, in yet another embodiment, a computer system is configured to perform single-die defect detection on a sample by identifying differences between an input and a reconstructed tile image in a tile image, and detecting defects in one of the tile images based on the identified differences. Figure 6 (As shown in step 624). In this type of defect detection, the collected (i.e., original or input) image can be compared with the idealized or reconstructed image. In this way, the input image can serve as the test image, and the reconstructed image can be used as a reference.
[0091] In other words, if a high-quality image with defects is input into a neural network, then after projection onto a manifold and reconstruction, the entire image, except for the defects themselves, should be accurately reconstructed. The differences can then be used to identify defects in the collected images. Specifically, by subtracting an idealized image from the original image, only the differences should be the defects. Therefore, this type of defect detection can also be viewed as single-image defect detection where defect detection is performed using self-generated references.
[0092] This type of defect detection can be performed on any image input to the neural network, not just on a bare chip. For example, this defect detection can be performed as cell detection, single-frame detection, single-array region detection, single-field detection, and the like.
[0093] Of course, single-die defect detection is only one type of defect detection in which the images described herein can be used. Specifically, the reconstructed and original images can be input into any suitable defect detection algorithm or method known in the art. In other words, the reconstructed image generated by the embodiments described herein and the original image input to the NN can be used for defect detection in the same way as any other image currently input to such algorithms and methods. For example, the defect detection algorithm could be the MDAT algorithm, which is commercially available on some inspection tools from KLA. The defect detection algorithm may also include deep learning (DL) type defect detection algorithms.
[0094] The embodiments described herein can also be configured for tool-to-tool matching, such as Figure 6 Step 626 is shown. In some such embodiments, training images are generated through different imaging subsystems, and the computer system is configured to adjust one or more parameters of the imaging subsystems to match the different imaging subsystems based on the difference between the input and reconstructed tile images in the tile images. For example, a user may want to achieve a perfect (or as close to perfect) match between two different electron beam tools (tool 1 and tool 2). A manifold can be learned on tool 1 (reference tool), and the embodiments described herein can adjust tool 2 to produce an image that minimizes the difference from the reconstructed image of the manifold. One or more parameters of tool 2 adjusted to produce an image that minimizes such differences may include any of the imaging subsystem parameters described herein. The embodiments described herein can identify and adjust such parameters in any suitable manner known in the art.
[0095] Compared to currently used methods and systems, the embodiments described herein offer several significant improvements and advantages for the image reconstruction and applications described herein. For example, the embodiments described herein for image restoration utilize advances in deep learning (DL), which have revolutionized the field of computer vision. DL methods (especially convolutional neural networks (CNNs)) have become the cornerstone for solving complex computer vision tasks. These methods offer several advantages over traditional image restoration techniques.
[0096] One advantage is improved accuracy. For example, DL models can learn and generalize from large amounts of data, leading to more accurate image analysis and recognition. Another advantage is feature extraction. CNNs automatically detect and utilize the most relevant features from images, bypassing the need for manual feature engineering. An additional advantage is versatility. DL models can handle a wide range of computer vision tasks, from image classification to object detection and segmentation. Yet another advantage is adaptability. These models can be trained on new data to improve their performance over time, making them adaptable to various scenarios.
[0097] By incorporating recent research in deep learning, the embodiments described in this paper aim to overcome the limitations of traditional methods and provide a more robust solution for image restoration and enhancement in computer vision. Instead of relying on assumed image priors to guide the algorithm in solving image restoration challenges, the embodiments described in this paper rely solely on the availability of a set of relatively high-quality images. These images should be generated under nuanced and relaxed conditions. The model is then trained to recognize how the initial image appears when presented alongside its degraded counterpart.
[0098] The advantages described above are provided through several important new features of the embodiments described herein. For example, due to the repetitive nature of chips in CAD designs, the inventors have found that a low-dimensional representation is necessary. Learning this manifold enables the embodiments described herein to reshape the image restoration problem into a projection and reconstruction process. Any deviations from the accurate manifold representation due to image degradation, noise, blurring, etc., can be recovered through the embodiments described herein.
[0099] Computer systems can be configured to store reconstructed images, original images, samples, imaging subsystems, etc., and / or any other results generated by the embodiments described herein, in any suitable computer-readable storage medium. Any of such images, information, results, etc., described herein can be stored in any manner known in the art. Storage media may include any storage media described herein or any other suitable storage media known in the art. After information has been stored, it can be accessed in the storage medium and used by any party in the methods or systems embodiments described herein, formatted for display to a user, or used by another software module, method, or system.
[0100] In some embodiments of the execution mode selection, for example, a computer system may be configured to store information about the selected mode for subsequent steps in the mode selection process and / or for, for example, the testing of a sample. The computer system may be configured to store information in a recipe or by generating a recipe for a process that will use the selected mode. A “recipe,” as used herein, may be broadly defined as a set of instructions that can be used with tools to perform a process on a sample. In this way, generating a recipe may include information on how to perform the process, which can then be used to generate instructions for performing the process. The information about the selected mode stored by the computer system may include any information that can be used to identify and / or use the selected mode (e.g., filename and its storage location, and the file may contain information about the mode, such as mode name, mode parameter values, etc.). The testing recipe may then be stored and used by a system or method (or another system or method) to test a sample or other sample to generate information about the sample or other sample (e.g., defect information).
[0101] Computer systems and / or inspection systems can be configured to perform inspection processes on a sample and / or other samples of the same type using the results of one or more steps described herein. This inspection process can produce results for any defects detected on the sample, such as information about the bounding box of the detected defect (e.g., location, etc.), a detection score, information about the defect classification (e.g., category label or ID, etc.), or any such suitable information known in the field. Results for defects can be generated by the computer system and / or inspection system in any suitable manner. Results for defects can have any suitable form or format, such as a standard document type. The computer system and / or inspection system can generate and store results, enabling the results to be used by the computer system and / or another system or method to perform one or more functions on the sample or another sample of the same type. For example, the information can be used by the computer system or another system or method to sample defects for defect inspection or other analysis, determine the root cause of defects, etc.
[0102] Similarly, the computer systems and / or other imaging subsystems described herein (e.g., metrological or defect inspection subsystems) can be configured to use the results of one or more steps described herein to perform other processes on samples and / or other samples of the same type. Such metrological or defect inspection processes can produce any suitable results known in the art, and the results of such processes can be generated and / or stored by computer systems and / or imaging subsystems as described herein. Those results can then be used in the same manner as described herein.
[0103] Functions that can be performed using this information also include, but are not limited to, modifying processes in a feedback or feedforward manner, such as manufacturing processes or steps that have been performed on or will be performed on the inspected sample or another sample. For example, a computer system can be configured to determine one or more modifications to a process that has been performed on or will be performed on a sample as described herein, based on the detected defects. The process modification may include any suitable changes to one or more parameters of the process. The computer system preferably determines the modification such that defects can be reduced or prevented on other samples in which the modified process is performed, defects on the sample can be corrected or eliminated in another process performed on the sample, defects can be compensated for in another process performed on the sample, etc. The computer system can determine such modifications in any suitable manner known in the art. The results of other processes described herein can also be used to determine such modifications.
[0104] The changes can then be sent to a semiconductor manufacturing system (not shown) or a computer system and a storage medium (not shown) accessible by the semiconductor manufacturing system. The semiconductor manufacturing system may be, or may not be, part of the system embodiments described herein. For example, the computer system and / or inspection system described herein may be coupled to the semiconductor manufacturing system, for example, via one or more common elements (e.g., housing, power supply, sample handling device or mechanism, etc.). The semiconductor manufacturing system may include any semiconductor manufacturing system known in the art, such as lithography tools, etching tools, chemical mechanical polishing (CMP) tools, deposition tools, and the like.
[0105] Therefore, as described herein, the embodiments can be used to establish new processes or formulations. The embodiments can also be used to modify existing processes or formulations, whether they are processes or formulations for a sample, or processes or formulations generated for one sample and adapted to another.
[0106] Each of the embodiments described above can be combined into a single embodiment. In other words, unless otherwise mentioned herein, none of the embodiments are mutually exclusive with any other embodiments.
[0107] Another embodiment relates to a computer-implemented method for image reconstruction. The method includes illuminating a sample with an energy source using an imaging subsystem and generating an image in response. The sample has a repeating pattern formed thereon based on a semiconductor device design, the semiconductor device being formed from the sample. The method further includes separating the image into patch images smaller than the original image. When one of the patch images is input into a neural network (NN) executed by a computer system, the method includes projecting the patch image onto a manifold containing feature vectors learned from training images, the image quality of which meets or exceeds predetermined criteria. Additionally, the method includes reconstructing the patch image from the feature vectors aligned to the patch image on the manifold, thereby producing a reconstructed patch image having one or more image qualities better than the input patch image. Separation is performed by a computer system, and projection and reconstruction are performed by the NN.
[0108] Each of the steps in the method may be performed as further described herein. The method may also include any other steps that can be performed by the imaging subsystem, computer system, and NN described herein. Furthermore, the method described above can be performed by any of the system embodiments described herein.
[0109] Additional embodiments relate to a non-transitory computer-readable medium storing program instructions executable on a computer system to perform a computer-implemented method for image reconstruction. Figure 7 An example of this embodiment is shown below. Specifically, as... Figure 7As shown, the non-transitory computer-readable medium 700 contains program instructions 702 that can be executed on a computer system 704. A computer-implemented method may include any step of any method described herein.
[0110] The program instructions 702 for implementing the method (such as the method described herein) may be stored on a computer-readable medium 700. The computer-readable medium may be a storage medium, such as a disk or optical disk, magnetic tape, or any other suitable non-transitory computer-readable medium known in the art.
[0111] Program instructions can be implemented in any of a variety of ways, including procedural, component-based, and / or object-oriented technologies. For example, program instructions can be implemented using ActiveX controls, C++ objects, JavaBeans, Microsoft Foundation Classes (“MFC”), SSE (Streaming SIMD Extensions), Python, Tensorflow, or other technologies or methods as needed.
[0112] Computer system 704 may be configured according to any of the embodiments described herein.
[0113] In view of this description, those skilled in the art will understand further modifications and alternative embodiments of various aspects of the invention. For example, methods and systems for image reconstruction are provided. Therefore, this description should be construed as illustrative only and is intended to teach those skilled in the art the general manner in which the invention is practiced. It should be understood that the forms of the invention shown and described herein should be considered as presently preferred embodiments. All as those skilled in the art will understand after benefiting from this description of the invention that elements and materials may be substituted for those described herein, parts and processes may be reversed, and certain properties of the invention may be utilized independently. Changes may be made to the elements described herein without departing from the spirit and scope of the invention as set forth in the appended claims.
Claims
1. A system configured for image reconstruction, comprising: An imaging subsystem configured to illuminate a sample with an energy source and generate an image in response thereto, wherein the sample has a repeating pattern formed thereon based on a semiconductor device design, the semiconductor device being formed from the sample; A computer system configured to separate the image into patch images smaller than the image; and A neural network, executed by the computer system and configured to: When the computer system inputs one of the tile images into the neural network, it projects the tile image onto a manifold that includes feature vectors learned from training images, the image quality of which meets or exceeds a predetermined criterion. and Reconstructing one of the tile images from the feature vector aligned to the manifold of the one of the tile images, thereby producing a reconstructed tile image with one or more image qualities better than the input one of the tile images.
2. The system of claim 1, wherein the manifold is a low-dimensional representation of the training image that preserves the relationships between data points in the training image.
3. The system of claim 1, wherein the neural network is further configured to learn the feature vector through unsupervised learning.
4. The system of claim 1, wherein the training images are generated by the imaging subsystem using the best known parameters of the imaging subsystem.
5. The system of claim 1, wherein the image quality of the reconstructed tile image includes less blurring than that of the input tile image.
6. The system of claim 1, wherein the one or more image qualities of the reconstructed tile image include less noise than the input one in the tile image.
7. The system of claim 1, wherein the computer system is further configured to align the reconstructed tile image to the design.
8. The system of claim 1, wherein the computer system is further configured to segment the reconstructed tile image based on one or more characteristics of the reconstructed tile image.
9. The system of claim 1, wherein the computer system is further configured to automatically calibrate the imaging subsystem based on the difference between the reconstructed tile image and the input in the tile image.
10. The system of claim 1, wherein the computer system is further configured to automatically calibrate the imaging subsystem based on the difference between the reconstructed patch image and the input in the patch image when performing the process on the sample with the imaging subsystem.
11. The system of claim 1, wherein the training images are generated for one or more first dies on the sample, wherein the projection and reconstruction are performed on additional patch images generated for second dies on the sample, thereby generating corresponding reconstructed additional patch images, and wherein the computer system is further configured to determine information about the mode of the imaging subsystem for generating the additional patch images based on the difference between the additional patch images and their corresponding reconstructed additional patch images.
12. The system of claim 1, wherein the training images are generated for one or more first slabs on the sample, wherein the projection and reconstruction are performed on additional patch images generated for second slabs on the sample using multiple modes of the imaging subsystem, thereby generating corresponding reconstructed additional patch images for different combinations of the second slabs and the multiple modes, and wherein the computer system is further configured to select one or more of the multiple modes for the process performed on the sample using the imaging subsystem based on the difference between the additional patch images and their corresponding reconstructed additional patch images.
13. The system of claim 1, wherein the training images are generated for one or more first dies on the sample, wherein the projection and reconstruction are performed on additional patch images generated for second dies on the sample, thereby generating corresponding reconstructed additional patch images, and wherein the computer system is further configured to determine cross-sample variations of the characteristics of the sample based on the differences between the additional patch images and their corresponding reconstructed additional patch images.
14. The system of claim 1, wherein the computer system is further configured to perform single-die defect detection on the sample by identifying differences between the input one in the tile image and the reconstructed tile image and detecting defects in the one in the tile image based on the identified differences.
15. The system of claim 1, wherein the training images are generated by different imaging subsystems, and wherein the computer subsystem is further configured to adjust one or more parameters of the imaging subsystem to match the different imaging subsystems based on the difference between the input in the tile image and the reconstructed tile image.
16. The system of claim 1, wherein the energy source is a light source.
17. The system of claim 1, wherein the energy source is an electron beam source.
18. The system of claim 1, wherein the imaging subsystem is further configured as an inspection subsystem.
19. A non-transitory computer-readable medium storing program instructions executable on a computer system to perform a computer-implemented method for image reconstruction, wherein the computer-implemented method includes: An imaging subsystem illuminates a sample with an energy source and generates an image in response thereto, wherein the sample has a repeating pattern based on a semiconductor device design formed thereon, the semiconductor device being formed from the sample; The image is separated into patch images smaller than the image itself; When one of the tile images is input into a neural network executed by the computer system, the tile image is projected onto a manifold that includes feature vectors learned from training images whose image quality meets or exceeds predetermined criteria. and Reconstructing one of the tiles in the tile image from the feature vector aligned to the manifold, thereby producing a reconstructed tile image with one or more image qualities better than the input tile image, wherein the projection and reconstruction are performed by the neural network.
20. A computer-implemented method for image reconstruction, comprising: An imaging subsystem illuminates a sample with an energy source and generates an image in response thereto, wherein the sample has a repeating pattern based on a semiconductor device design formed thereon, the semiconductor device being formed from the sample; The image is separated into patch images smaller than the image itself; When one of the tile images is input into a neural network executed by a computer system, the tile image is projected onto a manifold that includes feature vectors learned from training images whose image quality meets or exceeds predetermined criteria. and Reconstructing one of the tiles in the tile image from the feature vector aligned to the manifold, thereby producing a reconstructed tile image having one or more image qualities better than the input tile image, wherein the separation is performed by the computer system, and wherein the projection and reconstruction are performed by the neural network.
Citation Information
Patent Citations
Methods and systems for utilizing design data in combination with inspection data
US7570796B2
Methods and systems for utilizing design data in combination with inspection data
US7676077B2
Systems and methods for creating persistent data for a wafer and for using persistent data for inspection-related functions
US8126255B2
Electron-optical system for high-speed and high-sensitivity inspections
US8664594B1
Apparatus and methods for electron beam detection
US8692204B2
