Knowledge distillation for semiconductor-based applications

Abstract

This invention provides a method and system for determining information about a sample. The system includes a computer subsystem and one or more components executed by the computer subsystem. The one or more components include multiple deep learning (DL) models configured to determine information about the sample based on outputs generated for a sample using a learning mode of an imaging subsystem. The one or more components also include a knowledge distillation component configured to combine the outputs generated by the multiple DL models. Additionally, the one or more components include a final knowledge distillation DL model configured to determine information about the sample or an additional sample based on outputs generated for the sample or an additional sample using a runtime mode of the imaging subsystem. Before the final KD DL model determines the information, the knowledge distillation component is configured to perform supervised training of the final knowledge distillation DL model using the combined outputs.

Description

Technical Field

[0001] This invention generally relates to a method and system for determining information about a sample. A specific embodiment relates to determining information about a sample using a final knowledge distillation deep learning model trained by a knowledge distillation component using information generated by multiple deep learning models. Prior Technology

[0002] The following descriptions and examples are not considered prior art simply because they are included in this section.

[0003] Fabricating semiconductor devices (such as logic devices and memory devices) typically involves processing a substrate (such as a semiconductor wafer) using numerous semiconductor fabrication processes to form the various features and multiple layers of the semiconductor device. For example, lithography involves a semiconductor fabrication process that transfers a pattern from a photomask to a resist disposed on a semiconductor wafer. Additional examples of semiconductor fabrication processes include, but are not limited to, chemical mechanical polishing (CMP), etching, deposition, and ion implantation. Multiple semiconductor devices can be fabricated in a configuration on a single semiconductor wafer and then separated into individual semiconductor devices.

[0004] Inspection procedures are used at various steps during a semiconductor manufacturing process to detect defects in samples, contributing to higher yields and (therefore) higher profits in the manufacturing process. Inspection has always been an important part of manufacturing semiconductor devices. However, as semiconductor device sizes shrink, inspection becomes even more critical for successfully manufacturing acceptable semiconductor devices, as even small defects can lead to device malfunction.

[0005] Defect re-inspection typically involves re-detecting defects, such as those detected by an inspection procedure, and using a high-magnification optical system or a scanning electron microscope (SEM) to generate additional information about the defects at a higher resolution. Therefore, defect re-inspection is performed at discrete locations on the sample where defects were already detected by inspection. The higher-resolution data generated by defect re-inspection is more suitable for determining defect attributes, such as profile, roughness, and more accurate size information. Compared to inspection, information determined by defect re-inspection can usually more accurately classify defects into different defect types.

[0006] Measurement procedures are also used at various steps during a semiconductor manufacturing process to monitor and control the process. Measurement procedures differ from inspection procedures in that, unlike inspection procedures which detect defects on a sample, measurement procedures are used to measure one or more characteristics of a sample that cannot be determined using the currently used inspection tools. For example, a measurement procedure is used to measure one or more characteristics of a sample, such as the dimension of a feature formed on the sample during a process (e.g., linewidth, thickness, etc.), so that the effectiveness of the process can be determined based on these one or more characteristics. Furthermore, if one or more characteristics of a sample are unacceptable (e.g., outside a predetermined range), the measurement of these characteristics can be used to modify one or more parameters of the process so that additional samples manufactured by the process have acceptable characteristics.

[0007] The difference between a measurement procedure and a defect re-inspection procedure lies in the fact that, unlike a defect re-inspection procedure which re-examines defects detected by inspection, a measurement procedure can be performed at locations where defects were not detected. In other words, unlike defect re-inspection, the location on a sample to perform a measurement procedure can be independent of the results of an inspection procedure performed on the sample. Specifically, the location to perform a measurement procedure can be chosen independently of the inspection results. Furthermore, because the location on the sample to be measured can be chosen independently of the inspection results, unlike defect re-inspection where the location on the sample to be re-inspected cannot be determined until inspection results for the sample are generated and available, the location to perform a measurement procedure can be determined before an inspection procedure has already been performed on the sample.

[0008] Advances in deep learning have made it an attractive framework for use in procedures such as those described above. For example, certain testing procedures using machine learning or deep learning can be supervised through a convolutional neural network (CNN) or object detection network. While such machine learning or deep learning methods offer several advantages, they also have several disadvantages. For instance, some previously used models require multiple input patterns to achieve relatively high performance. In another instance, many current methods require substantially large training datasets, which are not always practically available or may incur substantially high ownership costs in terms of time and physical costs (such as wafers) in obtaining results.

[0009] Therefore, it would be advantageous to develop systems and methods for determining information about a sample that do not suffer from one or more of the disadvantages described above. Summary of the Invention

[0010] The following description of various embodiments should in no way be construed as limiting the scope of the appended patent application.

[0011] One embodiment relates to a system configured to determine information about a sample. The system includes a computer subsystem and one or more components executed by the computer subsystem, the components including multiple deep learning (DL) models configured to determine information about the sample based on outputs generated for a sample using one or more learning modes of an imaging subsystem. The components also include a knowledge distillation (KD) component configured to combine the outputs generated by the multiple DL models. Additionally, the components include a final knowledge distillation DL model configured to determine information about the sample or an additional sample based on outputs generated for the sample or an additional sample using one or more runtime modes of the imaging subsystem. Before the final knowledge distillation DL model determines the information, the KD component is configured to perform supervised training of the final knowledge distillation DL model using the combined outputs. The system can be further configured as described herein.

[0012] Another embodiment relates to a computer-implemented method for determining information about a sample. The method includes determining information about the sample by inputting output generated for a sample using one or more learning modes of an imaging subsystem into multiple deep learning (DL) models, and combining the outputs generated by the multiple DL models. The method also includes performing supervised training on a final knowledge distillation DL model using the combined output, and determining information about the sample or the additional sample by inputting output generated for the sample or an additional sample using one or more runtime modes of the imaging subsystem into the final knowledge distillation DL model. Determining information about the sample, combining the output, performing supervised training, and determining information about the sample or the additional sample are performed by a computer subsystem. Each step of the method described above may be performed as further described herein. Embodiments of the method described above may include any other step of any other method described herein. The method described above may be performed by any of the systems described herein.

[0013] Another embodiment relates to a non-transitory computer-readable medium storing program instructions executable on a computer system for performing a computer-implemented method for determining information of a sample. The computer-implemented method includes the steps of the method described above. The computer-readable medium may be further configured as described herein. The steps of the computer-implemented method may be performed as further described herein. Additionally, the computer-implemented method with executable program instructions may include any other steps of any other method described herein. Simple Explanation of the Diagram

[0014] Further advantages of the invention will become apparent to those skilled in the art from the following detailed description of preferred embodiments and from reference to the accompanying drawings, in which: Figures 1 and 1a are schematic side views illustrating one embodiment of the system configuration described herein; Figure 2 is a flowchart illustrating one embodiment of the steps that can be performed to determine information about a sample; and Figure 3 is a block diagram illustrating one embodiment of a non-transitory computer-readable medium storing program instructions for causing a computer system to execute one of the computer implementation methods described herein. While various modifications and alternatives to the invention are readily apparent, specific embodiments are shown by way of example in the drawings and will be described in detail herein. These drawings may not be to scale. However, it should be understood that the drawings and detailed description thereof are not intended to limit the invention to the specific forms disclosed, but rather, the invention is intended to be covered by all modifications, equivalents, and alternatives within the spirit and scope of the invention as defined by the appended claims. Implementation

[0015] Turning now to the diagrams, it should be noted that the diagrams are not drawn to scale. Specifically, some components in the diagrams are enlarged significantly to emphasize their characteristics. It should also be noted that the diagrams are not drawn to the same scale. The same component symbols have been used to indicate components that can be similarly configured and shown in more than one diagram. Unless otherwise stated herein, any component illustrated and shown may include any commercially available components.

[0016] Generally speaking, the embodiments described herein are configured to determine information about a sample for inspection applications (e.g., detecting defects on a sample) and / or other semiconductor-based applications (such as measurement and defect re-inspection).

[0017] In some embodiments, the sample is a wafer. This wafer may comprise any wafer known in semiconductor technology. Although certain embodiments may be described herein with respect to one or more wafers, the embodiments are not limited in terms of the samples they can be used with. For example, the embodiments described herein can be used with samples such as reduction masks, tablets, personal computer (PC) boards, and other semiconductor samples.

[0018] Figure 1 illustrates one embodiment of a system configured to determine information about a sample. In some embodiments, system 10 includes an imaging subsystem, such as imaging subsystem 100. The imaging subsystem includes and / or is coupled to a computer subsystem, such as computer subsystem 36 and / or one or more computer systems 102.

[0019] Generally, the imaging subsystem described herein includes at least one energy source, one detector, and one scanning subsystem. The energy source is configured to generate energy that is directed to a sample by the imaging subsystem. The detector is configured to detect energy from the sample and generate an output in response to the detected energy. The scanning subsystem is configured to change one of the locations on the sample where energy is directed and from which energy is detected. In one embodiment, as shown in Figure 1, the imaging subsystem is configured as a light-based imaging subsystem.

[0020] In the light-based imaging subsystem described herein, the energy directed to the sample includes light, and the energy detected from the sample also includes light. For example, in an embodiment of the system shown in Figure 1, the imaging subsystem includes an illumination subsystem configured to direct light to sample 14. The illumination subsystem includes at least one light source. For example, as shown in Figure 1, the illumination subsystem includes light source 16. The illumination subsystem is configured to direct light to the sample at one or more incident angles (which may include one or more oblique angles and / or one or more normal angles). For example, as shown in Figure 1, light from light source 16 is guided through optical element 18 and then lens 20 to reach sample 14 at an oblique incident angle. This oblique incident angle may include any suitable oblique incident angle, which may vary depending on, for example, the characteristics of the sample and the procedures to be performed on the sample.

[0021] The illumination subsystem can be configured to direct light to the sample at different times and at different incident angles. For example, the imaging subsystem can be configured to change one or more characteristics of one or more elements of the illumination subsystem so that light can be directed to the sample at an incident angle different from that shown in Figure 1. In one such example, the imaging subsystem can be configured to move the light source 16, the optical element 18, and the lens 20 to direct light to the sample at a different oblique incident angle or a normal (or near-normal) incident angle.

[0022] In some examples, the imaging subsystem can be configured to guide light to the sample simultaneously at more than one incident angle. For instance, the illumination subsystem may include more than one illumination channel, one of which may include the light source 16, optical element 18, and lens 20 shown in FIG. 1, and another of the illumination channels (not shown) may include similar elements that can be configured differently or identically, or may include at least one light source and possibly one or more other components (such as those further described herein). If such light is guided to the sample simultaneously with other light, one or more characteristics (e.g., wavelength, polarization, etc.) of the light guided to the sample at different incident angles may be different, such that the light induced by illumination of the sample at different incident angles can be distinguished from each other at the detector.

[0023] In another example, the illumination subsystem may contain only one light source (e.g., source 16 shown in Figure 1), and light from that light source may be separated into different optical paths (e.g., based on wavelength, polarization, etc.) by one or more optical elements (not shown) of the illumination subsystem. Light in each of the different optical paths can then be directed to the sample. Multiple illumination channels can be configured to direct light to the sample simultaneously or at different times (e.g., when different illumination channels are used to sequentially illuminate the sample). In another example, the same illumination channel can be configured to direct light with different characteristics to the sample at different times. For example, optical element 18 may be configured as a spectral filter, and the properties of the spectral filter may be modified in various ways (e.g., by replacing one spectral filter with another) to direct different wavelengths of light to the sample at different times. The illumination subsystem may have any other suitable configuration known in this art for guiding light with different or the same characteristics sequentially or simultaneously to the sample at different or the same incident angles.

[0024] Light source 16 may comprise a broadband plasma (BBP) light source. In this way, the light generated by the light source and directed to the sample may comprise broadband light. However, the light source may comprise any other suitable light source known in the art, configured to produce light at any suitable wavelength, such as any suitable laser. The laser may be configured to produce monochromatic or near-monochromatic light. In this way, the laser may be a narrow-band laser. The light source may also comprise a multicolor light source that produces light at multiple discrete wavelengths or bands.

[0025] Light from optical element 18 can be focused onto sample 14 by lens 20. Although lens 20 is shown as a single refractive optical element in Figure 1, in practice, lens 20 may comprise several refractive and / or reflective optical elements that combine to focus light from optical element 18 onto sample. The illumination subsystem shown in Figure 1 and described herein may include any other suitable optical elements (not shown). Examples of such optical elements include (but are not limited to) polarizing components, spectral filters, spatial filters, reflective optical elements, apodizers, beam splitters, apertures, and the like, which may include any such suitable optical elements known in the art. Furthermore, the system can be configured to change one or more of the elements of the illumination subsystem based on the type of illumination to be used for imaging.

[0026] The imaging subsystem may also include a scanning subsystem configured to change the position on the sample to which light is directed and detected, and possibly to cause the light to scan above the sample. For example, the imaging subsystem may include a stage 22 on which the sample 14 is placed during imaging. The scanning subsystem may include any suitable mechanical and / or robotic assembly (which includes stage 22) configured to move the sample such that light can be directed to and detected from different positions on the sample. Alternatively, the imaging subsystem may be configured such that one or more optical elements of the imaging subsystem perform certain scans of light above the sample, such that light can be directed to and detected from different positions on the sample. In examples of scanning light above the sample, the light may be scanned above the sample in any suitable manner (such as in a serpentine path or a spiral path).

[0027] The imaging subsystem further includes one or more detection channels. At least one of these detection channels includes a detector configured to detect light from the sample attributable to illumination of the sample by the imaging subsystem and to generate an output in response to the detected light. For example, the imaging subsystem shown in FIG1 includes two detection channels, one formed by a light collector 24, element 26, and detector 28, and the other formed by a light collector 30, element 32, and detector 34. As shown in FIG1, the two detection channels are configured to collect and detect light at different collection angles. In some examples, the two detection channels are configured to detect scattered light, and these detection channels are configured to detect light scattered from the sample at different angles. However, one or more of the detection channels may be configured to detect another type of light from the sample (e.g., reflected light).

[0028] As further illustrated in Figure 1, both detection channels are positioned within the plane of the paper, and the illumination subsystem is also positioned within the plane of the paper. Therefore, in this embodiment, both detection channels are positioned within the incident plane (e.g., centered therein). However, one or more of the detection channels may be positioned outside the incident plane. For example, the detection channel formed by the light collector 30, element 32, and detector 34 can be configured to collect and detect light scattered outside the incident plane. Therefore, this detection channel may generally be referred to as a "side" channel, and this side channel may be centered in a plane substantially perpendicular to the incident plane.

[0029] Although Figure 1 illustrates one embodiment of an imaging subsystem comprising two detection channels, the imaging subsystem may comprise a different number of detection channels (e.g., only one detection channel or two or more detection channels). In one such example, the detection channel formed by the light collector 30, element 32, and detector 34 may form a side channel as described above, and the imaging subsystem may include an additional detection channel (not shown) formed as another side channel positioned on the opposite side of the incident plane. Thus, the imaging subsystem may include a detection channel comprising the light collector 24, element 26, and detector 28, centered in the incident plane, and configured to collect and detect light at a scattering angle normal to or near the sample surface. This detection channel may therefore be commonly referred to as a "top" channel, and the imaging subsystem may also include two or more side channels configured as described above. Thus, the imaging subsystem may include at least three channels (i.e., one top channel and two side channels), and each of the at least three channels has its own light collector, each of which is configured to collect light at a different scattering angle compared to each of the other light collectors.

[0030] As further explained above, each of the detection channels included in the imaging subsystem can be configured to detect scattered light. Therefore, the imaging subsystem shown in Figure 1 can be configured for dark-field (DF) imaging of the sample. However, the imaging subsystem may also, or alternatively, include detection channels configured for bright-field (BF) imaging of the sample. In other words, the imaging subsystem may include at least one detection channel configured to detect light reflected from the specular surface of the sample. Therefore, the imaging subsystem described herein can be configured for DF-only imaging, BF-only imaging, or both DF and BF imaging. Although each of the condensers is shown as a single refractive optical element in Figure 1, each of the condensers may include one or more refractive optical elements and / or one or more reflective optical elements.

[0031] One or more detection channels may include any suitable detector known in this art, such as a photomultiplier tube (PMT), charge-coupled device (CCD), and time-delay integration (TDI) camera. The detector may also include non-imaging detectors or imaging detectors. If the detector is a non-imaging detector, each of the detectors may be configured to detect certain characteristics (such as intensity) of the scattered light, but may not be configured to detect these characteristics based on its position in the imaging plane. Thus, the output generated by each of the detectors in each of the detection channels included in the imaging subsystem may be a signal or data, but not an image signal or image data. In such an example, a computer subsystem (such as computer subsystem 36) may be configured to generate an image of the sample from the non-imaging output of the detector. However, in other examples, the detector may be configured as an imaging detector, which is configured to generate an imaging signal or image data. Therefore, the imaging subsystem can be configured to generate images in several ways.

[0032] It should be noted that Figure 1 is provided herein to generally illustrate a configuration of one of the imaging subsystems that may be included in the system embodiments described herein. Clearly, the imaging subsystem configuration described herein can be modified to optimize the performance of the imaging subsystem, as is typically done when designing a commercial imaging system. Alternatively, the system described herein can be implemented using existing systems such as those from the 29xx / 39xx series of tools commercially available from KLA Corporation (Miapida, California) (e.g., by adding the functionality described herein to an existing inspection system). For some of these systems, the methods described herein may be provided as optional functionality of the system (e.g., in addition to other system functionalities). Alternatively, the system described herein can be designed "from scratch" to provide a completely new system.

[0033] Computer subsystem 36 can be coupled to the detector of the imaging subsystem in any suitable manner (e.g., via one or more transmission media, which may include "wired" and / or "wireless" transmission media), so that the computer subsystem can receive the output generated by the detector. Computer subsystem 36 can be configured to perform several functions using the detector's output. For example, if the system is configured as an inspection system, the computer subsystem can be configured to use the detector's output to detect events (e.g., defects and potential defects) on the sample. Detection of events on the sample can be performed as further described herein.

[0034] Computer subsystem 36 can be further configured as described herein. For example, computer subsystem 36 can be configured to perform the steps described herein. Thus, the steps described herein can be performed "on the tool" of a computer subsystem coupled to or being a part of an imaging subsystem. Alternatively, computer system 102 can perform one or more of the steps described herein. Therefore, one or more of the steps described herein can be performed "outside the tool" of a computer system not directly coupled to an imaging subsystem.

[0035] Computer subsystem 36 (and other computer subsystems described herein) may also be referred to herein as a computer system. Each of the computer subsystems or systems described herein may take various forms, including a personal computer system, a video computer, a mainframe computer system, a workstation, a network appliance, an internet appliance, or other device. Generally, the term "computer system" may be broadly defined to encompass any device having one or more processors that executes instructions from a memory medium. A computer subsystem or system may also include any processor known in the art suitable for execution, such as a parallel processor. Additionally, a computer subsystem or system may include a computer platform with high-speed processing and software as a standalone tool or a network connectivity tool.

[0036] If the system comprises more than one computer subsystem, the different computer subsystems can be coupled to each other, allowing images, data, information, instructions, etc., to be transmitted between the computer subsystems. For example, computer subsystem 36 can be coupled to computer system 102 via any suitable transmission medium (shown by the dotted line in Figure 1), such transmission medium may include any suitable wired and / or wireless transmission medium known in this art. Two or more of these computer subsystems can also be effectively coupled via a shared computer-readable storage medium (not shown).

[0037] Although the imaging subsystem is described above as an optical or light-based imaging subsystem, in another embodiment, the imaging subsystem is configured as an electron beam imaging subsystem. In an electron beam imaging subsystem, the energy directed to the sample contains electrons, and the energy detected from the sample contains electrons. In one such embodiment shown in FIG. 1a, the imaging subsystem includes an electron column 122, and the system includes a computer subsystem 124 coupled to the imaging subsystem. The computer subsystem 124 can be configured as described above. Furthermore, this imaging subsystem can be coupled to another or more computer systems in the same manner described above and shown in FIG. 1.

[0038] As shown in Figure 1a, the electron column includes an electron beam source 126 configured to generate electrons that are focused onto the sample 128 by one or more elements 130. The electron beam source may include, for example, a cathode source or an emitter tip, and the one or more elements 130 may include, for example, a gun lens, an anode, a beam-limiting aperture, a gate valve, a beam current selection aperture, an objective lens, and a scanning subsystem; all of these elements may include any such suitable elements known in the art.

[0039] Electrons returning from the sample (e.g., secondary electrons) can be focused onto the detector 134 by one or more elements 132. One or more elements 132 may include, for example, a scanning subsystem, which may be the same scanning subsystem included in element 130.

[0040] The electron column may include any other suitable element known in this art. Additionally, the electron column may be further configured as described in the following U.S. patents: U.S. Patent No. 8,664,594, issued April 4, 2014, to Jiang et al.; U.S. Patent No. 8,692,204, issued April 8, 2014, to Kojima et al.; U.S. Patent No. 8,698,093, issued April 15, 2014, to Gubbens et al.; and U.S. Patent No. 8,716,662, issued May 6, 2014, to MacDonald et al., all of which are incorporated herein by reference in their entirety.

[0041] Although the electron column is shown in Figure 1a as configured such that electrons are guided to the sample at one oblique angle of incidence and scattered from the sample at another oblique angle, the electron beam can be guided to and scattered from the sample at any suitable angle. Furthermore, the electron beam imaging subsystem can be configured to produce output for the sample using multiple modes (e.g., depending on different illumination angles, collection angles, etc.) as further described herein. These multiple modes of the electron beam imaging subsystem can differ in the output production parameters of any of the measurement subsystems.

[0042] Computer subsystem 124 may be coupled to detector 134, as described above. The detector detects electrons returning from the surface of the sample, thereby forming an electron beam image (or other output) of the sample. Such electron beam images may include anything suitable for electron beam imaging. Computer subsystem 124 may be configured to use the output generated by detector 134 to detect events on the sample, as can be performed as further described herein. Computer subsystem 124 may be configured to perform any additional steps described herein. A system including one of the imaging subsystems shown in Figure 1a may be further configured as described herein.

[0043] It should be noted that Figure 1a is provided herein to generally illustrate a configuration of one of the electron beam imaging subsystems that may be included in the embodiments described herein. As with the optical imaging subsystem described above, the configuration of the electron beam imaging subsystem described herein can be modified to optimize the performance of the imaging subsystem, as is typically done when designing a commercial system. Alternatively, the system described herein can be implemented using an existing system (e.g., by adding the functionality described herein to an existing system), such as tools commercially available from KLA. For some such systems, the methods described herein can be provided as optional functionality of the system (e.g., in addition to other system functionalities). Another option is that the system described herein can be designed "from scratch" to provide a completely new system.

[0044] Although the imaging subsystem is described above as a light or electron beam imaging subsystem, it can be an ion beam imaging subsystem. This imaging subsystem can be configured as shown in Figure 1a, except that the electron beam source can be replaced with any suitable ion beam source known in this art. Furthermore, the imaging subsystem can include any other suitable ion beam imaging system, such as those included in commercially available focused ion beam (FIB) systems, helium ion microscopy (HIM) systems, and secondary ion mass spectrometry (SIMS) systems.

[0045] As further described above, the imaging subsystem can be configured to have multiple modes. Generally, a "mode" is defined by the parameter values ​​of the imaging subsystem used to generate the output for the sample. Therefore, different modes can differ in the value of at least one of the imaging parameters of the imaging subsystem (except for the location on the sample to which the output is generated). For example, for a light-based imaging subsystem, different modes can use different wavelengths of light. Modes can differ in the wavelength of light directed to the sample, as further explained herein (e.g., by using different light sources, different spectral filters, etc., for different modes). In another embodiment, different modes can use different illumination channels. For example, as described above, the imaging subsystem can include more than one illumination channel. Thus, different illumination channels can be used in different modes.

[0046] Multiple modes can also differ in illumination and / or collection / detection. For example, as further explained above, the imaging subsystem may include multiple detectors. Thus, one of the detectors can be used in one mode and another of the detectors can be used in another mode. Furthermore, modes can differ from each other in more than one way as described herein (e.g., different modes may have one or more different illumination parameters and one or more different detection parameters). The imaging subsystem can be configured to scan samples using different modes in the same scan or different scans, for example, depending on the ability to scan samples using multiple modes simultaneously.

[0047] In some instances, the systems described herein can be configured as inspection systems. However, the systems described herein can be configured as another type of semiconductor-related quality control system, such as a defect re-inspection system and a measurement system. For example, embodiments of the imaging subsystems described herein and shown in Figures 1 and 1a can be modified on one or more parameters to provide different imaging capabilities depending on the application to which they will be used. In one embodiment, the imaging subsystem is configured as an electron beam defect re-inspection subsystem. For example, if the imaging subsystem shown in Figure 1a is to be used for defect re-inspection or measurement rather than inspection, it can be configured to have a higher resolution. In other words, embodiments of the imaging subsystems shown in Figures 1 and 1a illustrate certain general and various configurations of an imaging subsystem, which can be tailored in several ways that are apparent to those skilled in the art to produce imaging subsystems with different imaging capabilities more or less suitable for different applications.

[0048] As described above, the imaging subsystem can be configured to direct energy (e.g., light, electrons) to a physical version of the sample and / or to scan the energy over a physical version of the sample, thereby generating an actual image of the physical version of the sample. In this way, the imaging subsystem can be configured as an "actual" imaging system rather than a "virtual" system. However, a storage medium (not shown) and computer subsystem 36 shown in Figure 1 can be configured as a "virtual" system. Specifically, the storage medium and computer subsystem are not part of the imaging subsystem 100 and do not have any ability to handle the physical version of the sample, but can be configured to perform a virtual inspector for a type of inspection function, a virtual measurement system for a type of measurement function, a virtual defect re-inspection tool for a type of defect re-inspection function, etc., using the output of the stored detectors. Systems and methods configured as "virtual" systems are described in jointly assigned U.S. Patent No. 8,126,255, issued February 28, 2012, to Bhaskar et al., and U.S. Patent No. 9,222,895, issued December 29, 2015, to Duffy et al., and U.S. Patent No. 9,816,939, issued November 14, 2017, to Duffy et al., all of which are incorporated herein by reference in their entirety. The embodiments described herein may be further configured as described in these patents. For example, a computer subsystem described herein may be further configured as described in these patents.

[0049] The system includes a computer subsystem that may contain any of the computer subsystems described above or any configuration thereof, and one or more components that are executed by that computer subsystem. For example, as shown in Figure 1, the system may include a computer subsystem 36 and one or more components 104 that are executed by that computer subsystem.

[0050] One or more components include multiple deep learning (DL) models (also referred to herein as "initial DL models") configured to determine information about a sample based on the output generated for that sample using one or more learning modes of an imaging subsystem. For example, one or more components may include initial DL model 1 (202), initial DL model 2 (204), initial DL model 3 (206), ..., and initial DL model N (208), as shown in Figure 2. Although at least four initial DL models are shown in Figure 2, one or more components may include any suitable number of initial DL models, such as two or more initial DL models, more than four initial DL models, etc. The DL models do not model the physical imaging procedure of any of the modes. Instead, the DL models are trained to determine information based on the input image in a generative rather than a decisional manner.

[0051] Each of the DL models (including the initial DL model and the final KD DL model further described below) may have or contain any suitable DL architecture, such as a convolutional neural network (CNN) architecture. The architecture of each of the DL models may be selected as further described herein. If one or more of the DL models are or contain a CNN, then each of those CNNs may contain layers of any suitable type with any suitable configuration known in this art, such as convolutional layers, pooling layers, fully connected layers, soft max layers, etc. The architecture of any of the DL models may vary depending on the information of the samples being evaluated, as one DL architecture may be more suitable for applications such as testing rather than measurement, while another DL architecture may be more suitable for measurement rather than testing. CNNs may be trained as described herein or in any other suitable manner known in this art.

[0052] In one embodiment, multiple DL models are configured as a model ensemble. In some embodiments, each of the multiple DL models is configured to determine the same type of information about a sample. A "model ensemble" is defined herein as a group of individual DL models trained independently or jointly to solve a common task. In some applications where the embodiments described herein can be used, the common task is to predict the location of defects on a sample. A significant advantage of the embodiments described herein is that although the individual DL models in the ensemble can be trained using multimodal data or one or more learning modes, the final distillation model uses only a single best mode or a few best modes to distill all the information from the ensemble. In this way, the embodiments can use a single DL model in a supervised manner to maintain maximum performance with minimal data.

[0053] The computer subsystem can acquire or generate input learning pattern images 200 (as further described herein), which are input by the computer subsystem to the initial DL model. The input learning pattern images can be generated by the imaging subsystem and / or the computer subsystem, as further described herein.

[0054] The images input to each of the DL models can be identical. In other words, all learning pattern images can be input to each of the DL models. However, the way each of the DL models uses the learning pattern images can be different. In other words, multiple DL models do not necessarily have the same architecture configuration with the same parameters (but they can have the same architecture configuration with different parameters, different architectures, etc.). In this way, multiple DL models may not simply be multiple instances of the same DL model.

[0055] In other examples, the inputs to two or more of the DL models can be different. For instance, as further explained herein, different models can be configured to use different modes of the imaging subsystem to determine sample information. In this way, the computer subsystem can input a first portion of a learning mode image into the initial DL model 1, a second portion of a learning mode image into the initial DL model 2, and so on, and these first and second portions can be different in one or more images and / or one or more learning modes. In this way, not all of the multiple DL models can be configured to use images generated using more than one learning mode to determine sample information. In other words, even if the multiple DL models have an overall multi-mode nature, not all of the multiple DL models need to use multiple learning modes of the input images to determine sample information.

[0056] In another embodiment, one and a second of a plurality of DL models are configured to determine sample information based on outputs generated by the first and second models respectively using only one or more learning modes of the imaging subsystem. In this way, one or more of the plurality of DL models can be configured to use images generated using only a single learning mode of the imaging subsystem, and different DL models can be configured for different learning modes; that is, a first DL model for a first learning mode, a second DL model for a second learning mode, and so on. Although different DL models can be configured for use with images generated using different single learning modes, two or more DL models can be configured for use with images generated using the same single learning mode. In these examples, two or more DL models can have different architectures or the same architecture with different parameters, such that the DL models may generate different outputs based on the same single learning mode image.

[0057] When at least one of a plurality of DL models is configured for use based on image determination sample information generated using only a single learning mode, the others of the plurality of DL models can also be configured for use based on image determination sample information generated using only other single learning modes of the imaging subsystem. In other words, for each learning mode of the imaging subsystem, one of the plurality of DL models can be configured for use based solely on image determination sample information generated using that learning mode. However, when at least one of the plurality of DL models is configured for use based on image determination sample information generated using only a single learning mode of the imaging subsystem, one or more of the other plurality of DL models can be configured for use based on image determination information generated using multiple learning modes of the imaging subsystem. In this way, the plurality of DL models can include different model combinations, one or more of which are configured for use based on image determination sample information generated using only a single learning mode of the imaging subsystem, and possibly one or more of the others of which are configured for use based on image determination sample information generated using more than one learning mode of the imaging subsystem.

[0058] In another embodiment, at least one of the plurality of DL models is configured to determine sample information based on the output generated for the sample using at least two of the learning modes of one or more imaging subsystems. This embodiment may be similar to the possibilities described above, such as (1) all of the plurality of DL models are configured for all of the learning modes, (2) not all of the plurality of DL models are configured for all of the learning modes, in which case different DL models may be configured for different combinations of learning modes, (3) one or more of the plurality of DL models are configured for two or more of the same or different learning modes, while the others of the plurality of DL models are configured for only one of the learning modes, etc.

[0059] Therefore, in summary, while each of the multiple DL models can be configured to determine the same type of information about a sample (which is application-dependent, e.g., for verifying the location of predicted defects in the application, for measuring the predicted structural characteristics of the application, etc.), the multiple DL models can be configured independently of each other. In this way, each model in the set can be different in terms of architecture, parameters, and the input data used. One or more of the multiple DL models can be configured to determine information about a sample based on images generated by each of the learning modes. One or more of the multiple DL models can be configured to determine information about a sample based on images generated by fewer than all of the learning modes. One or more of the multiple DL models can be configured to determine information about a sample based on images generated using only one learning mode. In that case, there may be at least one first DL model for a first learning mode and one second DL model for a second learning mode, or at least one first DL model for a first learning mode and one second DL model for more than one learning mode. In this way, not all of the multiple DL models can be configured to determine information based on images generated using the same single learning mode of the imaging subsystem (if a first DL model is configured only for a first learning mode, then at least one other DL model can be configured only for a second learning mode different from the first or multiple learning modes). The models included in the multiple DL models and the learning modes they are configured to be used for can be further selected as described herein.

[0060] In an additional embodiment, at least two of the multiple DL models are trained independently of each other before the multiple DL models determine the information of the samples. In another embodiment, at least two of the multiple DL models are trained jointly before the multiple DL models determine the information of the samples. In yet another embodiment, the multiple DL models are trained in a supervised manner before the multiple DL models determine the information of the samples. For example, each of the initial DL models shown in FIG2 can be trained independently or jointly in a supervised manner. Some models in the initial DL models can be trained independently of each other, and other models in the initial DL models can be trained jointly.

[0061] In general, supervised training of DL models involves using labeled images of a sample or another sample of the same type. In this way, training can be performed using runtime samples or a set of samples. For example, the computer subsystem can obtain a dataset of labeled images and split it into training, testing, and validation image sets. The computer subsystem can then perform supervised training in validation mode using a CNN or other suitable DL model, taking the images and their labels as input. The labels can vary depending on the information regarding the judgments made for the sample (e.g., once a defect location is predicted, once a metric is predicted, once a defect re-detection result is predicted, etc.). Supervised training can otherwise be performed in any suitable manner known in this art. The trained model can then be used to make judgments based on sample images generated for one or more set samples.

[0062] Any of the training described above can be performed by one or more computer subsystems included in the embodiments described herein. In this way, the embodiments described herein can be configured to perform one or more setup or training functions of multiple DL models. However, any of the training described above can be performed by another method or system (not shown), which can enable the embodiments described herein to obtain a trained DL model. In this way, the embodiments described herein can be configured to train only the final KD DL model further described herein and to perform runtime functions, for example, using the trained final KD DL model to determine information on one or more runtime samples that are the same as or different from the setup samples.

[0063] In one embodiment, one or more learning modes of the imaging subsystem include the best known mode of the imaging subsystem. In another embodiment, multiple DL models are configured to have the best known architecture for determining information about a sample. In yet another embodiment, at least two of the multiple DL models are configured to have different architectures. For example, the embodiments described herein can be configured to train individual DL models using the best architecture and the best single-mode or multi-mode data. The best architecture and single-mode or multi-mode data can be found by performing an exhaustive grid search or neural network architecture optimization. This search and / or optimization can result in the selection of a combination of the multiple DL models described above. The top N architectures and single-mode or multi-mode data that provide the best performance can be selected as individual models in a model set. After selecting individual models, they can be trained independently or jointly. Jointly training N individual models can require significant computational memory. With respect to the embodiments described above, even if two or more of the multiple DL models have different architectures and / or are for different learning modes, they can still be configured to determine the same type of information about a sample.

[0064] Regardless of whether any of the multiple DL models is configured to determine sample information based on images generated from the sample using only one or more learning modes, the input to the DL model may not only include images of the sample generated by the imaging subsystem. For example, the input to any of the DL models may include additional information such as one or more reference images corresponding to one or more sample images, one or more difference images generated by subtracting a reference sample image from a test sample image, sample design information, and the like. In this way, for any mode and any DL model, there can be multiple channels for inputting from one mode to another DL model.

[0065] One or more components may also include a knowledge distillation (KD) component configured to combine the outputs of multiple DL models. For example, as shown in Figure 2, one or more components may include a KD component 218. The KD component may have any architectural configuration and parameters suitable for performing the functions further described herein.

[0066] In some embodiments, the output generated by multiple DL models and combined by the KD component includes logits generated by at least two of the multiple DL models. For example, the output of the multiple DL models may include an individual logit file for each DL model. For example, as shown in Figure 2, each of the initial DL models may produce a logit result. Specifically, initial DL model 1 may produce logit 1 (210), initial DL model 2 may produce logit 2 (212), initial DL model 3 may produce logit 3 (214), and initial DL model N may produce logit N (216). Each of these logit results and / or files may be combined with any other information generated or output by the initial DL models and may be input to the KD component 218.

[0067] While a "rogitt" can have different meanings in different contexts, as used herein and interchangeably with the terms "rogitt result" and "rogitt file," a "rogitt" is generally defined as a model output that responds to the probability that an outcome is correct. In this way, a "rogitt" (as used in the context of that term) can encompass any output of any layer of a DL model described herein, representing the model's confidence in an outcome. Therefore, a "rogitt," "rogitt file," or "rogitt result" can be considered a measure of the goodness of a DL model's outcome. In some instances, for DL ​​purposes, a layer of a DL model fed into a logit function is often called a "rogitt layer," i.e., a layer fed into a softmax function or other regularization layer. The output value of a logitt layer is often referred to as a "rogitt," a "rogitt file," or a "rogitt result." The logit layer can produce values ​​from negative infinity to positive infinity, i.e., a regression output, and the softmax or normalization layer can transform those values ​​into values ​​from 0 to 1, i.e., a predicted probability. The output of the softmax layer is the probability of any outcome produced by the DL model, such as a classification. Therefore, although certain embodiments of "logit" are described herein, the output of a combination of KD components can include any output of a DL model that responds to a "good" level of the model's output.

[0068] In another embodiment, combining the outputs generated by multiple DL models includes generating an average Roger from Rogers generated by at least two of the multiple DL models. In yet another embodiment, combining the outputs generated by multiple DL models suppresses any unimportant or incorrect portions of the information determined by the multiple DL models. For example, a KD component may average individual Roger outputs into a single Roger output. In one such example, KD component 218 may output an average Roger 220 generated from Roger 1 (210), Roger 2 (212), Roger 3 (214), ... Roger N (216). Although it is most common to combine all of the Rogers generated by all of the multiple DL models, this is unnecessary and less than all of the Rogers generated by the multiple DL models may be combined. For example, if one of the Rogers clearly indicates that one of the multiple DL models is to be performed in a manner substantially different from each of the other models, that Roger may be discarded and the remaining Rogers may then be combined. However, averaging all logities essentially minimizes the performance of any anomalous DL models, thereby making any pre-combination logitie check step unnecessary.

[0069] Averaging the Logit output helps compress the most important information from individual DL models while suppressing unimportant information. Therefore, this operation helps retain the majority of the most important signals corresponding to the defect location (in the case of inspection) while suppressing false alarms. For example, as further explained herein, multiple DL models can be best-known models with the best known parameters for determining sample information. Therefore, generally, consensus can be reached among multiple DL models because at least most of them can determine the same information about the sample, such as predicting the presence of a defect at a sample location. However, with any procedure similar to that described herein, even when optimized for the sample and imaging subsystem, misjudgments, incorrect results, or other errors can occur. For example, even the best inspection procedure will detect some perturbations. In this way, even if multiple DL models include best-known models, all models will not produce accurate sample information predictions for all sample locations. Therefore, by combining the logit as described above, the accurate predictions that would normally be generated by the majority of multiple DL models will be enhanced, while the inaccurate predictions that would normally be generated by the minority of models will be suppressed.

[0070] In a specific but not limiting instance, if the predicted sample information is a predicted defect location, then most of the multiple DL models can accurately predict a defect location, while one or more of the multiple DL models cannot predict that defect location. Therefore, the consensus among multiple DL models that the sample location is a defect location will be clarified by combining the steps described above. The opposite situation can also occur. If only one of the multiple DL models predicts a sample location as a defect location, that predicted defect location will most likely be a disturbance or an incorrectly predicted defect location, and this will be suppressed by combining the steps described herein.

[0071] While averaging the outputs of multiple deep learning models is the easiest way to compress the most important information and suppress the less important information from the individual deep learning models, any other function that performs the same operation in a similar manner can be used instead of averaging. For example, combining these outputs could include determining the median of the outputs, performing principal component analysis (PCA) on the outputs, performing linear discriminant analysis (LDA) on the outputs, and so on.

[0072] In this way, the combined output of multiple DL models can more accurately reflect the actual sample information compared to the output of individual models. Therefore, using the combined output to train one of the final KD DL models as described in this paper can yield a better-performing DL model, even when that DL model determines sample information based on images generated from the sample using only a single mode or one or more runtime modes of the imaging subsystem.

[0073] One or more components further include a final knowledge distillation DL model configured to determine information about a sample or an additional sample based on outputs generated using one or more runtime modes of the imaging subsystem for that sample or an additional sample. For example, one or more components may include a final KD DL model 224, as shown in Figure 2, which uses one or more input runtime mode images 226 to produce an output containing determined information 228. The final KD DL model may have any suitable configuration known in this art, such as a CNN configuration. Regarding the initial DL models described above, the final KD DL model may have an architecture different from or the same as one or more of the initial DL models, and may have one or more different parameters. The architecture of the final KD DL model may be determined empirically or in any other suitable manner.

[0074] The final KD DL model can determine information from a sample that is the same as the initial DL model or from a different sample. In this way, the runtime sample from which the final KD DL model determines its information may or may not be the same as the setup sample from which the initial DL model determines its information. If the runtime sample and the setup sample are different, they can be of the same type. For example, the runtime sample and the setup sample may have the same design and may be processed in the same way before imaging. However, if the runtime sample and the setup sample are similar enough that the same model can be used to determine the information of the sample, then these samples do not need to have the same design and / or be processed in the same way before imaging. Furthermore, although the final KD DL model can be described herein as configured for determining the information of a sample, after the final KD DL model is trained as further explained herein, it can be used to determine the information of a desired number of runtime samples or setup samples.

[0075] Therefore, the embodiments described herein can utilize one or more learning pattern data for KD on one or more runtime patterns or even just a single pattern. For example, preliminary results have shown inventors that using multi-pattern input data achieves better performance compared to using single-pattern input. However, one disadvantage of using multi-pattern data is that it adds considerable computation time during inference, including data flow time and DL model computation time. Furthermore, further experiments have shown that using a set of DL models configured to solve a common task provides enhanced performance compared to individual DL models. However, using a set of DL models trained on multi-pattern input data can increase computation time and unreasonable memory requirements. The embodiments described herein address this problem by attempting to incorporate all the information contained in the initial DL set into a single final KD DL model. Therefore, the embodiments will have a significant advantage because they will significantly improve current inference performance without adding any additional computation time. Furthermore, data limitations may exist, where certain input patterns may be unavailable across the entire sample due to issues during the data collection phase. This can cause problems when attempting to train the currently used DL model. The embodiments described herein address this issue because they only require knowledge learned from previous individual DL models and do not require the input data itself.

[0076] Prior to the final KD DL model decision information, the KD component is configured for supervised training of the final KD DL model using combined inputs. For example, as shown in Figure 2, the KD component 218 can perform supervised training 222 on the final KD DL model 224 using the combined output of the initial DL model (shown as averaged Roger 220 in this embodiment). In this way, the embodiments described herein can use KD to train a final KD DL model (e.g., a single CNN) using one or more runtime pattern data or single-pattern data, leveraging information from multiple DL models (e.g., an ensemble of CNNs) trained using learning pattern data. Supervised training of the final KD DL model may involve using the imaging subsystem output generated using one or more top-level runtime patterns and the combined output of multiple DL models (e.g., averaged Roger output) as inputs to the final KD DL model and training the final KD DL model using KD via supervised training.

[0077] KD can generally be defined as a procedure that uses past information from other DL models (or teacher models) to improve the performance of a single model (or student model). In the embodiments described herein, information learned from various DL models using learning patterns is distilled and used as input to a final KD DL model using one or more runtime patterns or even one single pattern as input. This training helps to significantly improve performance and reduce computation time during inference. For example, the embodiments described herein may be able to use substantially less training data compared to training other supervised models. Because the embodiments described herein can perform training with only a minimal amount of data compared to the amount of data required to train a comparable supervised model, the embodiments described herein can reduce computation time during inference.

[0078] Supervised KD training in machine learning (ML) uses annotated data (e.g., in the testing case, annotating defect locations with ground truth) and information learned from past DL models to train a new, improved branch of the DL model. In the embodiments described herein, different combinations of DL / ML models are used to learn different features that can be used as prior information to train the final KD DL model. This helps the final KD DL model learn more information about the samples (such as defect patterns), thereby further improving performance during inference.

[0079] In one embodiment, one or more runtime modes of the imaging subsystem are selected from one or more learning modes based on information from sample determinations by multiple DL models. In this way, the embodiments described herein can be configured for optical (or other) mode optimization for defect detection (or measurement, defect re-detection, etc.) via KD. For example, the final KD DL model can be configured to determine sample information using the output of the imaging subsystem generated by the topmost single mode or modes. The topmost single mode or modes of the imaging subsystem can be based on prior knowledge of past imaging subsystem performance. However, the topmost single mode or modes can also be determined based on the outputs generated by multiple DL models. For example, by comparing sample information determined by two or more of the multiple DL models, multiple DL models or modes that generate optimal sample information can be determined. Then, one or more learning modes used with those multiple DL models can be identified as optimal based on the information generated by those multiple DL models. In another instance, the best-performing DL model among multiple DL models can be identified by comparing sample information determined by two or more of the multiple DL models, and any common learning patterns among the DL models can be examined to select a single pattern or several patterns for use at runtime. Alternatively, a greedy search algorithm or another suitable method or algorithm can be used to find the upper-level pattern based on information generated by the multiple DL models and / or any other available information about the learning patterns of the imaging subsystem and their effectiveness in determining sample information.

[0080] In this manner, one or more modes included in one or more runtime modes may be a subgroup of learning modes used to generate inputs for multiple DL models, or one or more different modes not in the original input and / or specified by the application or a user. Additionally, one or more input runtime mode images 226 may be a subgroup of the initial learning modes, a group of modes different from the initial mode, and / or a combination of old and new modes.

[0081] In another embodiment, during the execution of a procedure on a sample or additional samples to utilize information from the final KD DL model, the imaging subsystem generates the sample output using only one or more runtime modes of the imaging subsystem, and the computer subsystem inputs the output generated using only one or more runtime modes into the final KD DL model alone. In this way, during runtime, the imaging subsystem can generate the sample output (e.g., an image) using only a single mode or several modes of the imaging subsystem (e.g., by scanning the sample using only one mode or only several runtime modes). Therefore, unlike some previously used DL models that require multiple input modes to achieve relatively high performance, the embodiments described herein can utilize all multi-mode information during setup and distill it into a single-mode (or a limited number of modes) final KD DL model for runtime use. In other words, once the learning mode data or several learning mode data has been input into multiple DL models to generate outputs for training the final KD DL model, the procedure (e.g., inspection, measurement, defect re-detection) no longer requires the learning mode or several learning modes. Therefore, the multiple DL models described herein are not used during runtime. Instead, during runtime, samples are scanned using only one or more runtime modes, and their outputs are the sole input to the final KD model.

[0082] In this manner, the embodiments described herein provide the ability to replace one set of many individual DL models with a single DL model by distilling features from learning pattern data or several learning pattern data into a single pattern or one or more runtime patterns. Therefore, the embodiments described herein also provide a systematic method for reducing the number of patterns required in applications (such as defect / reduced mask inspection) (for inspection or another application described herein) while providing similar or better detection sensitivity compared to multi-pattern procedures. This method can be applied to the optical inspection (and other) tools described herein to increase tool throughput and reduce tool ownership costs, especially for multi-pattern inspection and other procedures. Therefore, one advantage of the embodiments described herein is that they can significantly improve performance during inference while reducing computation time. Furthermore, the embodiments described herein use a single knowledge-distilled DL model instead of an individual set of DL models during runtime, which saves significant inference computation time while maintaining substantially higher performance.

[0083] In some embodiments, the information determined by the final KD DL model for a sample or additional samples includes the predicted defect location on the sample. In this way, the embodiments described herein can predict the location of a defect on a BBP or other image using a DL-based CNN or another DL model by utilizing information learned using an ensemble of well-trained individual CNNs or other DL models. In other words, the embodiments described herein distill key features from one or more learning patterns into one or more runtime patterns or a single pattern for determining sample information, such as predicting defect locations. One advantage of the embodiments described herein is that they can predict defect locations with a higher signal-to-noise ratio (SNR) than noise suppression algorithms and other supervised CNN models. For example, using an ensemble of models as described herein to generate training data for the final KD DL model can reduce prediction discrepancies and improve accuracy (thereby providing a better SNR for the predicted defect location in validation applications by better suppressing noise signals while maintaining the defect signal than other methods).

[0084] The predicted defect location determined by the final KD DL model can be identified in an inspection procedure where a relatively large area on the sample is scanned by an imaging subsystem, and the resulting image is then examined to identify potential defects. In addition to the predicted defect location, the final KD DL model can also be configured to determine other information about the predicted defect location, such as defect classification and possible defect attributes. Generally, through supervised training as described herein and using an appropriate architecture, the final KD DL model can be configured to generate one or more classes of inspection results for a sample. Therefore, essentially, the final KD DL model can have multiple output channels, each for a different type of information. The outputs from these multiple channels can then be combined into a single inspection result file for the sample (e.g., a KLARF file generated by some KLA inspection tool). In this way, multiple types of information can exist in the inspection result file for any location on the sample.

[0085] In a similar manner, this procedure can be a defect re-detection procedure. Unlike an inspection procedure, a defect re-detection procedure typically revisits discrete locations on a sample where a defect has been detected. An imaging subsystem configured for defect re-detection can generate sample images that can be input into the final KD DL model, as described herein. The final KD DL model can be trained and configured to determine whether a defect actually exists at the location of a defect identified by inspection and to determine one or more attributes of the defect, such as a defect shape, size, roughness, background pattern information, etc., and / or to determine a defect classification (e.g., a bridging defect, a missing feature defect, etc.). The final KD DL model can be trained and configured in other ways as described above.

[0086] As described above, in some embodiments, the imaging subsystem can be configured for the measurement of a sample. In one such embodiment, decision information includes determining one or more characteristics of a sample structure in an input image. For example, the DL model described herein can be trained using an image labeled with metric information. The metric information can include any metric information of interest, which can vary depending on the structure on the sample. Examples of this metric information include, but are not limited to, critical dimensions (CD), such as the linewidth of the sample structure and other dimensions. Once the final KD DL model has been trained as described herein, it can be used to predict metric information based on an unlabeled (test) sample image. The unlabeled sample image can include any image generated by any metric tool that can have a configuration (such as the configuration described herein or any other suitable configuration known in this art). In this way, the embodiments described herein can advantageously use a sample image generated by a metric tool to predict metric information of the sample and any one or more sample structures contained in the input image.

[0087] The computer subsystem can also be configured to generate results containing determined information, which may include any of the results or information described herein. The results of the determined information can be generated by the computer subsystem in any suitable manner. All embodiments described herein can be configured to store the results of one or more steps of the embodiments in a computer-readable storage medium. Such results may include any of the results described herein and may be stored in any manner known in the art. Results containing determined information may have any suitable form or format, such as a standard file type. The storage medium may include any storage medium described herein or any other suitable storage medium known in the art.

[0088] After the results have been stored, they can be accessed on storage media and used by any of the methods or system embodiments described herein, formatted for display to a user, used by another software module, method, or system to perform one or more functions on the sample or another sample of the same type. For example, the results generated by the computer subsystem described herein may include information on any defects detected on the sample (such as location), the bounding box of the detected defects, detection scores, information on defect classification (such as category labels or IDs), any defect attributes determined based on any of the images, predicted sample structural measurements, dimensions, shape, or any such suitable information known in the art. This information may be used by the computer subsystem or another system or method to perform additional functions on the sample and / or the detected defects, such as sampling the defects for defect re-inspection or other analysis, determining one of the root causes of the defects, etc.

[0089] Such functionality also includes, but is not limited to, modifying a procedure (such as a manufacturing procedure) or step that has been or will be performed on a sample in a feedback or feedforward manner. For example, the computer subsystem can be configured to determine one or more changes to a procedure that has been performed on the sample and / or will be performed on the sample based on determination information. Changes to the procedure may include any suitable changes to one or more parameters of the procedure. In one such example, the computer subsystem preferably determines such changes to reduce or prevent defects on other samples to which a modified procedure is performed, to correct or eliminate defects on the sample when another procedure is performed on the sample, or to compensate for defects when another procedure is performed on the sample. The computer subsystem can determine such changes in any suitable manner known in this art.

[0090] These changes can then be sent to a semiconductor fabrication system (not shown) or to a storage medium (not shown) accessible by both the computer subsystem and the semiconductor fabrication system. The semiconductor fabrication system may or may not be part of the system embodiments described herein. For example, the imaging subsystem and / or computer subsystem described herein may be coupled to the semiconductor fabrication system via one or more common elements (such as a housing, a power supply, a sample handling device or mechanism, etc.). The semiconductor fabrication system may include any semiconductor fabrication system known in the art, such as a lithography tool, an etching tool, a chemical mechanical polishing (CMP) tool, a deposition tool, and the like.

[0091] The embodiments described herein have several advantages in addition to those already stated. For example, the embodiments described herein provide a way to distill single or multiple learning pattern data into one or more runtime patterns or even a single pattern while maintaining or improving sensitivity. In other words, any combination of learning pattern data can be used as input when training individual models in the ensemble, and the final KD model only requires one pattern (e.g., the topmost single pattern), although more than one pattern can be used at runtime. Therefore, the method described herein can distill multi-pattern training data into only a single pattern. Furthermore, the embodiments described herein can utilize information learned by other supervised models to further improve sensitivity. In other words, the embodiments described herein provide a way to improve the performance of a different DL model using an ensemble of models. The embodiments described herein can also improve performance during runtime inference. Moreover, the embodiments described herein can reduce computation time during runtime inference without significantly sacrificing performance processing power. The embodiments described herein can also provide significantly higher SNR based on decision information (such as predicted defect locations). In another example, the embodiment may have a relatively stable sensitivity to changes in the procedure, such as changes on the sample due to changes in the procedure performed on the sample, changes in the sample image input to the final KD DL model due to changes in the procedure performed on the sample, and / or changes in the imaging procedure, etc.

[0092] Each of the embodiments described above can be combined to form a single embodiment. In other words, unless otherwise stated herein, all embodiments are not mutually exclusive with any other embodiments.

[0093] Another embodiment relates to a computer-implemented method for determining information about a sample. The method includes determining information about the sample by inputting outputs generated for a sample using one or more learning modes of an imaging subsystem into multiple deep learning (DL) models, and combining the outputs generated by the multiple DL models. The method also includes performing supervised training on a final KD DL model using the combined outputs and determining information about the sample or the additional sample by inputting outputs generated for the sample or an additional sample using one or more runtime modes of the imaging subsystem into the final KD DL model. The determination of the sample information, the combination of the outputs, the performance of supervised training, and the determination of the sample or the additional sample information are performed by a computer subsystem that can be configured according to any of the embodiments described herein.

[0094] Each step of the method may be performed as further described herein. The method may also include any other steps that may be performed by the imaging subsystem and / or computer subsystem described herein. Furthermore, the method described above may be performed by any of the system embodiments described herein.

[0095] An additional embodiment relates to a non-transitory computer-readable medium storing program instructions executable on a computer system to perform a computer-implemented method for determining information of a sample. One such embodiment is illustrated in Figure 3. Specifically, as shown in Figure 3, the non-transitory computer-readable medium 300 includes program instructions 302 executable on a computer system 304. The computer-implemented method may include any steps of any method set forth herein.

[0096] Program instructions 302 for implementing methods such as those described herein may be stored on a computer-readable medium 300. The computer-readable medium may be a storage medium, such as a magnetic disk or optical disc, a magnetic tape, or any other suitable non-transitory computer-readable medium known in the art.

[0097] Program instructions can be implemented in any of the following ways, including process-based techniques, component-based techniques and / or object-oriented techniques, and other techniques. For example, depending on the need, ActiveX controls, C++ objects, JavaBeans, Microsoft Foundation Classes ("MFC"), SSE (Streaming SIMD Extensions), Python, Tensorflow, or other technologies or methods can be used to implement these program instructions.

[0098] Computer system 304 can be configured according to any of the embodiments described herein.

[0099] In view of this description, those skilled in the art will understand various modifications and alternative embodiments of the present invention. For example, a method and system for determining information about a sample are provided. Therefore, this description should be considered illustrative only and is intended to teach those skilled in the art the general manner in which the present invention is performed. It should be understood that the forms of the invention shown and described herein should be considered as presently preferred embodiments. As will be apparent to those skilled in the art after benefiting from this description, the elements and materials illustrated and described herein can be substituted, the components and procedures can be reversed, and the specific properties of the invention can be utilized independently. Changes can be made to the elements described herein without departing from the spirit and scope of the invention as set forth in the following claims.

[0100] 10: System 14: Sample 16: Light source / source 18: Optical Components 20: Lens 22: Platform 24:Light collector 26: Components 28: Detector 30:Light collector 32: Components 34: Detector 36: Computer Subsystem 100: Imaging Subsystem 102: Computer System 104: Components 122: Electron column 124: Computer Subsystem 126: Electron Beam Source 128: Sample 130: Components 132: Components 134: Detector 200: Input learning mode image 202: Initial Deep Learning Model 1 204: Initial Deep Learning Model 2 206: Initial Deep Learning Model 3 208: Initial deep learning model N 210: Roger 1 212: Roger 2 214: Roger 3 216: Roger N 218: Knowledge Distillation Component 220: Average Roger 222: Supervised Training 224: Final Knowledge Distillation Deep Learning Model 226: Input runtime mode image 228: Information as determined 300: Non-transitory computer-readable media / Computer-readable media 302: Program Instruction 304: Computer System

Claims

1. A system configured for determining information about a sample, comprising: A computer subsystem; And one or more components executed by the computer subsystem, wherein the one or more components include: a plurality of deep learning models configured to determine information about a sample based on outputs generated for a sample using two or more learning modes of an imaging subsystem, wherein two or more of the plurality of deep learning models are further configured to determine the information based on outputs generated for the sample using different combinations of the two or more learning modes of the imaging subsystem; a knowledge distillation component configured to combine the outputs generated by the plurality of deep learning models; and a final knowledge distillation deep learning model configured to determine information about the sample or the additional sample based on outputs generated for the sample or the additional sample using one or more runtime modes of the imaging subsystem, wherein before the final knowledge distillation deep learning model determines the information, the knowledge distillation component is further configured to perform supervised training on the final knowledge distillation deep learning model using the combined outputs.

2. The system of request item 1, wherein the plurality of deep learning models are further configured as a model set.

3. The system of claim 1, wherein each of the plurality of deep learning models is further configured to determine the same type of information for the sample.

4. The system of claim 1, wherein one of the plurality of deep learning models, a first and a second, are further configured to determine information about the sample based on the output generated by the first and a second of the two or more learning modes of the imaging subsystem for the sample.

5. The system of claim 1, wherein at least one of the plurality of deep learning models is further configured to determine information about the sample based on the output generated for the sample using at least two of the two or more learning modes of the imaging subsystem.

6. The system of claim 1, wherein at least two of the plurality of deep learning models are trained independently of each other before the plurality of deep learning models determine the information of the sample.

7. The system of claim 1, wherein at least two of the plurality of deep learning models are jointly trained before the plurality of deep learning models determine the information of the sample.

8. The system of request item 1, wherein the plurality of deep learning models are trained in a supervised manner before the plurality of deep learning models determine the information of the sample.

9. The system of claim 1, wherein the two or more learning modes of the imaging subsystem include the best known mode of the imaging subsystem.

10. The system of claim 1, wherein the plurality of deep learning models are further configured to have the best known architecture in order to determine the information of the sample.

11. The system of claim 1, wherein at least two of the plurality of deep learning models are further configured to have different architectures.

12. The system of claim 1, wherein the output generated by the plurality of deep learning models and combined by the knowledge distillation component includes logit generated by at least two of the plurality of deep learning models.

13. The system of claim 1, wherein the output generated by combining the plurality of deep learning models includes an average logit generated by at least two of the plurality of deep learning models.

14. The system of claim 1, wherein combining the outputs generated by the plurality of deep learning models suppresses a portion of the information determined by a component less than a major component of the plurality of deep learning models.

15. The system of claim 1, wherein the one or more runtime modes of the imaging subsystem are selected from two or more learning modes based on the information determined by the multiple deep learning models for the sample.

16. The system of claim 1, wherein during the execution of a procedure for determining the information using the final knowledge distillation deep learning model on the sample or the additional sample, the imaging subsystem generates the output of the sample using only the one or more runtime modes of the imaging subsystem, and the computer subsystem inputs the output generated using only the one or more runtime modes into only the final knowledge distillation deep learning model.

17. The system of claim 1, wherein the information determined by the final knowledge distillation deep learning model for the sample or the additional sample includes the predicted defect location on the sample.

18. The system of claim 1, wherein the imaging subsystem is configured as a light-based imaging subsystem.

19. A non-transitory computer-readable medium storing program instructions executable on a computer system for performing a computer-implemented method for determining information of a sample, wherein the computer-implemented method includes: The method involves determining information about a sample by inputting the output generated for a sample using two or more learning modes of an imaging subsystem into multiple deep learning models, wherein two or more of the multiple deep learning models are further configured to determine the information based on the output generated for the sample using different combinations of the two or more learning modes of the imaging subsystem; combining the outputs generated by the multiple deep learning models; performing supervised training on a final knowledge distillation deep learning model using the combined outputs; and determining information about the sample or the additional sample by inputting the output generated for the sample or an additional sample using one or more runtime modes of the imaging subsystem into the final knowledge distillation deep learning model.

20. A computer-implemented method for determining information of a sample, comprising: The method involves determining information about a sample by inputting the output generated for a sample using two or more learning modes of an imaging subsystem into multiple deep learning models, wherein two or more of the multiple deep learning models are further configured to determine the information based on the output generated for the sample using different combinations of the two or more learning modes of the imaging subsystem; combining the outputs generated by the multiple deep learning models; performing supervised training on a final knowledge distillation deep learning model using the combined output; and determining information about the sample or the additional sample by inputting the output generated for the sample or the additional sample using one or more runtime modes of the imaging subsystem into the final knowledge distillation deep learning model, wherein determining the information about the sample, combining the outputs, performing supervised training, and determining the information about the sample or the additional sample are performed by a computer subsystem.

Images