Machine learning based defect inspection for semiconductor samples

Abstract

A system and method for defect inspection of a semiconductor sample is provided. The method comprises the following steps: acquiring a runtime image of the semiconductor sample; generating a reference image based on the runtime image using a machine learning (ML) model; and performing defect inspection on the runtime image using the generated reference image. The ML model is previously alternately trained between two training modes using a training set: a random mode in which the ML model is configured to generate a prediction reference image having a random PV from a pattern change (PV) distribution; and a deterministic mode in which the ML model is configured to generate a prediction reference image having a predetermined PV selected from the PV distribution, the ML model learning the PV distribution based on the PV observed in the training set.

Description

Technical Field

[0001] The subject matter of the present disclosure relates generally to the field of inspection of semiconductor specimens, and more particularly to the application of machine learning-based defect inspection of the specimens. Background Art

[0002] The current demand for high density and high performance associated with ultra-large scale integration of manufactured devices requires sub-micron features, and requires increasing transistor and circuit speeds, as well as enhanced reliability. As semiconductor processes advance, pattern dimensions such as line widths and other types of critical dimensions continue to shrink. This demand requires the formation of device feature structures with high precision and high uniformity, which in turn requires careful monitoring of the manufacturing process, including automated inspection of the device while it is still in the form of a semiconductor wafer.

[0003] Runtime inspection can generally employ a two-stage process, e.g., inspecting a sample and then reviewing the sampled locations for potential defects. Inspection generally involves generating some output (e.g., image, signal, etc.) of the sample by directing light or electrons to a wafer and detecting the light or electrons from the wafer. In the first stage, the surface of the sample is inspected at high speed and relatively low resolution. Defect detection is typically performed by applying a defect detection algorithm to the inspection output. A defect map is generated to show suspected locations on the sample that have a high probability of defects. In the second stage, at least some of the suspected locations are more thoroughly analyzed at a relatively high resolution to determine different parameters of the defect, such as class, thickness, roughness, size, etc.

[0004] During or after manufacturing the sample to be inspected, the inspection can be performed by using a non-destructive inspection tool. As non-limiting examples, various non-destructive inspection tools include scanning electron microscopes, atomic force microscopes, optical inspection tools, and the like.

[0005] The inspection process may include multiple inspection steps. The manufacturing process of the semiconductor device may include various processes, such as etching, deposition, planarization, growth such as epitaxial growth, implantation, etc. The inspection step may be performed multiple times, for example after certain process steps and / or after manufacturing certain layers, etc. Additionally or alternatively, each inspection step may be repeated multiple times, for example for different wafer positions, or for the same wafer position with different inspection settings.

[0006] Inspection processes are used at various steps during semiconductor manufacturing to detect and classify defects on samples, as well as to perform metrology-related operations. The effectiveness of inspection can be improved by automating one or more processes such as, for example, defect detection, automatic defect classification (ADC), automatic defect review (ADR), image segmentation, automatic metrology-related operations, etc.

[0007] Automated inspection systems ensure that manufactured parts meet expected quality standards and provide useful information regarding adjustments that may be needed to manufacturing tools, equipment and / or components, depending on the type of defects identified.

[0008] In some cases, machine learning techniques can be used to assist in automated inspection processes to promote increased throughput. For example, supervised machine learning can be used to provide accurate and efficient solutions for automating specific inspection applications based on well-annotated training images. Summary of the invention

[0009] According to certain aspects of the subject matter of the present disclosure, there is provided a computerized system for performing defect inspection on a semiconductor sample, the system comprising a processing circuit system configured to: obtain a runtime image of the semiconductor sample; generate a reference image based on the runtime image using a machine learning (ML) model, wherein the ML model was previously trained using a training set alternating between two training modes: a random mode, wherein the ML model is configured to generate a predicted reference image having a random PV from a pattern variation (PV) distribution, and a deterministic mode, wherein the ML model is configured to generate a predicted reference image having a predetermined PV selected from the PV distribution, the ML model learning the PV distribution based on the PVs observed in the training set; and perform defect inspection on the runtime image using the generated reference image.

[0010] In addition to the above features, the system according to the aspects of the subject matter of the present disclosure may include one or more of the features (i) to (xv) listed below in any desired combination or arrangement that is technically possible:

[0011] (i) The training set includes one or more pairs of training images, each pair including a defective image as an input image and a corresponding nominal image as a target image, or including two nominal images with different PVs as an input image and a target image, respectively.

[0012] (ii). The training comprises, for a given pair of training images: in a random mode, the ML model generates a predicted reference image with a random PV from a PV distribution based on the input image, and optimizes the ML model based on the predicted reference image relative to the target image; and in a deterministic mode, the ML model generates a predicted reference image with a predetermined PV selected from the PV distribution based on the target image, and optimizes the ML model based on the predicted reference image relative to the target image.

[0013] (iii) One or more pairs include at least one pair of defective images and nominal images corresponding to the defective images. The training includes: in a random mode, processing the defective images by the ML model to obtain a predicted reference image with a random PV from the PV distribution, and optimizing the ML model to reduce the difference between the predicted reference image and the nominal image; and in a deterministic mode, processing the nominal image by the ML model to obtain a predicted reference image with a predetermined PV selected from the PV distribution and similar to the PV of the nominal image, and optimizing the ML model to reduce the difference between the predicted reference image and the nominal image.

[0014] (iv). The defective image is a synthetic defective image generated by injecting defects into the corresponding nominal image, and the synthetic defective image is associated with the ground truth of the injected defects.

[0015] (v) The ML model includes an encoder, a first decoder, and a second decoder connected to the encoder and the first encoder. A predicted reference image is obtained as an output of the first encoder. The training further includes: obtaining a segmentation map including segments indicating the presence of defects as an output of the second decoder, and optimizing the ML model based on the segmentation map relative to a benchmark truth.

[0016] (vi) One or more pairs include at least one pair of a first nominal image and a second nominal image with different PVs. The training includes: in a random mode, processing the first nominal image by the ML model to obtain a predicted reference image with a random PV from the PV distribution, and optimizing the ML model to reduce the difference between the predicted reference image and the second nominal image; and in a deterministic mode, processing at least one of the first nominal image or the second nominal image by the ML model to generate at least one predicted reference image with a predetermined PV selected from the PV distribution and similar to the PV of the at least one nominal image, and optimizing the ML model to reduce the difference between the at least one predicted reference image and the at least one nominal image.

[0017] (vii). The ML model is a probabilistic generative model.

[0018] (viii). A random PV is randomly selected from a PV distribution, and a predetermined PV is selected as the center of the PV distribution so as to be similar to a PV in a corresponding input image of the ML model.

[0019] (ix). The training in random mode enables the ML model to learn to identify all PVs within the PV distribution as normal and distinguish defective features therefrom, and the training in deterministic mode enables the ML model to learn to generate reference images with PVs similar to those of the input image.

[0020] (x) The reference image shares a PV similar to the PV of the runtime image. The processing circuit system is configured to perform defect inspection by directly comparing the runtime image with the reference image to obtain a defect inspection result indicating a defect distribution on the semiconductor sample. The defect inspection result reduces residual PV and improves detection sensitivity.

[0021] (xi) The processing circuit system is configured to: use the ML model to repeat the generation of the reference image multiple times to obtain multiple reference images with random PVs; and generate a composite reference image based on the multiple reference images so that the composite reference image has a PV similar to the PV of the runtime image. Use the composite reference image to perform defect inspection on the runtime image.

[0022] (xii). A composite image is generated by selecting the minimum value of the corresponding pixel value from multiple reference images for each pixel.

[0023] (xiii). The processing circuit system is further configured to use the ML model to generate a segmentation map for a runtime image, the segmentation map including segments indicating the presence of defects.

[0024] (xiv) The ML model includes an encoder, a first decoder, and a second decoder connected to the encoder and the first encoder. A reference image is obtained as an output of the first encoder, and a segmentation map is obtained as an output of the second decoder.

[0025] (xv). The one or more image pairs further include at least one pair of a nominal image and a synthetic defective image, wherein the synthetic defective image is generated by injecting defects into the nominal image.

[0026] According to other aspects of the subject matter of the present disclosure, a computerized method for defect inspection of a semiconductor sample is provided, the method comprising: obtaining a runtime image of the semiconductor sample; generating a reference image based on the runtime image using a machine learning (ML) model, wherein the ML model was previously trained using a training set alternating between two training modes: a random mode, wherein the ML model is configured to generate a predicted reference image having a random PV from a pattern variation (PV) distribution, and a deterministic mode, wherein the ML model is configured to generate a predicted reference image having a predetermined PV selected from the PV distribution, the ML model learning the PV distribution based on the PVs observed in the training set; and performing defect inspection on the runtime image using the generated reference image.

[0027] According to other aspects of the subject matter of the present disclosure, there is provided a computerized method for training a machine learning (ML) model that can be used for defect inspection of semiconductor samples, the method comprising: obtaining a training set comprising one or more pairs of training images, each pair comprising an input image and a target image corresponding to the input image; and using the training set to train the ML model alternately between two training modes, the training comprising, for a given pair of training images: in a random mode, generating by the ML model a predicted reference image having a random PV from a PV distribution based on the input image, and optimizing the ML model based on the predicted reference image relative to the target image, wherein the PV distribution is derived based on variations observed in the training set; and in a deterministic mode, generating by the ML model a predicted reference image having a predetermined PV selected from the PV distribution based on the target image, and optimizing the ML model based on the predicted reference image relative to the target image itself.

[0028] These aspects of the subject matter of the present disclosure may comprise, mutatis mutandis, one or more of the features (i) to (xv) listed above with respect to the system, in any desired combination or arrangement technically possible.

[0029] According to other aspects of the subject matter of the present disclosure, a non-transitory computer-readable medium is provided, the non-transitory computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform a computerized method for defect inspection on a semiconductor sample, the method comprising: obtaining a runtime image of the semiconductor sample; generating a reference image based on the runtime image using a machine learning (ML) model, wherein the ML model was previously trained using a training set alternating between two training modes: a random mode, wherein the ML model is configured to generate a predicted reference image having a random PV from a pattern variation (PV) distribution, and a deterministic mode, wherein the ML model is configured to generate a predicted reference image having a predetermined PV selected from a PV distribution, the ML model learning the PV distribution based on the PVs observed in the training set; and performing defect inspection on the runtime image using the generated reference image.

[0030] According to other aspects of the subject matter of the present disclosure, a non-transitory computer-readable medium is provided, the non-transitory computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform a computerized method of training a machine learning (ML) model that can be used for defect inspection of semiconductor samples, the method comprising: obtaining a training set comprising one or more pairs of training images, each pair comprising an input image and a target image corresponding to the input image; and using the training set to train the ML model alternately between two training modes, the training comprising, for a given pair of training images: in a random mode, generating, by the ML model, a predicted reference image having a random PV from a PV distribution based on the input image, and optimizing the ML model based on the predicted reference image relative to the target image, wherein the PV distribution is derived based on variations observed in the training set; and in a deterministic mode, generating, by the ML model, a predicted reference image having a predetermined PV selected from the PV distribution based on the target image, and optimizing the ML model based on the predicted reference image relative to the target image itself.

[0031] These aspects of the subject matter of the present disclosure may comprise, mutatis mutandis, one or more of the features (i) to (xv) listed above with respect to the system, in any desired combination or arrangement technically possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to understand the present disclosure and to see how it may be carried out in practice, embodiments will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which:

[0033] Figure 1 A generalized block diagram of an inspection system according to certain embodiments of the presently disclosed subject matter is shown.

[0034] Figure 2 A generalized flow chart for training a machine learning model that can be used for defect inspection of semiconductor samples according to certain embodiments of the presently disclosed subject matter is shown.

[0035] Figure 3 A generalized flow chart for training an ML model using image pairs of defective images and corresponding nominal images, according to certain embodiments of the subject matter of the present disclosure is shown.

[0036] Figure 4 A generalized flow chart for training an ML model using image pairs of two nominal images, according to certain embodiments of the subject matter of the present disclosure is shown.

[0037] Figure 5 A generalized flow chart for run-time defect inspection of semiconductor samples using a trained ML model according to certain embodiments of the subject matter of the present disclosure is shown.

[0038] Figure 6 A generalized flow chart for training an ML model with two decoders, according to certain embodiments of the subject matter of the present disclosure, is shown.

[0039] Figure 7 Schematic diagram showing a training process in random mode and deterministic mode using image pairs of defective images and corresponding nominal images, according to some embodiments of the presently disclosed subject matter.

[0040] Figure 8 Schematically illustrates a training process for an ML model with two decoders using image pairs of nominal images and synthetic defective images, according to some embodiments of the subject matter of the present disclosure. DETAILED DESCRIPTION

[0041] The process of semiconductor manufacturing generally requires multiple sequential processing steps and / or layers, each of which may produce errors that may result in yield loss. Examples of various processing steps may include lithography, etching, deposition, planarization, growth (such as, for example, epitaxial growth), and implantation, etc. Various inspection operations such as defect-related inspections (e.g., defect detection, defect review, and defect classification, etc.) and / or metrology-related inspections may be performed at different processing steps / layers during the manufacturing process to monitor and control the process. For example, the inspection operation may be performed multiple times after certain processing steps and / or after manufacturing certain layers, etc.

[0042] Various detection methods can be used to detect defects on a sample. As an example, a typical die-to-reference detection algorithm is often used, such as, for example, die-to-die (D2D). In D2D, an inspection image of a target die is captured. In order to detect defects in the inspection image, one or more reference images are captured from one or more reference dies (e.g., one or more adjacent dies) of the target die. The inspection image and the reference image are aligned and compared to each other. One or more difference images can be generated based on the difference between the pixel values ​​of the inspection image and the pixel values ​​derived from the one or more reference images. A detection threshold is then applied to the difference map, and a defect map indicating defect candidates in the target die is created.

[0043] There are certain disadvantages in the above-mentioned die-to-reference detection method. For example, the D2D method requires the acquisition of at least two images (i.e., in the case of a test image and a reference image), which doubles the image acquisition time of the inspection tool. In the case of using multiple references, the method greatly increases the image acquisition time depending on the number of reference images. In addition, before comparison, the test image and the reference image need to be aligned. In some cases, these images require additional preprocessing to compensate for the noise representing the changes between the two images (such as, for example, process changes and color changes). These inevitably increase the processing time of the detection method, thereby affecting the detection throughput (TpT). In addition, the detection sensitivity may also be affected due to residual changes and noise that are not eliminated by preprocessing.

[0044] With the continuous advancement of semiconductor manufacturing processes, semiconductor devices are developed into more and more complex structures, and their feature sizes are gradually reduced, which makes it more difficult for conventional methods to meet inspection performance.

[0045] Therefore, certain embodiments of the subject matter of the present disclosure propose the use of a machine learning-based defect inspection system that does not have one or more of the above-mentioned disadvantages. The present disclosure proposes the use of a machine learning (ML) model instead of additional acquisition of an actual reference image to generate a synthetic reference image, and using the generated reference image to perform defect inspection operations at runtime. The proposed runtime inspection system greatly reduces the image acquisition time of the tool and eliminates image pre-processing efforts (such as image registration and noise filtering), thereby improving inspection throughput and defect detection sensitivity. Specifically, the ML model is trained in a specific manner so that a reference image with a high probability of being defect-free can be generated while sharing a PV similar to the PV of the runtime image. When compared with the runtime image, such a reference image can produce an improved difference map in which the residual noise caused by process variations is greatly reduced, thereby improving defect detection sensitivity, as will be described in detail below.

[0046] Based on this, please note Figure 1 , which shows a functional block diagram of an inspection system according to certain embodiments of the subject matter of the present disclosure.

[0047] exist Figure 1 The inspection system 100 shown in the figure can be used to inspect semiconductor samples (e.g., wafers, dies, and / or portions thereof) as part of a sample manufacturing process. As mentioned above, inspection mentioned in this article can be interpreted as covering any type of operation involving defect inspection / detection, defect classification, segmentation, and / or metrology operations on the sample. The system 100 includes one or more inspection tools 120, which are configured to scan the sample and capture images thereof to be further processed for various inspection applications.

[0048] The term "inspection tool(s)" as used herein should be broadly interpreted to cover any tool that may be used in an inspection-related process, including, as non-limiting examples, scanning (in a single or multiple scans), imaging, sampling, reviewing, measuring, sorting, and / or other processes performed on a sample or portion thereof. Without limiting the scope of the present disclosure in any way, it should also be noted that the inspection tool 120 may be implemented as various types of inspection machines, such as an optical inspection machine, an electron beam inspection machine (e.g., a scanning electron microscope (SEM), an atomic force microscope (AFM), or a transmission electron microscope (TEM), etc.), etc.

[0049] The one or more inspection tools 120 may include one or more inspection tools and / or one or more review tools. In some cases, at least one of the inspection tools 120 may be an inspection tool that is configured to scan a sample (e.g., an entire wafer, an entire die, or a portion thereof) to capture an inspection image (typically at a relatively high speed and / or low resolution) to detect potential defects (i.e., defect candidates). During inspection, during exposure, the wafer may be moved in steps relative to a detector of the inspection tool (or the wafer and the tool may be moved in opposite directions relative to each other), and the wafer may be scanned stepwise along a strip of the wafer by the inspection tool, wherein the inspection tool images a portion / portion of the sample at a time (within the strip). As an example, the inspection tool may be an optical inspection tool. At each step, light may be detected from a rectangular portion of the wafer, and such detected light may be converted into multiple intensity values ​​at multiple points in the portion, thereby forming an image corresponding to a portion / portion of the wafer. For example, in optical inspection, an array of parallel laser beams may scan the surface of the wafer along the strip. The strips are laid in parallel rows / columns that are connected to each other to establish an image of the surface of the wafer one strip at a time. For example, the tool may scan the wafer along a strip from top to bottom, then switch to the next strip and scan the wafer from bottom to top, and so on, until the entire wafer is scanned and an inspection image of the wafer is collected.

[0050] In some cases, at least one of the inspection tools 120 may be an audit tool configured to capture audit images of at least some of the defect candidates detected by the inspection tool to determine whether the defect candidates are indeed defects of interest (DOIs). Such audit tools are typically configured to inspect fragments of a sample one at a time (typically at a relatively low speed and / or high resolution). As an example, the audit tool may be an electron beam tool, such as a scanning electron microscope (SEM), or the like. An SEM is an electron microscope that generates an image of a sample by scanning the sample with a focused electron beam. The electrons interact with the atoms in the sample, thereby generating various signals containing information about the surface morphology and / or composition of the sample. An SEM is capable of accurately inspecting and measuring features during the manufacture of semiconductor wafers.

[0051] The inspection tool and the review tool can be different tools located at the same or different locations, or a single tool operating in two different modes. In some cases, the same inspection tool can provide low-resolution image data and high-resolution image data. The resulting image data (low-resolution image data and / or high-resolution image data) can be transmitted to the system 101 directly or via one or more intermediate systems. The present disclosure is not limited to any particular type of inspection tool and / or the resolution of the image data generated by the inspection tool. In some cases, at least one of the inspection tools 120 has a metrology capability and can be configured to capture images and perform metrology operations on the captured images. Such inspection tools are also referred to as metrology tools.

[0052] According to some embodiments of the subject matter of the present disclosure, inspection system 100 includes a computer-based system 101 that is operably connected to an inspection tool 120 and is capable of automatically inspecting semiconductor samples for defects at runtime based on runtime images obtained during sample fabrication. System 101 is also referred to as a defect inspection system.

[0053] System 101 includes processing circuitry 102 that is operably connected to hardware-based I / O interface 126 and configured to provide processing required by an operating system, such as reference Figures 2 to 6 Detailed description. Processing circuit system 102 may include one or more processors (not shown separately) and one or more memories (not shown separately). One or more processors of processing circuit system 102 may be configured to execute several functional modules individually or in any appropriate combination according to computer-readable instructions implemented on a non-transitory computer-readable memory included in the processing circuit system. Such functional modules are hereinafter referred to as being included in the processing circuit system.

[0054] The one or more processors mentioned in this article may represent one or more general-purpose processing devices, such as microprocessors, central processing units, etc. More specifically, a given processor may be one of the following: a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor implementing other instruction sets, or a processor implementing a combination of instruction sets. One or more processors may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. One or more processors are configured to execute instructions for performing the operations and steps described herein.

[0055] The memory mentioned in this document may include one or more of the following: internal memory such as processor registers and cache, main memory such as read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.

[0056] According to some embodiments of the subject matter of the present disclosure, system 101 may be a runtime defect inspection system configured to perform defect inspection operations using a trained machine learning (ML) model based on runtime images obtained during sample manufacturing. In this case, one or more functional modules included in processing circuitry 102 of system 101 may include an ML model 106 previously trained to generate a reference image, and include a defect inspection module 108.

[0057] Specifically, the processing circuit system 102 can be configured to obtain a runtime image of a semiconductor sample acquired by an inspection tool via an I / O interface 126, and provide the runtime image as an input to a machine learning model (e.g., ML model 106) for processing. The ML model 106 can generate a reference image based on the runtime image. The ML model 106 is previously trained using a training set during a setup / training phase. Specifically, the ML model is trained alternately between two training modes: a random mode, in which the ML model is configured to generate a predicted reference image with a random PV from a pattern variation (PV) distribution, and a deterministic mode, in which the ML model is configured to generate a predicted reference image with a predetermined PV selected from a PV distribution. The ML model learns a PV distribution based on the PV observed in the training set. The defect inspection module 108 can be configured to perform defect inspection on the runtime image using the generated reference image.

[0058] In this case, the ML model 106 and the defect inspection module 108 can be viewed as part of a defect inspection recipe that can be used to perform a runtime defect inspection operation on the acquired runtime image. The system 101 can be viewed as a runtime defect inspection system that can use the defect inspection recipe to perform runtime defect related operations. Figure 5 Describes the details of the runtime checking process.

[0059] In some embodiments, the system 101 may be configured as a training system capable of training an ML model using a specific training set during a training / setting phase. In this case, one or more functional modules included in the processing circuit system 102 of the system 101 may include a training module 104 and an ML model to be trained 106. Specifically, the training module 104 may be configured to obtain a training set including one or more pairs of training images, each pair including an input image and a target image corresponding to the input image.

[0060] The training module 104 may be configured to train the ML model 106 using the training set. Specifically, the ML model may be trained alternately between two training modes: a random mode and a deterministic mode. In the random mode, for a given pair of training images, the ML model may process the input image and generate a predicted reference image with a random PV from a PV distribution. The ML model may be optimized based on the predicted reference image relative to the target image. The ML model may learn the PV distribution based on the PV observed in the training set. In the deterministic mode, the ML model may generate a predicted reference image with a predetermined PV selected from the PV distribution based on the target image, and optimize the ML model based on the predicted reference image relative to the target image itself.

[0061] As described above, the ML model, after being trained, can be used to generate reference images for runtime images. Figures 2 to 4 and Figure 6 Describes the details of the training process.

[0062] Will refer to Figures 2 to 6 The operation of systems 100 and 101 , processing circuitry 102 , and functional modules therein are described in further detail.

[0063] According to certain embodiments, the ML model 106 mentioned in this document can be implemented as various types of machine learning models. In some cases, the ML model can be implemented as a probabilistic generation model, such as, for example, a variational autoencoder (VAE), a generative adversarial network (GAN), and a Bayesian network, or a collection / combination thereof. The learning algorithm used by the ML model can be any of the following: supervised learning, unsupervised learning, self-supervised learning, or semi-supervised learning, etc. The subject matter of the present disclosure is not limited to a specific type of ML model or a specific type of learning algorithm used by the ML model.

[0064] In some embodiments, the ML model can be implemented as a deep neural network (DNN). The DNN can include multiple layers organized according to a corresponding DNN architecture. As a non-limiting example, the layers of the DNN can be organized according to a convolutional neural network (CNN) architecture, a recurrent neural network architecture, a recursive neural network architecture, a generative adversarial network (GAN) architecture, etc. Optionally, at least some layers can be organized into multiple DNN subnetworks. Each layer of the DNN may include multiple basic computing units (CEs), which are generally referred to as dimensions, neurons, or nodes in the art.

[0065] The weighted values ​​and / or thresholds associated with the CE of the deep neural network and its connections can be initially selected before training, and can be further iteratively adjusted or modified during training to achieve an optimal set of weighted values ​​and / or thresholds in the trained DNN. After each iteration, the difference between the actual output generated by the DNN module and the target output associated with the corresponding training data set can be determined. The difference can be referred to as an error value. When the loss / cost function indicating the error value is less than a predetermined value, or when a limited change in performance between iterations is achieved, the training can be determined to be complete. The input data set used to adjust the weights / thresholds of the deep neural network is called a training set.

[0066] Note that the teachings of the subject matter of the present disclosure are not constrained by a particular architecture of ML or DNN as described above.

[0067] It should be noted that although certain embodiments of the present disclosure relate to a processing circuit system 102 configured to perform the above-described operations, the functionality / operation of the aforementioned functional modules may be performed in various ways by one or more processors in the processing circuit system 102. As an example, the operation of each functional module may be performed by a specific processor or a combination of processors. Therefore, the operations of various functional modules may be performed by corresponding processors (or combinations of processors) in the processing circuit system 102, such as processing runtime images and performing defect inspections, etc., while optionally, these operations may be performed by the same processor. The present disclosure should not be limited to being interpreted as a single processor that always performs all operations.

[0068] In some cases, the inspection system 100 may include one or more inspection modules, such as, for example, a defect detection module, an automatic defect review module (ADR), an automatic defect classification module (ADC), a metrology operation module, and / or other inspection modules, in addition to the system 101. The one or more inspection modules may be implemented as stand-alone computers, or the functionality of the one or more inspection modules (or at least a portion thereof) may be integrated with the inspection tool 120. In some cases, the output of the system 101 (e.g., a trained ML model, a generated reference image, defect inspection results) may be provided to one or more inspection modules (such as an ADR, an ADC, etc.) for further processing.

[0069] According to some embodiments, the system 100 may include a storage unit 122. The storage unit 122 may be configured to store any data required to operate the system 101, such as data relating to inputs and outputs of the system 101, and intermediate processing results generated by the system 101. As an example, the storage unit 122 may be configured to store images of samples generated by the inspection tool 120 and / or derivatives thereof, such as, for example, runtime images, training sets as described above. Thus, these input images may be retrieved from the storage unit 122 and provided to the processing circuit system 102 for further processing. The output of the system 101 (such as, for example, a trained ML model, a generated reference image, defect inspection results) may be sent to the storage unit 122 for storage.

[0070] In some embodiments, the system 100 may optionally include a computer-based graphical user interface (GUI) 124 that is configured to enable user-specified input related to the system 101. For example, a visual representation of a sample, including an image of the sample, etc., may be presented to the user (e.g., via a display forming part of the GUI 124). Options to define certain operating parameters may be provided to the user via the GUI. The user may also view operating results or intermediate processing results, such as, for example, generated reference images, defect inspection results, etc., on the GUI.

[0071] In some cases, the system 101 may be further configured to send the results of the operation to the inspection tool 120 via the I / O interface 126 for further processing. In some cases, the system 101 may be further configured to send the results to the storage unit 122 and / or an external system (e.g., a yield management system (YMS) of a manufacturing plant (semiconductor foundry)). In the context of semiconductor manufacturing, the yield management system (YMS) is a data management, analysis and tool system, especially during manufacturing upgrades, the system collects data from semiconductor foundries and helps engineers find ways to increase yields. YMS helps semiconductor manufacturers and factories manage a large amount of production analysis with fewer engineers. These systems analyze yield data and generate reports. Integrated device manufacturers (IMDs), semiconductor foundries, fabless semiconductor companies, and outsourced semiconductor assembly and testing (OSATs) can use YMS.

[0072] Those skilled in the art will readily appreciate that the teachings of the subject matter of the present disclosure are not limited to Figure 1 Constraints of the system shown. Figure 1 Each system component and module in the can be composed of any combination of software, hardware and / or firmware, and relatedly, the software, hardware and / or firmware are executed on an appropriate device or devices, and the one or more devices perform the functions defined and explained herein. The equivalent and / or modified functionality described with respect to each system component and module can be combined or divided in another way. Therefore, in some embodiments of the subject matter of the present disclosure, the system may include more than Figure 1 The components, modules and functions shown may include fewer, additional, modified and / or different components, modules and functions.

[0073] Figure 1 Each component in the can represent multiple specific components, which are suitable for operating independently and / or cooperatively to process various data and electrical inputs, and for implementing operations related to the computerized inspection system. In some cases, multiple instances of a component can be utilized for performance, redundancy and / or availability reasons. Similarly, in some cases, multiple instances of a component can be utilized for functionality or application reasons. For example, different parts of a specific functionality can be placed in different instances of a component.

[0074] It should be noted that the present invention can be implemented in a distributed computing environment. Figure 1 The inspection system shown, wherein Figure 1One or more of the aforementioned components and functional modules shown may be distributed across several local and / or remote devices. As an example, the inspection tool 120 and the system 101 may be located at the same entity (hosted by the same device in some cases), or distributed across different entities. As another example, as described above, in some cases, the system 101 may be configured as a training system for training an ML model, while in some other cases, the system 101 may be configured as a runtime defect inspection system using the trained ML model. The training system and the runtime inspection system may be located at the same entity (hosted by the same device in some cases), or distributed across different entities, depending on the specific system configuration and implementation needs.

[0075] In some examples, certain components are implemented using a cloud, such as in a private cloud or a public cloud. Where the various components of the inspection system are not completely located in one location or one physical entity, communication between the various components of the inspection system may be achieved through any signaling system or communication components, modules, protocols, software languages, and drive signals, and the communication may be wired and / or wireless as appropriate.

[0076] It should also be noted that in some embodiments, at least some of the inspection tools 120, storage unit 122, and / or GUI 124 may be external to the inspection system 100 and operate in data communication with the systems 100 and 101 via the I / O interface 126. As described above, the system 101 may be implemented as one or more stand-alone computers used in conjunction with the inspection tools and / or additional inspection modules. Alternatively, the corresponding functions of the system 101 may be at least partially integrated with one or more inspection tools 120, thereby improving and enhancing the functionality of the inspection tools 120 in inspection-related processes.

[0077] Although not necessarily the case, the operational processing of systems 101 and 100 may be similar to that described with respect to Figures 2 to 6 Some or all of the steps of the method described herein may correspond to the following. Figures 2 to 6 The described methods and possible implementations thereof may be implemented by the systems 101 and 100. Therefore, it is noted that, mutatis mutandis, Figures 2 to 6 Embodiments discussed in connection with the described methods may also be implemented as various embodiments of the systems 101 and 100 , and vice versa.

[0078] refer to Figure 2 , shows a generalized flow chart for training a machine learning model that can be used for defect inspection of semiconductor samples according to certain embodiments of the subject matter of the present disclosure.

[0079] A training set including one or more pairs of training images may be obtained (202) (e.g., by a training module 104 in the processing circuit system 102). Each pair includes an input image and a target image corresponding to the input image. In a stochastic mode, the input image generally refers to an image fed to the ML model as an input to be processed, and the target image refers to an image used for comparison with a predicted image of the ML model in order to optimize the ML model. In a deterministic mode, a target image, which is a nominal image in the training image pair, is used as both an input image and a target image, depending on the composition of the training image pair, as will be described in detail below.

[0080] As an example, a pair of training images may include a defective image as an input image and a corresponding nominal image as a target image. A defective image as used in this article refers to an image that includes defective features representing actual defects on a sample or includes defective features representing actual defects on a sample with a high probability. A nominal image is also referred to as a defect-free image, which is a clean image without defective features or does not include any defective features with a high probability. The nominal image corresponds to the defective image in the sense of capturing a similar area containing a pattern similar to that of the defective image. Using such an image pair for training, the ML model can learn a nonlinear mapping relationship between the two populations of defective images and nominal images.

[0081] As another example, a pair of training images may include two nominal images with different pattern variations used as input image and target image, respectively. The pattern variation (PV) mentioned in this article should be broadly interpreted to include different variations, deviations or distortions associated with any pattern, structure or image, such as curved lines, edge roughness, surface roughness, CD variation / shift, missing patterns and grayscale variations, etc. It should be noted that the pattern variation mentioned in this article should not be limited to any specific size or resolution. If one or more measurements and / or detections indicate that one or more characteristics of the design formed on the sample are outside the expected range of values ​​of these one or more characteristics, then it can be considered that there is a "variation".

[0082] PV can be caused by all kinds of variations, such as, for example, process variations. Process variations can refer to variations in a sample caused by changes in the sample manufacturing process. As an example, the manufacturing process may cause certain structures / patterns to be slightly offset / scaled / distorted between different inspection images, which causes pattern variations in the image. As another example, the manufacturing process may cause thickness variations in the sample, which affects the reflectivity and, therefore, the grayscale of the resulting inspection image. For example, die-to-die material thickness variations may result in different reflectivities between two of the dies, which results in different background grayscale values ​​for the images of the two dies. Two images with different PVs mentioned in the present disclosure mean that the two images, although capturing similar patterns, may have different levels of PV caused by the effects of process variations.

[0083] In some cases, as described above, the training set for training the ML model may include one or more pairs of training images, each pair including a defective image and a corresponding nominal image. In some cases, as described above, the training set may include one or more pairs of training images, each pair including two nominal images with different PVs. In some other cases, the training set for training the ML model may include two types of training image pairs in any suitable combination.

[0084] In some cases, training images such as a defective image and a nominal image in a pair may be "real world" images (i.e., actual images) of a semiconductor sample acquired by an inspection tool during a sample manufacturing process. As an example, a defective image may be an inspection image captured by the tool from an inspection area of ​​the sample, the inspection image being verified to contain defective features. For example, in a defect detection process (in which case the inspection image is considered to be a defective image with a high probability of containing defective features), or in a defect review process (e.g., in a manual review by a user or in an automatic review by an ADR process), the inspection image may be verified to be a defective image containing defective features. The tool may capture nominal images from one or more reference areas of the inspection area (such as, for example, one or more adjacent dies of an inspection die in a D2D inspection), which reference areas are known to be non-defective or have a high probability of being non-defective. The nominal image may also be any inspection image that is verified to not contain any defective features.

[0085] In some embodiments, in addition to the "real world" training images, the training set used to train the ML model can be enriched by using one or more synthetic images simulated for semiconductor samples. For example, a defective image can be simulated by synthetically injecting defective features into a nominal image. Various methods of image augmentation can be used, and image augmentation is used to inject synthetic defects into the image based on certain characteristics of the defects (such as, for example, type, size, and expected location, etc.), and the present disclosure is not limited to a specific implementation. In this case, the training set used to train the ML model can further include one or more pairs of training images, each pair including a nominal image and a synthetic defective image, and the synthetic defective image is generated by injecting the synthetic defects into the nominal image. When injecting defects, the defect characteristics of the injected defects (such as, for example, type, size, location in the image, and type) are all known, which can be used as a baseline truth for the synthetic defective image. Such image pairs can be used alone or in combination with the above-mentioned image pairs, which will be described in further detail below.

[0086] continue Figure 2 In the description of the invention, the ML model may be previously trained (204) using a training set (e.g., by a training module 104 in the processing circuit system 102) alternating between two training modes: a random mode, in which the ML model is configured to generate a predicted reference image having a random PV from a pattern variation (PV) distribution, and a deterministic mode, in which the ML model is configured to generate a predicted reference image having a predetermined PV selected from the PV distribution. The PV distribution may be derived based on the pattern variations observed in the training set, and the ML model may learn the PV distribution during training.

[0087] Specifically, the training for the ML model includes: for a given pair of training images: in a random mode, the ML model generates (206) a predicted reference image with a random PV from a PV distribution based on the input image, and optimizes the ML model based on the predicted reference image relative to the target image; and in a deterministic mode, the ML model generates (208) a predicted reference image with a predetermined PV selected from the PV distribution based on the target image (note that in the case of an image pair with two nominal images, either image can be regarded as a target image and the other image can be regarded as a corresponding input image), and optimizes the ML model based on the predicted reference image relative to the target image itself. The reference image used in this article refers to a nominal / defect-free image that does not have defective features or has a high probability of not including any defective features, so that it can be used as a reference for a corresponding defective image for defect inspection.

[0088] refer to Figure 3, shows a generalized flow chart for training an ML model using image pairs of defective images and corresponding nominal images, according to certain embodiments of the subject matter of the present disclosure.

[0089] Specifically, in a random mode, a defective image may be provided as an input to the ML model. The ML model may process (302) the defective image to obtain a predicted reference image with a random PV from a PV distribution. A nominal image is used as a target image. The ML model may be optimized to reduce the difference between the predicted reference image and the nominal image.

[0090] In a deterministic mode, a nominal image in the image pair can be used as both an input image and a target image. The ML model can process (304) the nominal image to obtain a predicted reference image having a predetermined PV selected from a PV distribution and similar to the PV of the nominal image. The ML model can be optimized to reduce the difference between the predicted reference image and the nominal image itself.

[0091] Figure 7 is a schematic diagram of a training process in a random mode and a deterministic mode using image pairs of defective images and corresponding nominal images, according to certain embodiments of the presently disclosed subject matter.

[0092] Box 700 shows a random pattern of training processing using an image pair with a defective image 702 and a corresponding nominal image 712. As shown, a defective image 702 having a defective feature 704 is provided as an input to an ML model 706 to be processed. The output of the ML model 706 is a predicted reference image 708. The predicted image 708 is evaluated relative to the nominal image 712 in the image pair using a loss function 710. As an example, the loss function 710 may include a reconstruction loss, which may be represented by a difference metric representing the difference between the predicted reference image and the nominal image, for example. The ML model 706 may be optimized by reducing / minimizing the value of the loss function 710. As an example, the ML model 706 may be optimized using a loss function such as, for example, a mean square error (MSE), a sum of absolute differences (SAD), a structural similarity index measure (SSIM), or an edge-preserving loss function. It should be noted that the term "minimize" or "minimize" as used herein refers to an attempt to reduce the difference represented by the loss function to a certain level / degree (which may be predefined), but does not necessarily have to reach an actual minimum value.

[0093] As described above, the ML model 706 can be implemented as various types of learning models. In some embodiments, the ML model can be implemented as a probabilistic generation model, such as, for example, a variational autoencoder (VAE), a GAN, and a Bayesian model. VAE is used as an example for illustration, and VAE is a variation of an autoencoder, which is a neural network that is generally used for data reproduction by learning efficient data encoding and reconstructing its input (e.g., by minimizing the difference between input and output). An autoencoder generally has an input layer, an output layer, and one or more hidden layers connecting them. Generally, an autoencoder can be considered to include two components: an encoder and a decoder. The autoencoder learns to encode the data from the input layer into a short code (i.e., an encoder), and then decodes the code into an output that closely matches the original data (i.e., a decoder component). The output of the encoder is referred to as a code, a latent variable, or a potential representation in a latent space representing an input image. The code can pass through a hidden layer in a decoder and can be reconstructed into an output image corresponding to the input image in the output layer.

[0094] The main difference between a VAE and an autoencoder is that a VAE aims to learn the underlying probability distribution of the training data, for example by learning a low-dimensional latent representation of the training data called a latent variable, so that it can sample new data from the learned distribution during inference. In other words, the latent space of a VAE (and in the case of deep or very deep VAEs, also some of the sampling layers in the decoder) can represent a range of possible values ​​of the latent variables, rather than a fixed set of values ​​as in the latent space of an autoencoder. In this way, a VAE can produce / generate multiple different samples all from the same distribution.

[0095] Specifically, as illustrated in block 700, the encoder 714 of the VAE 706 may be configured to map an input image (e.g., defective image 702) to a latent variable in a latent space 716 representing corresponding features from the input image. The VAE may also model the probability distribution of these features, thereby allowing visualization of pattern changes at different levels that affect these features. The VAE may learn the probability distribution of different features in the training set during training. As an example, the ML model may learn a PV distribution 720 (such as a normal distribution, for example) representing pattern changes of different features of the images in the training set (which may be represented by, for example, a latent vector) during training. The PV distribution 720 may be derived in different ways based on different VAE network structures and configurations. As an example, in a typical VAE network, the PV distribution 720 may be provided by the latent space 716. As another example, in a relatively deep VAE, the PV distribution 720 may be provided as an overall distribution based on a distribution from the latent space 716 and multiple distributions from multiple decoder layers in the decoder 718.

[0096] In block 700, the ML model 706 is trained in a random mode, wherein the ML model is configured to generate a prediction reference image with a random PV from a PV distribution 720, i.e., each time the ML model generates a prediction image 708 with a PV randomly selected from the PV distribution. As described above, the ML model can be optimized by reducing the difference between the prediction reference image 708 and the target image (e.g., the nominal image 712 in this case). Since the prediction reference images with various PVs randomly selected from the PV distribution are generated, training the ML model in a random mode enables the ML model to learn that all PVs should be identified as normal as long as they are within the PV distribution, and defective features should be distinguished from all PVs and identified as abnormal.

[0097] In some embodiments, a VAE may be trained using a loss function that includes two terms: a reconstruction loss that evaluates the difference between the input image and the target image, and a regularization loss that forces the probability distribution in the latent space to be normal, as exemplified above with respect to loss function 710. In this case, loss function 710 should be considered to include two terms.

[0098] Block 730 illustrates a deterministic mode of the training process, in which the ML model 706 is configured to generate a predicted reference image having a predetermined PV selected from the PV distribution 720. As an example, in some cases, the predetermined PV may be selected as the peak 736 of the PV distribution 720, which represents the most likely PV level in the input image. In other words, each time the ML model generates a predicted image 732 having a fixed level of PV similar to the PV of the input image. In this case, the ML model 706 effectively acts as an autoencoder that provides a predicted output based on a fixed value of the latent variable rather than its probability distribution.

[0099] As shown, the nominal image 712 of the image pair is provided as an input to the ML model 706 to be processed. The output of the ML model 706 is a predicted reference image 732. The predicted image 732 is evaluated relative to the nominal image 712 itself using a loss function 734. As an example, the loss function 734 may include a reconstruction loss, which may be represented by a difference metric, for example, representing the difference between the predicted reference image and the nominal image. The ML model 706 may be optimized by reducing / minimizing the value of the loss function 734.

[0100] As an autoencoder, the ML model 706 is trained and optimized to learn representative features in the input nominal image. When generating a predicted image with a predetermined PV, the ML model 706 learns to generate a reference image with a PV level similar to that of the input image.

[0101] The ML model may be trained in any suitable alternating manner between the two modes, and the present disclosure is not limited to a particular order and / or number of training iterations in each mode. As an example, in some cases, for a given image pair, the training for the ML model may begin in a random mode for one or more iterations and then alternate to a deterministic mode, or alternatively, may proceed in other ways. In some cases, a batch of image pairs may be used to first train the ML model in one mode and then switch to another mode.

[0102] Now go to Figure 4 , shows a generalized flow chart for training an ML model using image pairs of two nominal images, according to certain embodiments of the subject matter of the present disclosure.

[0103] As described above, when the ML model is trained in a random mode using an image pair of two nominal images (e.g., a first nominal image and a second nominal image) having different PVs, either nominal image may be used as an input image of the ML model, and the other nominal image may be used as a target image. Specifically, a first nominal image is provided as an input of the ML model. The ML model may process (402) the first nominal image to obtain a predicted reference image having a random PV from a PV distribution. The second nominal image is used as a target image. The ML model may be optimized to reduce a difference between the predicted reference image and the second nominal image.

[0104] As described above, the ML model is used as a VAE in random mode. Each time, the ML model will generate a predicted image with a PV randomly selected from the PV distribution. Since the predicted reference images with various random PVs from the PV distribution are generated, training the ML model in random mode enables the ML model to learn all these PVs, and as long as these PVs are within the PV distribution, they should be identified as normal behavior.

[0105] In a deterministic mode, the ML model acts as an autoencoder. Any of the nominal images may be used as both an input image and a target image. As an example, the ML model may process (404) at least one of the first nominal image and the second nominal image to generate at least one predicted reference image having a predetermined PV selected from a PV distribution and similar to the PV of the nominal image. The ML model may be optimized to reduce the difference between the at least one predicted reference image and the at least one nominal image itself. As described above, the predetermined PV may be selected as the peak / center of the PV distribution, which represents the most likely PV level in the input image. In other words, each time the ML model aims to generate a predicted image having a fixed level of PV similar to the PV of the input image.

[0106] It should be noted that the training set used to train the ML model can consist of any combination of image pairs of the two types. As an example, in some cases, the training set may only include image pairs of corresponding defective images and nominal images. In some other cases, the training set may only include image pairs of two nominal images. In other cases, the training set may include image pairs of the two types, with any suitable number of each type. As described above, the training set can be previously used to train the ML model in any suitable alternating manner between the two modes. By using alternating training, the ML model can learn to recognize all PVs as normal behavior, thereby distinguishing defective features from normal behavior. In addition, for a given input image, the model can learn to generate a reference image with a PV similar to the PV of the input image.

[0107] The ML model, once trained in the specific manner and deployed in inference, can effectively remove defective features from runtime images and generate reference images with the same PV as the runtime images. Such reference images, when compared to runtime images, can produce improved difference maps in which residual noise caused by process variations is greatly reduced, thereby improving defect detection sensitivity.

[0108] According to some embodiments, optionally, the one or more image pairs in the training set may include at least one pair of a nominal image and a synthetic defective image, and the synthetic defective image is generated by injecting the synthetic defect into the nominal image. As an example, in the image pair of the defective image and the nominal image, as described above with reference to Figure 3 The defective image may be a synthetic defective image generated by injecting synthetic defects into a nominal image.

[0109] Defects can be injected into the nominal image in various ways. As an example, defects can be injected directly into the actual nominal image. For example, image tiles containing defective features can be directly integrated into the nominal image based on certain defect characteristics, such as the type and size of the defective features, and (one or more) target locations in the nominal image to be injected. One or more image characteristics of the image tile can be manipulated before pasting into the nominal image. As an example, the manipulation can be based on the following: geometric transformation, grayscale intensity modification, filtering, style transfer, and contrast modification, etc. As another example, in some cases, defects can be first injected into the design data, and a synthetic defective image can be simulated based on the design data with the injected defects. The present disclosure is not limited to a specific manner of defect injection to generate a synthetic defective image.

[0110] The synthetic defective image thus generated is naturally associated with the reference real information of the injected defect, such as the location, type and size of the defect injected in the synthetic defective image. The synthetic defective image and the reference real information can be used to train an ML model for defect detection. In this case, in addition to the first decoder that provides the output of the predicted reference image as described above, the ML model may further include a second decoder that acts as a segmentation head. In other words, the ML model can be reconstructed to include an encoder and a first decoder, and a second decoder connected to the encoder and the first decoder, respectively.

[0111] Figure 6 A generalized flow chart for training an ML model with two decoders, according to certain embodiments of the subject matter of the present disclosure, is shown.

[0112] When training the ML model using image pairs of nominal images and synthetic defective images, synthetic defective images are generated by injecting synthetic defects into the nominal images, and the training process is similar to the reference with additional processing steps as described below Figure 3 The processing described.

[0113] In a random mode, a synthetic defective image is provided as an input to the ML model. The ML model processes (602) the synthetic defective image (e.g., through an encoder and a first decoder), and a predicted reference image with a random PV from a PV distribution can be obtained as an output of the first decoder. In addition, a segmentation map can be obtained (604) as an output of a second decoder (e.g., a segmentation head), the segmentation map including segments representing the presence of defects in the input image.

[0114] Figure 8 An ML model 800 is shown, which includes an encoder 802, a first decoder 804, and a second decoder 806 (also referred to as a segmentation head) operably connected to the encoder and the first decoder. A synthetic defective image 808 is generated by injecting defects 810 into a nominal image 812. A training image pair is formed including the synthetic defective image 808 and the nominal image 812. The synthetic defective image 808 is provided as an input to the ML model 800 to be processed. Specifically, the input is processed in two paths within the ML model 800. First, it is processed in the path of the encoder 802 and the first decoder 804 to generate a predicted reference image 814 as an output of the first decoder 804.

[0115] As the input is processed through the layers in the ML model, certain layers of the encoder and first decoder may provide layer outputs, for example, in the form of feature maps. For example, in the case of a VAE with a convolutional encoder and decoder, the output of each layer may be represented as a 2D output feature map. The size of the feature maps becomes smaller as they progress through the encoder, and becomes larger as they progress through the decoder. For example, the output feature map may be generated by convolving each filter of a particular layer over the width and height of the input feature map, and by producing a two-dimensional activation map that gives the response of the filter at each spatial location. Stacking the activation maps of all filters along the depth dimension forms a full output feature map for a particular layer. The output feature map of each layer in the encoder and decoder may be used to represent the features learned by the VAE.

[0116] In some embodiments, the second decoder 806 may collect outputs from layers in the encoder 802 and the first decoder 804 , such as, for example, output feature maps derived from layers thereof, and generate a segmentation map 818 based on the collected outputs.

[0117] In some cases, a segmentation map 818 may be generated at a pixel level, where each value in the segmentation map corresponds to a pixel in the input image and indicates the segment to which the corresponding pixel belongs. In some other cases, a segmentation map may be generated at a structure level or a region level, where each value in the segmentation map corresponds to a structure or region in the input image and indicates the segment to which the corresponding structure or region belongs. The segments may include defective segments (indicating the presence of defects) and non-defective segments. In some cases, defective segments may further indicate the type of defect present in the segment.

[0118] The ML model can be optimized based on two outputs. Specifically, a first loss function 816 can be used to evaluate the difference between the predicted reference image 814 and the nominal image 812, and the ML model can be optimized (606) to reduce the difference indicated by the first loss function. In addition, a second loss function 820 can be used to evaluate the generated segmentation map 818 relative to a ground truth segmentation map 809 associated with the input image, in which the segmentation 811 of the injected defect 810 is marked / circled, as shown. The ML model can be optimized (608) to reduce the difference between the predicted defect and the ground truth of the injected defect as indicated by the defective segment in the segmentation map.

[0119] With reference to above Figure 3The training in deterministic mode is performed in a similar manner as described in block 304 of . The nominal image in the image pair may be used as both the input image and the target image. The ML model may process (610) the nominal image to obtain a predicted reference image having a predetermined PV selected from the PV distribution and similar to the PV of the nominal image. The ML model may be optimized to reduce the difference between the predicted reference image and the nominal image itself. The nominal image may then be processed in a second branch via a second decoder 806, and the segmentation map 818 generated in this case should be a clean map without any defect segmentation. The segmentation map may be evaluated relative to a corresponding clean reference ground truth segmentation map indicating the absence of defects.

[0120] Given a runtime image, the ML model constructed and trained in this way can directly generate a segmentation map indicating the presence of defects in the runtime image. Alternatively, the ML model can also be used to generate a reference image for the runtime image, which can be used for defect inspection in the runtime image.

[0121] Now go to Figure 5 , shows a generalized flow chart for performing runtime defect inspection on semiconductor samples using a trained ML model according to certain embodiments of the subject matter of the present disclosure.

[0122] As described above, semiconductor samples are usually made of multiple layers. During the manufacturing process of the sample, for example, after a processing step of a particular layer, the inspection process of the sample may be performed multiple times. In some cases, a sampling set of processing steps may be selected for in-line inspection based on the known effects of multiple processing steps on device characteristics or yield. An image of the sample or a portion thereof may be acquired at the sampling set of processing steps for inspection.

[0123] For purposes of illustration only, certain embodiments described below are described with respect to images of a given processing step / layer for a sample set of processing steps. Those skilled in the art will readily appreciate that the teachings of the subject matter of the present disclosure, such as processes for machine learning-based inspection, may be performed after any layer and / or processing step of the sample. The present disclosure should not be limited to the number of layers included in the sample and / or the specific layer(s) to be inspected.

[0124] A runtime image of a semiconductor sample may be obtained (502) (e.g., acquired by an inspection tool 120) during runtime inspection of the sample. A semiconductor sample may refer to a semiconductor wafer, die, or portion thereof, as used herein, and the semiconductor sample is manufactured and inspected in a semiconductor foundry during its manufacturing process. An image of a sample may refer to an image that captures at least a portion of the sample. As an example, the image may capture a target region or target structure of interest (e.g., a structural feature or pattern on a semiconductor sample) for inspection of the semiconductor sample. For example, the image may be an electron beam (e-beam) image acquired by an electron beam tool at runtime during in-line inspection of the semiconductor sample.

[0125] The trained ML model (e.g., ML model 106) can process the runtime image to generate (504) a reference image corresponding to the runtime image. Figures 2 to 4 and Figure 6 The ML model is previously trained during a training / setup phase. Specifically, the ML model is previously trained using a training set alternating between two training modes: a random mode, in which the ML model is configured to generate a prediction reference image with a random PV from a pattern variation (PV) distribution, and a deterministic mode, in which the ML model is configured to generate a prediction reference image with a predetermined PV selected from a PV distribution. The ML model learns a PV distribution based on the PVs observed in the training set.

[0126] Defect inspection may be performed (506) on the runtime image using the generated reference image (e.g., by the defect inspection module 108). As an example, defect inspection may be performed by directly comparing the runtime image to the generated reference image to obtain defect inspection results indicating a distribution of defects on the semiconductor sample. Defect inspection may refer to one or more of the following operations: defect detection, defect review, and defect classification. The ML model may be used to generate a reference image in any of these operations, as described in detail below.

[0127] As described above, defect detection refers to capturing inspection images of samples according to defect detection algorithms, such as, for example, in D2D inspection, and detecting potential defects based on the images. For each inspection image, a trained ML model can process the inspection image and generate a corresponding reference image. The reference image can be used for comparison with the inspection image to generate a defect map indicating the distribution of defect candidates on the semiconductor sample. In some cases, the defect candidate list can be further selected from the defect map as candidates with a higher probability of being defects of interest (DOI).

[0128] After defect detection, the defect candidates can be provided to a defect review tool (e.g., ADR). The review tool is configured to capture a review image (usually with a higher resolution) at the location of the corresponding candidate defect, and review the review image to determine whether the candidate defect is indeed a DOI. Similarly, in this case, the trained ML model can be used to process the review image and generate a corresponding reference image. The reference image can be used for comparison with the review image. The output of the review tool may include label data associated with the defect candidates, respectively, and the label data provides information on whether each defect candidate is a DOI.

[0129] In some cases, a defect classification tool (such as, for example, ADC) is used in addition to or instead of a defect review (DR) tool. As an example, the classification tool can provide label data that provides information on whether each defect candidate is a DOI, and for defect candidates labeled as DOI, the classification tool also provides the category or type of DOI. The trained ML model can be used in a similar manner here.

[0130] As described above, the ML model trained in the alternating training mode can learn to recognize all PVs as normal behavior, thereby distinguishing defective features from normal behavior. In addition, for a given input image, the model can learn to generate a reference image with a PV similar to that of the input image.

[0131] The ML model, once trained with the specific training process and deployed for runtime defect inspection, can effectively remove defective features from runtime images and generate reference images with the same PV as the runtime images. When compared with the runtime images, such reference images can produce improved difference maps in which residual noise and variations caused by process variations are greatly reduced, thereby improving defect detection sensitivity.

[0132] In some embodiments, generating a reference image using the ML model as described in reference block 504 may be repeated multiple times to obtain multiple reference images with random PVs. A composite reference image may be generated based on multiple reference images, for example by combining / averaging the multiple reference images in various ways. For example, in some cases, a composite image may be generated by selecting the minimum value of the corresponding pixel value from multiple reference images for each pixel. The selected value for each pixel constitutes a composite image. The composite reference image may further eliminate residual variations and noise between the reference image and the runtime image, thereby having a PV that is closer to the PV of the runtime image. The composite reference image may be used to perform defect inspection on the runtime image.

[0133] In some embodiments, in addition to or in lieu of a reference image, the trained ML model can generate a segmentation map. The segmentation map includes segments indicating the presence of defects. Figure 6 and Figure 8 As described, the ML model in this case may include an encoder, a first decoder, and a second decoder operably connected to the encoder and the first decoder. A reference image may be obtained as an output of the first decoder, and a segmentation map may be obtained as an output of the second decoder. Figure 6 and Figure 8 The training set used to train such an ML model includes at least one pair of a nominal image and a synthetic defective image, and the synthetic defective image is generated by injecting defects into the nominal image.

[0134] Note that the examples shown in the present disclosure (such as, for example, exemplary ML models, loss functions, defect inspection applications, etc.) are illustrated for exemplary purposes and should not be considered to limit the present disclosure in any way. Other suitable examples / implementations may be used in addition to or instead of the above.

[0135] One of the advantages of certain embodiments of the subject matter of the present disclosure as described herein is that instead of additional image acquisition of actual reference images, a machine learning model is provided that is capable of generating reference images for synthesis of runtime images and performing defect inspection operations using the generated reference images. The proposed system significantly reduces the image acquisition time of the inspection tool.

[0136] Another advantage of certain embodiments of the subject matter of the present disclosure as described herein is that an ML model trained in a specific alternating training mode can learn to recognize all PVs as normal behavior, thereby distinguishing defective features from normal behavior. In addition, for a given input image, the model can learn to generate a reference image with a PV similar to that of the input image.

[0137] The ML model, once trained with the specific training process and deployed for runtime defect inspection, can effectively remove defective features from runtime images and generate reference images that have a high probability of being defect-free while sharing PVs similar to those of the runtime images. When compared with the runtime images, such reference images can produce improved difference maps in which residual noise caused by process variations is greatly reduced, thereby improving defect detection sensitivity.

[0138] In addition to or in lieu of a reference image, a further advantage of certain embodiments of the subject matter of the present disclosure as described herein may also be that the trained ML model provides an output of a segmentation map. The segmentation map may provide a direct indication of the presence of a defect. In this case, the ML model is specifically constructed to include an encoder, a first decoder, and a second decoder operably connected to the encoder and the first decoder, and the ML model is specifically trained as described above. The reference image may be obtained as an output of the first decoder, and the segmentation map may be obtained as an output of the second decoder.

[0139] It is to be understood that the disclosure is not limited in its application to the details set forth in the description contained herein or illustrated in the drawings.

[0140] In this detailed description, many specific details are set forth in order to fully understand the present disclosure. However, it will be appreciated by those skilled in the art that the subject matter of the present disclosure can be practiced without these specific details. In other cases, well-known methods, processes, components and circuits are not described in detail to avoid obscuring the subject matter of the present disclosure.

[0141] Unless otherwise specifically stated, as is apparent from this discussion, it should be understood that throughout the specification, discussions utilizing terms such as "obtain," "inspect," "provide," "train," "use," "generate," "execute," "optimize," "select," "enable," "compare," "repeat," "inject," "evaluate," etc. refer to the actions (one or more) and / or processes (one or more) of a computer that manipulates data and / or transforms data into other data, the data being represented as physical quantities, such as electronic quantities, and / or the data representing physical objects. The term "computer" should be broadly interpreted to encompass any type of hardware-based electronic device with data processing capabilities, including, as non-limiting examples, the inspection systems, defect inspection systems, and corresponding portions thereof disclosed in the present application.

[0142] As used herein, the terms "non-transitory memory" and "non-transitory storage medium" should be interpreted broadly to cover any volatile or non-volatile computer memory suitable for the subject matter of the present disclosure. The terms should be considered to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more instruction sets. The terms should also be understood to include any medium capable of storing or encoding an instruction set executed by a computer and causing the computer to perform any one or more of the methods of the present disclosure. Thus, the terms should be considered to include, but are not limited to, read-only memory ("ROM"), random access memory ("RAM"), magnetic disk storage media, optical storage media, flash memory devices, and the like.

[0143] The term "sample" used in this specification should be interpreted broadly to cover any type of physical object or substrate used to manufacture semiconductor integrated circuits, magnetic heads, flat panel displays, and other semiconductor products, including wafers, masks, graticules, and other structures, and combinations and / or parts thereof. The sample is also referred to herein as a semiconductor sample, and the sample can be produced by manufacturing equipment that performs the corresponding manufacturing process.

[0144] The term "inspection" used in this specification should be broadly interpreted to cover any type of operation involving defect detection, defect review and / or various types of defect classification, segmentation, and / or metrology operations during and / or after the sample manufacturing process. Inspection is performed by using a non-destructive inspection tool during or after the manufacture of the sample to be inspected. As a non-limiting example, the inspection process may include runtime scanning (in a single scan or multiple scans), imaging, sampling, detection, review, measurement, classification and / or other operations performed on the sample or part thereof using the same or different inspection tools. Similarly, the inspection can be performed before the sample to be inspected is manufactured, and the inspection may include, for example, generating one or more inspection recipes and / or other setup operations. Note that unless otherwise specifically stated, the term "inspection" or its derivatives used in this specification are not limited to the resolution or size of the inspection area. As non-limiting examples, various non-destructive inspection tools include scanning electron microscopes (SEMs), atomic force microscopes (AFMs), optical inspection tools, etc.

[0145] The term "metrology operation" as used in this specification should be broadly interpreted to cover any metrology operation process for extracting metrology information related to one or more structural elements on a semiconductor sample. In some embodiments, the metrology operation may include measurement operations, such as, for example, critical dimension (CD) measurements performed on certain structural elements on the sample, including but not limited to the following: dimensions (e.g., line width, line spacing, contact diameter, size of elements, edge roughness, grayscale statistics, etc.); shape of elements; distances within or between elements; related angles; overlap information associated with elements corresponding to different design levels, etc. For example, by using image processing techniques to analyze measurement results such as measured images. Note that unless otherwise specifically stated, the term "metrology" or its derivatives used in this specification are not limited to measurement techniques, measurement resolution, or size of inspection areas.

[0146] The term "defect" as used in this specification should be interpreted broadly to cover any type of abnormal or undesirable feature / functionality formed on a sample. In some cases, a defect may be a defect of interest (DOI), which is a real defect that has some impact on the functionality of the manufactured device, and the customer is therefore interested in the defect to be detected. For example, any "killer" defect that may cause yield loss may be denoted as a DOI. In some other cases, a defect may be a negligible impairment (also known as a "false alarm" defect) because it has no impact on the functionality of the finished device and does not affect yield.

[0147] The term "defect candidate" used in this specification should be interpreted broadly to cover suspected defect locations on a sample that are detected as having a higher probability of being a defect of interest (DOI). Therefore, when reviewed / tested, a defect candidate may actually be a DOI, or in some other cases, it may be damage as described above, or it may be random noise caused by different variations (e.g., process variations, color variations, mechanical and electrical variations, etc.) during inspection.

[0148] The term "design data" as used in this specification should be interpreted broadly to cover any data indicating the hierarchical physical design (layout) of a sample. The design data may be provided by a respective designer and / or may be derived from a physical design (e.g., by complex simulations, simple geometric and Boolean operations, etc.). The design data may be provided in different formats, such as, as non-limiting examples, the GDSII format, the OASIS format, etc. The design data may be presented in a vector format, a grayscale intensity image format, or other formats.

[0149] The terms "image(s)" or "image data" as used in this specification should be interpreted broadly to cover any original image / frame of a sample captured by an inspection tool during a manufacturing process, derivatives of the captured image / frame obtained by various pre-processing stages, and / or computer-generated synthetic images (in some cases based on design data). Depending on the specific manner of scanning (e.g., one-dimensional scanning such as line scanning, two-dimensional scanning in x and y directions, or point scanning at specific points, etc.), the image data can be represented in different formats, such as, for example, as a grayscale profile, a two-dimensional image, or discrete pixels, etc. It should be noted that in some cases, in addition to the image (e.g., captured image, processed image, etc.), the image data referred to in this document may also include digital data associated with the image (e.g., metadata, manual attributes, etc.). It should also be noted that an image or image data may include data related to a processing step / layer of interest or multiple processing steps / layers of a sample.

[0150] It should be understood that, unless otherwise specifically stated, certain features of the subject matter of the present disclosure described in the context of separate embodiments may also be provided in combination in a single embodiment. Conversely, various features of the subject matter of the present disclosure described in the context of a single embodiment may also be provided individually or in any suitable subcombination. In this detailed description, many specific details are set forth in order to fully understand the methods and apparatus.

[0151] It should also be understood that the system according to the present disclosure can be implemented at least in part on a suitably programmed computer. Likewise, the present disclosure contemplates a computer-readable computer process for performing the method of the present disclosure. The present disclosure further contemplates a non-transitory computer-readable memory for performing the method of the present disclosure, the non-transitory computer-readable memory tangibly containing a process of instructions that can be executed by a computer.

[0152] The present disclosure is capable of other embodiments and can be practiced and executed in various ways. Therefore, it should be understood that the words and terms used herein are for descriptive purposes and should not be considered restrictive. Therefore, those skilled in the art will understand that the concepts upon which the present disclosure is based can be easily used as the basis for designing other structures, methods and systems for several purposes of achieving the subject matter of the present disclosure.

[0153] Those skilled in the art will readily appreciate that various modifications and changes may be made to the embodiments of the present disclosure described above without departing from the scope of the present disclosure as defined by the appended claims.

Claims

1. A computerized system for performing defect inspection on a semiconductor sample, the system comprising processing circuitry configured to: obtaining a runtime image of the semiconductor sample; generating a reference image based on the runtime image using a machine learning (ML) model, wherein the ML model was previously trained using a training set alternating between two training modes: a random mode, wherein the ML model is configured to generate a predicted reference image having a random PV from a pattern variation (PV) distribution; and a deterministic mode, wherein the ML model is configured to generate a predicted reference image having a predetermined PV selected from the PV distribution, the ML model learning the PV distribution based on PVs observed in the training set; and A defect check is performed on the runtime image using the generated reference image.

2. The computerized system of claim 1 , wherein the training set comprises one or more pairs of training images, each pair comprising a defective image as an input image and a corresponding nominal image as a target image, or comprising two nominal images with different PVs as the input image and the target image, respectively.

3. The computerized system of claim 2, wherein the training comprises, for a given pair of training images: In a random mode, a predicted reference image having a random PV from a PV distribution is generated by the ML model based on the input image, and the ML model is optimized based on the predicted reference image relative to the target image; and In a deterministic mode, a predicted reference image having a predetermined PV selected from the PV distribution is generated by the ML model based on the target image, and the ML model is optimized based on the predicted reference image relative to the target image.

4. The computerized system of claim 1, wherein the ML model is a probabilistic generative model.

5. The computerized system of claim 1, wherein the random PV is randomly selected from the PV distribution, and the predetermined PV is selected as the center of the PV distribution so as to be similar to a PV in an input image to the ML model.

6. The computerized system of claim 1 , wherein the training in the random mode enables the ML model to learn to identify all PVs within the PV distribution as normal and distinguish defective features therefrom, and the training in the deterministic mode enables the ML model to learn to generate reference images that share PVs similar to those of an input image.

7. The computerized system of claim 1 , wherein the reference image shares a PV similar to a PV of the runtime image, and the processing circuit system is configured to perform the defect inspection by directly comparing the runtime image with the reference image to obtain a defect inspection result indicating a defect distribution on the semiconductor sample, wherein the defect inspection result reduces residual PV and improves detection sensitivity.

8. The computerized system of claim 1 , wherein the processing circuit system is configured to: repeat the generating of a reference image multiple times using the ML model to obtain multiple reference images with random PVs; and generate a composite reference image based on the multiple reference images, and wherein the defect inspection is performed on the runtime image using the composite reference image.

9. The computerized system of claim 8, wherein the composite image is generated by selecting, for each pixel, a minimum value of corresponding pixel values ​​from the plurality of reference images.

10. The computerized system of claim 1, wherein the processing circuitry is further configured to use the ML model to generate a segmentation map for the runtime image, the segmentation map comprising segments representing the presence of defects in the runtime image.

11. The computerized system of claim 10, wherein the ML model comprises an encoder, a first decoder, and a second decoder connected to the encoder and the first encoder, wherein the reference image is obtained as an output of the first encoder and the segmentation map is obtained as an output of the second decoder.

12. The computerized system of claim 11, wherein the one or more image pairs further include at least one pair of a nominal image and a synthetic defective image, the synthetic defective image being generated by injecting defects into the nominal image.

13. A computerized method for training a machine learning (ML) model that can be used to perform defect inspection on semiconductor samples, the method comprising: Obtaining a training set comprising one or more pairs of training images, each pair comprising an input image and a target image corresponding to the input image; as well as The ML model is trained alternately between two training modes using the training set, the training comprising, for a given pair of training images: generating, by the ML model based on the input image, a predicted reference image having a random PV from a pattern variation (PV) distribution in a random mode, and optimizing the ML model based on the predicted reference image relative to the target image, wherein the ML model learns the PV distribution based on the PV observed in the training set; as well as In a deterministic mode, a predicted reference image having a predetermined PV selected from the PV distribution is generated by the ML model based on the target image, and the ML model is optimized based on the predicted reference image relative to the target image itself.

14. The computerized method of claim 13, wherein each pair of training images comprises a defective image as an input image and a corresponding nominal image as a target image, or comprises two nominal images with different process variations (PVs) as the input image and the target image, respectively.

15. The computerized system of claim 14, wherein the one or more pairs include at least one pair of a defective image and a nominal image corresponding to the defective image, and the training comprises: in a random mode, processing the defective image by the ML model to obtain a predicted reference image having a random PV from the PV distribution, and optimizing the ML model to reduce a difference between the predicted reference image and the nominal image; as well as In the deterministic mode, the nominal image is processed by the ML model to obtain a predicted reference image having a predetermined PV selected from the PV distribution and similar to the PV of the nominal image, and the ML model is optimized to reduce the difference between the predicted reference image and the nominal image.

16. The computerized system of claim 14, wherein the one or more pairs include at least one pair of a first nominal image and a second nominal image having different PVs, and the training comprises: In the random mode, the first nominal image is processed by the ML model to obtain a predicted reference image having a random PV from the PV distribution, and the ML model is optimized to reduce a difference between the predicted reference image and the second nominal image; as well as In the deterministic mode, at least one of the first nominal image or the second nominal image is processed by the ML model to generate at least one predicted reference image having a predetermined PV selected from the PV distribution and similar to the PV of the at least one nominal image, and the ML model is optimized to reduce the difference between the at least one predicted reference image and the at least one nominal image.

17. The computerized system of claim 13, wherein the training in the random mode enables the ML model to learn to identify all PVs within the PV distribution as normal and distinguish defective features therefrom, and the training in the deterministic mode enables the ML model to learn to generate reference images having PVs similar to the PVs of an input image.

18. The computerized system of claim 15, wherein the defective image is a synthetic defective image generated by injecting defects into the corresponding nominal image, the synthetic defective image being correlated to a ground truth of the injected defects.

19. The computerized system of claim 18, wherein the ML model comprises an encoder, a first decoder, and a second decoder connected to the encoder and the first encoder, wherein the predicted reference image is obtained as an output of the first encoder, and the training further comprises: A segmentation map including segments indicating the presence of defects is obtained as an output of the second decoder, and the ML model is optimized based on the segmentation map relative to the benchmark truth.

20. A non-transitory computer-readable storage medium tangibly embodying a process of instructions which, when executed by a computer, cause the computer to perform a method of defect inspection for a semiconductor sample, the method comprising: obtaining a runtime image of the semiconductor sample; generating a reference image based on the runtime image using a machine learning (ML) model, wherein the ML model was previously trained using a training set alternating between two training modes: a random mode, wherein the ML model is configured to generate a predicted reference image having a random PV from a pattern variation (PV) distribution; and a deterministic mode, wherein the ML model is configured to generate a predicted reference image having a predetermined PV selected from the PV distribution, the ML model learning the PV distribution based on PVs observed in the training set; and A defect check is performed on the runtime image using the generated reference image.

Images