Machine learning based defect inspection for semiconductor samples

Abstract

A system and method of semiconductor sample inspection are provided. The method includes: obtaining a plurality of images of a semiconductor sample acquired by an inspection tool; processing the plurality of images using a first machine learning (ML) model for defect detection, thereby obtaining a set of images marked with detected defects from the plurality of images, where the first ML model is pre-trained using a first training set, the first training set comprises a subset of synthesized defective images and a subset of nominal images, and each synthesized defective image comprises one or more synthesized defects; and training a second ML model using a second training set, the second training set comprising at least a portion of the set of images marked with detected defects, where the second ML model is usable for defect detection after being trained, the second ML model having improved detection performance relative to the first ML model.

Description

Technical Field

[0001] The subject matter of the present disclosure relates generally to the field of inspection of semiconductor samples, and more particularly to machine learning-based defect inspection. Background Art

[0002] The current demand for high density and high performance associated with ultra-large scale integration of manufactured devices requires sub-micron feature structures, and requires increased transistor and circuit speeds, as well as enhanced reliability. As semiconductor processes advance, pattern dimensions such as line widths and other types of critical dimensions continue to shrink. Such demands require the formation of device feature structures with high precision and high uniformity, which in turn requires careful monitoring of the manufacturing process, including automated inspection of the device while it is still in the form of a semiconductor wafer.

[0003] Runtime inspection can generally employ a two-stage procedure, e.g., inspecting a sample and then reviewing the sampled locations for potential defects. Inspection generally involves generating some output (e.g., image, signal, etc.) for the sample by directing light or electrons to a wafer and detecting the light or electrons from the wafer. In the first stage, the surface of the sample is inspected at high speed and relatively low resolution. Defect detection is typically performed by applying a defect detection algorithm to the inspection output. A defect map is generated to show suspected locations on the sample that have a high probability of defects. In the second stage, at least some of the suspected locations are more thoroughly analyzed at a relatively high resolution to determine different parameters of the defect, such as class, thickness, roughness, size, etc.

[0004] During or after manufacturing the sample to be inspected, the inspection can be performed by using a non-destructive inspection tool. As non-limiting examples, various non-destructive inspection tools include scanning electron microscopes, atomic force microscopes, optical inspection tools, and the like.

[0005] The inspection process may include multiple inspection steps. The manufacturing process of the semiconductor device may include various procedures, such as etching, deposition, planarization, growth such as epitaxial growth, implantation, etc. The inspection step may be performed multiple times, for example after certain process procedures and / or after manufacturing certain layers, etc. Additionally or alternatively, each inspection step may be repeated multiple times, for example for different wafer positions, or for the same wafer position with different inspection settings.

[0006] Inspection processes are used at various steps during semiconductor manufacturing for the purpose of process control, such as defect-related operations and metrology-related operations. The effectiveness of inspection can be improved by automating one or more processes such as defect detection, automatic defect classification (ADC), automatic defect review (ADR), image segmentation, automatic metrology-related operations, etc.

[0007] Automated inspection systems ensure that manufactured parts meet expected quality standards and provide useful information about adjustments that may be needed to manufacturing tools, equipment, and / or components based on the type of defects identified. In some cases, machine learning techniques can be used to assist the automated inspection process in order to promote higher performance. Summary of the invention

[0008] According to certain aspects of the subject matter of the present disclosure, a computerized system for semiconductor sample inspection is provided, the system comprising a processing circuit system configured to: obtain a plurality of images of the semiconductor sample acquired by an inspection tool; process the plurality of images using a first machine learning (ML) model for defect detection, thereby obtaining a set of images labeled with detected defects from the plurality of images, wherein the first ML model is pre-trained using a first training set, the first training set comprising a subset of synthetic defective images and a subset of nominal images, each synthetic defective image containing one or more synthetic defects; and train a second ML model using a second training set, the second training set comprising at least a portion of the set of images labeled with detected defects and at least a portion of the subset of nominal images, wherein the second ML model can be used for defect detection after being trained, and the second ML model has improved detection performance relative to the first ML model.

[0009] In addition to the above features, the system according to this aspect of the subject matter of the present disclosure may include one or more of the features (i) to (xi) listed below in any desired combination or arrangement that is technically possible:

[0010] (i). The second ML model can be based on the first ML model.

[0011] (ii) The second ML model may be a new ML model different from the first ML model.

[0012] (iii). The processing may include: for each given image in a plurality of images, providing a segmentation map, the segmentation map comprising a value representing a tag, the tag indicating a defect-related segment corresponding to the given image; and selecting a set of images from the plurality of images, each image in the set of images being associated with a segmentation map comprising one or more tags indicating the presence of a defect.

[0013] (iv) The processing may further include: for each given image, providing an additional output based on features extracted from the given image, the additional output representing an image-level defect presence indication. In such a case, the selection may include selecting a set of images marked with detected defects based on consistency between the segmentation map and the additional output.

[0014] (v) The processing may further include: for each given image, providing an uncertainty map, the uncertainty map comprising values ​​representing the respective likelihoods of defect-related segments indicated by the segmentation map. In such a case, the selection may include selecting a set of images marked with detected defects based on the segmentation map and the uncertainty map.

[0015] (vi). A subset of synthetic defective images may be generated by injecting one or more defects in the design data of the sample and simulating synthetic defective images based on the design data with the injected defects.

[0016] (vii). The simulation may be performed by simulating physical effects of one or more processes, the one or more processes including a manufacturing process for manufacturing a sample based on design data, a scanning process for scanning the manufactured sample, and a signal processing process for processing the scanned signal to generate a synthetic defective image.

[0017] (viii). The simulation may be performed using an image conversion model trained to convert the design data of the sample into a synthetic image.

[0018] (ix). Design data can be generated based on actual images of samples.

[0019] (x). The first training set may further include a subset of real defective images.

[0020] (xi). Improved detection performance may be represented by at least one of the following detection measures: precision, recall, filter rate, and false alarm rate.

[0021] According to other aspects of the subject matter of the present disclosure, a method for inspecting a semiconductor sample is provided, the method comprising: obtaining a plurality of images of the semiconductor sample acquired by an inspection tool; processing the plurality of images using a first machine learning (ML) model for defect detection, thereby obtaining a set of images labeled with detected defects from the plurality of images, wherein the first ML model is pre-trained using a first training set, the first training set comprising a subset of synthetic defective images and a subset of nominal images, each synthetic defective image containing one or more synthetic defects; and training a second ML model using a second training set, the second training set comprising at least a portion of the set of images labeled with detected defects, wherein the second ML model can be used for defect detection after being trained, and the second ML model has improved detection performance relative to the first ML model.

[0022] These aspects of the subject matter of the present disclosure may comprise, mutatis mutandis, one or more of the features (i) to (xi) listed above with respect to the system, in any desired combination or arrangement technically possible.

[0023] According to other aspects of the subject matter of the present disclosure, a non-transitory computer-readable medium is provided, the non-transitory computer-readable medium including instructions that, when executed by a computer, cause the computer to perform a method for semiconductor sample inspection, the method including: obtaining a plurality of images of the semiconductor sample acquired by an inspection tool; processing the plurality of images using a first machine learning (ML) model for defect detection, thereby obtaining a set of images labeled with detected defects from the plurality of images, wherein the first ML model is pre-trained using a first training set, the first training set including a subset of synthetic defective images and a subset of nominal images, each synthetic defective image containing one or more synthetic defects; and training a second ML model using a second training set, the second training set including at least a portion of the set of images labeled with detected defects, wherein the second ML model can be used for defect detection after being trained, and the second ML model has improved detection performance relative to the first ML model.

[0024] These aspects of the subject matter of the present disclosure may comprise, mutatis mutandis, one or more of the features (i) to (xi) listed above with respect to the system, in any desired combination or arrangement technically possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to understand the present disclosure and to see how it may be carried out in practice, embodiments will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which:

[0026] Figure 1 A generalized block diagram of an inspection system according to certain implementations of the presently disclosed subject matter is shown.

[0027] Figure 2 A generalized flow chart of defect inspection according to certain implementations of the presently disclosed subject matter is shown.

[0028] Figure 3 A generalized flow chart for selecting a set of images having detected defects is shown, according to certain implementations of the presently disclosed subject matter.

[0029] Figure 4 A generalized flow chart for generating a subset of synthetic imperfect images according to certain implementations of the presently disclosed subject matter is shown.

[0030] Figure 5 A training process for a first ML model according to certain implementations of the subject matter of the present disclosure is shown.

[0031] Figure 6A schematic diagram of an exemplary first ML model providing multiple outputs according to certain implementations of the subject matter of the present disclosure is shown.

[0032] Figure 7 A schematic diagram illustrating employing a first ML model and a second ML model in a defect inspection system according to certain implementations of the subject matter of the present disclosure is shown. DETAILED DESCRIPTION

[0033] In the following detailed description, many specific details are set forth in order to fully understand the present disclosure. However, it will be appreciated by those skilled in the art that the subject matter of the present disclosure can be practiced without these specific details. In other cases, well-known methods, procedures, components and circuits are not described in detail to avoid obscuring the subject matter of the present disclosure.

[0034] Unless otherwise specifically stated, as will be apparent from the following discussion, it should be understood that throughout the specification, discussions utilizing terms such as "inspect," "obtain," "process," "extract," "compare," "generate," "train," "acquire," "subsample," "store," "search," "compute," "partition," "review," or similar terms refer to one or more actions and / or one or more processes of a computer that manipulates data and / or transforms data into other data, the data being represented as physical quantities, such as electronic quantities, and / or the data representing physical objects. The term "computer" should be broadly interpreted to encompass any type of hardware-based electronic device with data processing capabilities, such as personal computers, servers, computing systems, communication devices, and any other electronic computing devices, and as non-limiting examples, computers include the inspection systems, defect inspection systems, training systems, and corresponding portions thereof disclosed in the present application.

[0035] As used herein, the terms "non-transitory memory" and "non-transitory storage medium" should be broadly interpreted to cover any volatile or non-volatile computer memory suitable for the subject matter of the present disclosure. These terms should be considered to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more instruction sets. These terms should also be understood to include any medium capable of storing an instruction set executed by a computer or encoding the instruction set and causing the computer to perform any one or more of the methods of the present disclosure. Therefore, these terms should be considered to include, but are not limited to, read-only memory ("ROM"), random access memory ("RAM"), magnetic disk storage media, optical storage media, flash memory devices, and the like.

[0036] The term "sample" used in this specification should be broadly interpreted to cover any type of physical object or substrate used to manufacture semiconductor integrated circuits, magnetic heads, flat panel displays, and other semiconductor products, including wafers, masks, graticules, and other structures, and combinations and / or parts thereof. The sample is also referred to herein as a semiconductor sample, and the sample can be produced by a manufacturing device that performs a corresponding manufacturing process.

[0037] The term "inspection" used in this specification should be broadly interpreted to cover any type of operation involving defect detection, defect review and / or various types of defect classification, segmentation, and / or metrology operations during and / or after the sample manufacturing process. Inspection is performed by using a non-destructive inspection tool during or after the manufacture of the sample to be inspected. As a non-limiting example, the inspection process may include runtime scanning (in a single scan or multiple scans), imaging, sampling, detection, review, measurement, classification and / or other operations performed on the sample or part thereof using the same or different inspection tools. Similarly, the inspection can be performed before the sample to be inspected is manufactured, and the inspection may include, for example, generating one or more inspection recipes and / or other setup operations. Note that unless otherwise specifically stated, the term "inspection" or its derivatives used in this specification are not limited to the resolution or size of the inspection area. As non-limiting examples, various non-destructive inspection tools include scanning electron microscopes (SEMs), atomic force microscopes (AFMs), optical inspection tools, etc.

[0038] The term "metrology operation" as used in this specification should be broadly interpreted to cover any metrology operation procedure for extracting metrology information related to one or more structural elements on a semiconductor sample. In some embodiments, the metrology operation may include measurement operations, such as critical dimension (CD) measurements performed on certain structural elements on the sample, including but not limited to the following: dimensions (e.g., line width, line spacing, contact diameter, size of elements, edge roughness, grayscale statistics, etc.), shape of elements, distances within or between elements, related angles, overlap information associated with elements corresponding to different design levels, etc. For example, by using image processing techniques to analyze measurement results such as measured images. Note that unless otherwise specifically stated, the term "metrology" or its derivatives used in this specification are not limited in terms of measurement techniques, measurement resolution, or size of inspection areas.

[0039] The term "defect" as used in this specification should be interpreted broadly to cover any type of abnormal or undesirable feature / functionality formed on a sample. In some cases, a defect may be a defect of interest (DOI), which is a real defect that has some impact on the functionality of the manufactured device and is therefore a defect that the customer is interested in detecting. For example, any "killer" defect that may cause yield loss may be denoted as a DOI. In some other cases, a defect may be a negligible detriment (also known as a "false alarm" defect) because it has no impact on the functionality of the finished device and does not affect yield.

[0040] The term "design data" as used in this specification should be interpreted broadly to cover any data indicating the hierarchical physical design (layout) of a sample. The design data may be provided by a corresponding designer, and / or may be derived from a physical design (e.g., through complex simulations, simple geometry and Boolean operations, etc.). The design data may be provided in different formats, such as, as non-limiting examples, the GDSII format, the OASIS format, etc. The design data may be presented in a vector format, a grayscale intensity image format, or other formats.

[0041] The terms "image" or "image data" as used in this specification should be broadly interpreted to cover any original image / frame of a sample captured by an inspection tool during the manufacturing process, derivatives of the captured image / frame obtained by various pre-processing stages, and / or computer-generated composite images (in some cases based on design data). Depending on the specific manner of scanning (e.g., one-dimensional scanning such as line scanning, two-dimensional scanning in x and y directions, or point scanning at specific points, etc.), the image data can be represented in different formats, for example, as grayscale profiles, two-dimensional images, or discrete pixels, etc. It should be noted that in some cases, in addition to the image (e.g., captured image, processed image, etc.), the image data mentioned in this document may also include digital data associated with the image (e.g., metadata, manual attributes, etc.). It should also be noted that the image or image data may include data related to the processing step / layer of interest or multiple processing steps / layers of the sample.

[0042] It should be understood that, unless otherwise specifically stated, certain features of the subject matter of the present disclosure described in the context of separate embodiments may also be provided in combination in a single embodiment. Conversely, various features of the subject matter of the present disclosure described in the context of a single embodiment may also be provided individually or in any suitable subcombination. In the following detailed description, many specific details are set forth in order to fully understand the method and apparatus.

[0043] The process of semiconductor manufacturing generally requires multiple sequential processing steps / layers, each of which may produce errors that may result in yield loss. Examples of various processing steps may include lithography, etching, deposition, planarization, growth (e.g., epitaxial growth), and implantation, etc. Various defect inspection operations such as defect detection, defect review, and defect classification may be performed at different processing steps / layers during the manufacturing process to monitor and control the process. Inspection operations may be performed multiple times, such as after certain processing steps / layers, etc.

[0044] As mentioned above, in some cases, machine learning (ML) techniques can be used to assist the defect inspection process in order to provide accurate and effective solutions. In order to provide a well-trained, accurate ML model that is robust to various variations in actual production, the training images must be sufficient in terms of quantity, quality, and variation, and these images need to be annotated with accurate labels for supervised learning.

[0045] However, in some cases, it may be difficult to collect such training data. As an example, real defects (i.e., DOIs) are often scarce in number and subtle in appearance, so these defects are often buried in damage and noise and are very difficult to detect. Therefore, considering the different variations caused by some physical treatments of the samples (e.g., processing variations and color variations), the training defect samples for certain types of DOIs may be very limited in number and do not include enough variations of DOIs. ML models trained with insufficient defect training samples may not be able to detect unrepresented defects in production, and thus may not achieve the desired detection sensitivity (e.g., the detection results may have a high false alarm rate and a low DOI capture rate).

[0046] In addition, obtaining labeled data for training images containing defects can be very challenging, as manual identification and annotation of such rare and hard-to-detect real defects usually takes time and effort, and can be error-prone in some cases. Inaccurate labeling can mislead the ML model, causing it to fail to identify the actual DOI or misclassify defects at runtime, thus affecting detection performance.

[0047] In some cases, defect injection techniques (e.g., injecting defect image patches into nominal images) are used to generate synthetic defects in order to enrich training defect samples. However, such synthetically generated defects may often appear artificial and unrealistic relative to actual / real defects. In addition, synthetically generated defects may also not be sufficient to represent different variations of actual defects, resulting in the ML model trained with such synthetic defect training samples being unable to detect actual defects with the desired detection performance.

[0048] Therefore, certain embodiments of the subject matter of the present disclosure propose a system and method for defect inspection of semiconductor samples, which does not have one or more of the above-mentioned disadvantages. The present disclosure proposes a two-stage learning process, which includes: obtaining a plurality of actually acquired images of a semiconductor sample; and processing the plurality of images using a first machine learning (ML) model to obtain a set of images labeled with detected defects from the plurality of images; and training a second ML model using a second training set, the second training set including at least a portion of the set of images labeled with detected defects. The first ML model used in the process is pre-trained using a first training set, the first training set including a subset of synthetic defective images and a subset of nominal images, each synthetic defective image containing one or more synthetic defects. The training of the first ML model is referred to as the first stage of the learning process. The training of the second ML model is referred to as the second stage of the learning process. The second ML model can be used for defect detection after being trained, and the second ML model has improved detection performance relative to the first ML model, which will be described in detail below.

[0049] Based on this, please note Figure 1 , which shows a functional block diagram of an inspection system according to certain embodiments of the subject matter of the present disclosure.

[0050] exist Figure 1 The inspection system 100 shown in the can be used to inspect semiconductor samples (e.g., wafers, dies, or portions thereof) as part of a sample manufacturing process. As described above, references to inspection herein can be interpreted as covering any type of operation involving defect inspection / detection, various types of defect classification, segmentation, and / or metrology operations on the sample. The system 100 includes one or more inspection tools 120 configured to scan the sample and capture images thereof to be further processed for various inspection applications.

[0051] The term "inspection tool" as used herein should be broadly interpreted to encompass any tool that can be used in an inspection-related process, including, by way of non-limiting example, scanning, imaging, sampling, reviewing, measuring, sorting, and / or other processes performed on a sample or portion thereof. The inspection tool 120 can be implemented as various types of machines. In some embodiments, the inspection tool can be implemented as an electron beam machine / tool, such as a scanning electron microscope (SEM), an atomic force microscope (AFM), or a transmission electron microscope (TEM), among others.

[0052] As an example, a scanning electron microscope (SEM) is an electron microscope that produces an image of a sample by scanning the sample with a focused electron beam. The SEM is capable of accurately inspecting and measuring features during the manufacture of semiconductor wafers. The electrons interact with the atoms in the sample, thereby producing various signals containing information about the surface topography and / or composition of the sample.

[0053] According to some embodiments, the inspection tool 120 may include one or more inspection tools and / or one or more review tools. The inspection tool may scan the sample to capture the inspection image and detect potential defects according to the defect detection algorithm. The output of the detection module is a defect map indicating the distribution of defect candidates on the semiconductor sample. The review tool may be configured to capture review images at the locations of the defect candidates in the map and review these review images to determine whether the defect candidates are indeed DOIs. In some cases, at least one of the inspection tools 120 has metrology capabilities. Such inspection tools are also referred to as metrology tools. The metrology tool may be configured to generate image data in response to scanning the sample and perform metrology operations based on the image data.

[0054] In some cases, the same inspection tool can provide low-resolution image data and high-resolution image data. The resulting image data can be transmitted to system 101 directly or via one or more intermediate systems. The present disclosure is not limited to any particular type of inspection tool and / or representation / resolution of image data produced by the inspection tool.

[0055] According to some embodiments of the subject matter of the present disclosure, the inspection system 100 includes a computer-based system 101 that is operably connected to an inspection tool 120. In some embodiments, the system 101 can be configured as a training system capable of training a machine learning (ML) model during a training / setup phase. The ML model, after being trained, can be used to perform defect inspection on semiconductor samples at run time. In such a case, the system 101 can also be referred to as a training system for the ML model.

[0056] In some embodiments, the system 101 can be configured as a runtime defect inspection system using the aforementioned trained ML model based on runtime images of semiconductor samples acquired during the manufacturing process of the semiconductor samples. Such a system 101 is also referred to as a defect inspection system.

[0057] System 101 includes processing circuitry 102 that is operably connected to hardware-based I / O interface 126 and configured to provide the processing required to operate the system, as described with reference to FIG. Figures 2 to 5Detailed description. Processing circuit system 102 may include one or more processors (not shown separately) and one or more memories (not shown separately). One or more processors of processing circuit system 102 may be configured to execute several functional modules individually or in any appropriate combination according to computer-readable instructions implemented on a non-transitory computer-readable memory included in the processing circuit system. Such functional modules are hereinafter referred to as being included in the processing circuit system.

[0058] The one or more processors mentioned in this article may represent one or more general-purpose processing devices, such as microprocessors, central processing units, etc. More specifically, a given processor may be one of the following: a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor implementing other instruction sets, or a processor implementing a combination of instruction sets. One or more processors may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. One or more processors are configured to execute instructions for performing the operations and steps discussed herein.

[0059] The memory mentioned in this document may include one or more of the following: internal memory such as processor registers and cache, main memory such as read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.

[0060] As described above, in some embodiments, the system 101 can be configured as a training system capable of training an ML model during a training / setup phase, wherein the ML model can be used for runtime defect inspection after being trained. In such a case, one or more functional modules included in the processing circuit system 102 may include a first ML model 104, a training module 106, and a second ML model 108. When a plurality of images of a semiconductor sample acquired by an inspection tool (e.g., the inspection tool 120) are obtained, the first ML model 104 can be configured to process the plurality of images for defect detection, thereby obtaining a set of images marked with detected defects from the plurality of images. It should be noted that the first ML model for defect detection refers to a trained ML model pre-trained using a first training set (e.g., by the training module 106), the first training set including a subset of synthetic defective images and a subset of nominal images, each synthetic defective image containing one or more synthetic defects.

[0061] The inspection system 100 may further include a synthetic image generator 110 configured to generate a subset of synthetic defective images for training the first ML model. In some embodiments, the synthetic image generator 110 may include an image conversion model 112 that can be used to generate synthetic defective images based on design data stored in the design data server 124.

[0062] The training module 106 can be configured to train a second ML model 108 using a second training set, the second training set including at least a portion of the set of images labeled with detected defects. The second ML model 108 can be used for defect detection after being trained, and the second ML model 108 has improved detection performance relative to the first ML model.

[0063] In some embodiments, the system 101 may be configured to perform runtime defect inspection based on runtime images of samples acquired during a manufacturing process of the samples in a semiconductor factory. In such a case, one or more functional modules included in the PMC 102 may include a trained second ML model 108.

[0064] Reference Figures 2 to 5 The operation of system 101, processing circuitry 102, and the functional modules therein are described in further detail.

[0065] According to certain embodiments, the first ML model 106 and the second ML model 112 mentioned herein may be implemented as various types of machine learning models. The learning algorithm used by the ML model may be any of the following: supervised learning, unsupervised learning, or semi-supervised learning, etc. The subject matter of the present disclosure is not limited to a specific type of ML model or a specific type of learning algorithm used by the ML model.

[0066] In some embodiments, the first ML model 106 and / or the second ML model 112 can be implemented as a deep neural network (DNN). The DNN can refer to a supervised or unsupervised DNN model, which includes layers organized according to a corresponding DNN architecture. As a non-limiting example, the layers of the DNN can be organized according to a convolutional neural network (CNN) architecture, a recurrent neural network architecture, a recursive neural network architecture, a generative adversarial network (GAN) architecture, or other architectures. Optionally, at least some layers can be organized into multiple DNN subnetworks. Each layer of the DNN can include multiple basic computing elements (CEs), which are generally referred to as dimensions, neurons, or nodes in the art.

[0067] The weighted values ​​and / or thresholds associated with the CE of the deep neural network and its connections can be initially selected before training, and can be further iteratively adjusted or modified during training to achieve an optimal set of weighted values ​​and / or thresholds in the trained DNN. After each iteration, the difference between the actual output generated by the DNN module and the target output associated with the corresponding training data set can be determined. The difference can be referred to as an error value. When the loss / cost function representing the error value is less than a predetermined value, or when a limited change in performance between iterations is achieved, the training can be determined to be complete. The set of input data used to adjust the weights / thresholds of the deep neural network is called a training set.

[0068] It should be noted that the teachings of the subject matter of the present disclosure are not constrained by the specific architecture of the ML model or DNN as described above.

[0069] It should be noted that although some embodiments of the present disclosure mention that the processing circuit system 102 is configured to perform the above operations, the functionality / operation of the aforementioned functional modules can be performed in various ways by one or more processors in the processing circuit system 102. As an example, the operation of each functional module can be performed by a specific processor or by a combination of processors. Therefore, the operations of various functional modules can be performed by the corresponding processor (or combination of processors) in the processing circuit system 102, such as obtaining multiple images, processing multiple images using a first ML model, and training a second ML model, etc., and optionally, at least some of these operations can be performed by the same processor. The present disclosure should not be limited to being interpreted as always a single processor performing all operations.

[0070] In some cases, in addition to the system 101, the inspection system 100 may include one or more inspection modules, such as a defect detection module and / or an automatic defect review module (ADR) and / or an automatic defect classification module (ADC) and / or a metrology-related module and / or other inspection modules that can be used to inspect semiconductor samples. One or more inspection modules can be implemented as independent computers, or the functionality of one or more inspection modules (or at least a portion thereof) can be integrated with the inspection tool 120. In some cases, the output of the system 101, such as the trained second ML model, detected defects, etc., can be provided to one or more inspection modules for further processing. In some cases, the trained second ML model 108 can be included in one or more inspection modules. Optionally, the trained second ML model can be shared between the inspection modules, or alternatively, each of the one or more inspection modules can include its own second ML model.

[0071] According to some embodiments, the system 101 may include a storage unit 122. The storage unit 122 may be configured to store any data required to operate the system 101, such as data related to input and output of the system 101, and intermediate processing results generated by the system 101. As an example, the storage unit 122 may be configured to store runtime images / training images and / or derivatives thereof generated by the inspection tool 120, and synthetic defective images generated by the synthetic image generator 110. Thus, the image may be retrieved from the storage unit 122 and provided to the processing circuit system 102 for further processing.

[0072] In some embodiments, the system 101 may optionally include a computer-based graphical user interface (GUI) 124 that is configured to allow user-specified input related to the system 101. For example, a visual representation of a sample, including image data of the sample, may be presented to the user (e.g., via a display forming part of the GUI 124). The user may be provided with options to define certain operating parameters, such as configuration of an ML model, etc. The user may also review the results of the operation, such as detected defects, on the GUI.

[0073] In some cases, the system 101 may be further configured to send the operation results to the inspection tool 120 via the I / O interface 126 for further processing. In some cases, the system 101 may be further configured to send the results to the storage unit 122 and / or an external system (e.g., a yield management system (YMS) of a manufacturing plant (semiconductor factory)). In the context of semiconductor manufacturing, the yield management system (YMS) is a data management, analysis and tool system, especially during manufacturing upgrades, which collects data from semiconductor factories and helps engineers find ways to increase production. YMS helps semiconductor manufacturers and semiconductor factories manage a large amount of production analysis with fewer engineers. These systems analyze yield data and generate reports. Integrated device manufacturers (IMDs), semiconductor factories, fabless semiconductor companies and outsourced semiconductor assembly and testing (OSATs) can use YMS.

[0074] Those skilled in the art will readily appreciate that the teachings of the subject matter of the present disclosure are not limited to Figure 1 Constraints of the system shown. Figure 1 Each system component and module in the can be composed of any combination of related software, hardware and / or firmware, which are implemented on an appropriate device or devices, and the one or more devices perform the functions defined and explained herein. The equivalent and / or modified functionality described with respect to each system component and module can be combined or divided in another way. Therefore, in some embodiments of the subject matter of the present disclosure, the system can include more Figure 1The components, modules and functions shown may include fewer, additional, modified and / or different components, modules and functions.

[0075] Figure 1 Each component in the can represent multiple specific components, which are suitable for operating independently and / or cooperatively to process various data and electrical inputs, and for implementing operations related to the computerized inspection system. In some cases, multiple instances of a component can be utilized for performance, redundancy and / or availability reasons. Similarly, in some cases, multiple instances of a component can be utilized for functionality or application reasons. For example, different parts of a specific functionality can be placed in different instances of a component.

[0076] It should be noted that the present invention can be implemented in a distributed computing environment. Figure 1 The inspection system shown, wherein Figure 1 One or more of the aforementioned functional modules shown may be distributed across several local and / or remote devices. As an example, the inspection tool 120 and the system 101 may be located at the same entity (in some cases hosted by the same device), or distributed across different entities, depending on the specific system configuration and implementation needs. As another example, the system 101 (or its first ML model 104 and / or second ML model 108) and the synthetic image generator 110 (or its image conversion model 112) may be located at the same entity, or distributed across different entities. In some examples, certain components utilize cloud implementations implemented, for example, in a private or public cloud. In the case where the various components of the inspection system are not entirely located in one location or one physical entity, communication between the various components of the inspection system may be achieved through any signaling system or communication component, module, protocol, software language, and drive signal, and the communication may be appropriately wired and / or wireless.

[0077] It should also be noted that in other embodiments, at least some of the inspection tools 120, storage unit 122, and / or GUI 124 may be external to the inspection system 100 and operated in data communication with the system 101 via the I / O interface 126. As described above, the system 101 may be implemented as one or more stand-alone computers to be used in conjunction with the inspection tools and / or additional inspection modules. Alternatively, the corresponding functions of the system 101 may be at least partially integrated with one or more inspection tools 120, thereby improving and enhancing the functionality of the inspection tools 120 in the inspection-related processes.

[0078] Although not necessarily the case, the operational processing of systems 101 and 100 may be similar to that described with respect to Figures 2 to 5 Some or all of the steps of the method described herein may correspond to the following. Figures 2 to 5The described methods and possible implementations thereof may be implemented by the systems 101 and 100. Therefore, it should be noted that, mutatis mutandis, Figures 2 to 5 Embodiments discussed in connection with the described methods may also be implemented as various embodiments of the systems 101 and 100 , and vice versa.

[0079] refer to Figure 2 , showing a generalized flow chart of defect inspection according to certain embodiments of the subject matter of the present disclosure.

[0080] A plurality of images of a semiconductor sample acquired by an inspection tool may be obtained (202) (e.g., by the PMC 102 from the inspection tool 120). The semiconductor sample herein may refer to a semiconductor wafer, die, or portion thereof, which is manufactured and inspected in a semiconductor factory during its manufacturing process. An image of a sample may refer to an image capturing at least a portion of the sample. As an example, the image may capture a given area of ​​interest or a given structure on the semiconductor sample to be inspected (e.g., a structural feature or pattern on the semiconductor sample). For example, the image may be an electron beam (e-beam) image acquired by an electron beam tool at run time during in-line inspection of the semiconductor sample. It should be noted that, in contrast to a synthetic image, the plurality of images refers to a true image of the sample actually acquired by the tool, as will be described below.

[0081] The manufacturing process of the sample typically includes multiple processing steps. In some cases, a sampling set of processing steps can be selected for online inspection based on the known effects of multiple processing steps on device characteristics or output. An image of the sample or a portion thereof can be acquired at the sampling set of processing steps for inspection. For illustrative purposes only, certain embodiments described below are described with respect to images acquired for specific processing steps / layers of the sampling set of processing steps. Those skilled in the art will readily appreciate that the teachings of the subject matter of the present disclosure also apply to multiple processing steps of a sample.

[0082] A first ML model for defect detection (e.g., ML model 104) may be used to process (204) a plurality of images, thereby obtaining a set of images marked with detected defects from the plurality of images. Detected defects referred to herein may refer to any type of defective features on a sample, such as bridges, particles, line cuts, protrusions, intrusions, missing patterns, etc. In some cases, defects may also include variations associated with any pattern, structure, or image, such as bent lines, edge roughness, surface roughness, CD variations / shifts, grayscale variations, etc. It should be noted that the variations referred to herein should not be limited to any particular size or resolution. A "variation" may be considered to exist if one or more measurements and / or detections indicate that one or more characteristics of a design formed on a sample are outside an expected range of values ​​for these one or more characteristics.

[0083] According to certain embodiments, processing of images may include: for each given image in a plurality of images, providing by a first ML model a segmentation map including a value representing a tag, the tag indicating a defect-related segment corresponding to the given image; and selecting a set of images from the plurality of images, each image in the set of images being associated with a segmentation map including one or more tags indicating the presence of a defect. The set of selected images constitutes a set of images marked with detected defects. Images marked with at least one detected defect are also referred to as defective images. Defective images as used herein refer to images that include features of defects on a sample or images that have a high probability of including features of defects on a sample.

[0084] In some cases, a segmentation map of a given image may be generated at a pixel level, wherein each label corresponds to a pixel in the given image and indicates the segment to which the corresponding pixel belongs. In some other cases, a segmentation map may be generated at a structure level or a region level, wherein each label corresponds to a structure or a region in the given image and indicates the segment to which the corresponding structure or region belongs. The segments used for labeling may include defective segments and non-defective segments. In some cases, defective segments may further indicate the type of defects present in the segment.

[0085] According to certain embodiments, in order to generate a segmentation map, a score map corresponding to a given image is first generated by a first ML model. For example, the score map may include a score indicating a pixel-level probability of a defect in a given image. A predefined defect threshold may be applied to the scores in the score map. For example, for pixels with a score higher than a threshold, a value of 1 is applied, otherwise a value of 0 is applied. In this way, the score map can be transformed into a binary segmentation map, where a value of 1 indicates a defective segment to which the corresponding pixel in the given image belongs (i.e., these pixels are defective pixels), and a value of 0 indicates a non-defective segment to which the corresponding pixel in the given image belongs (i.e., these pixels are non-defective pixels). In some cases, before applying the defect threshold, the score map can be merged at the structural level or the regional level, thereby generating a segmentation map at the structural level or the regional level.

[0086] It should be noted that although the segmentation map is illustrated above as a binary segmentation map derived from a score map, the term segmentation map mentioned in this document should be broadly interpreted to cover score maps and / or binary segmentation maps.

[0087] According to certain embodiments, the first ML model can be implemented using various DNN architectures such as CNN, GAN, etc. Assuming CNN is taken as an exemplary embodiment of the first ML model, CNN generally has a structure including an input layer and an output layer, as well as multiple hidden layers. The hidden layers of CNN generally include a series of convolutional layers, followed by additional layers such as pooling layers, fully connected layers, and normalization layers. In some cases, CNN can be viewed as consisting of two main functionalities / parts: an encoder and a decoder. The encoder performs feature extraction by encoding the input image into various semantic-level features. The decoder decodes these features into segmentation maps. Encoders and decoders generally include convolutional layers, fully connected layers, activation functions, normalization layers, and / or pooling layers.

[0088] In some cases, the convolutional layers in the feature extraction portion include a set of learnable filters and parameters. Each filter of a particular layer can be convolved over the width and height of the input volume of a given input image, calculating the dot product between the entry of the filter and the input, and then applying the activation function, and generating an activation map that gives the response of the filter at each spatial position. Stacking the activation maps of all filters along the depth dimension forms the full output feature map of a given convolutional layer. In this way, CNN learns filters that are activated when a certain type of feature is detected at a certain spatial position in the input. The output feature map is a representation of the features extracted in the input image.

[0089] Because CNN includes a series of convolutional layers, the output feature map can be extracted from any of the convolutional layers of the CNN. Typically, the early layers of a deep neural network learn lower-level features, while the deeper layers learn higher-level features. According to certain embodiments of the present disclosure, the CNN may provide additional outputs based on feature maps extracted from at least one of the convolutional layers (in addition to the output of the segmentation map). The additional output may represent an image-level defect presence indication generated by inputting the output of one or more convolutional layers into a classification head. As an example, the classification head may be composed of several convolutional layers, followed by a global average pull, a fully connected layer, and a sigmoid activation function.

[0090] In such a case, the set of images labeled with detected defects can be selected based on the consistency between the segmentation map and the additional output. As an example, in the case where both outputs indicate the presence of a defect, a given image will be selected to be included in the set of images. In this way, there is a higher certainty that the selected image is an actual defective image (rather than a false alarm).

[0091] Figure 3A generalized flow chart for selecting a set of images having detected defects according to certain embodiments of the subject matter of the present disclosure is shown. For each given image of a plurality of images, a segmentation map can be provided (302), the segmentation map including values ​​representing labels indicating defect-related segments corresponding to the given image. In addition, for each given image, an additional output representing an image-level defect presence indication can be provided (304) based on features extracted from the given image. In such a case, a set of images labeled with detected defects can be selected (306) based on both the segmentation map and the additional output. For example, such selection can be based on consistency between the two outputs.

[0092] Figure 6 A schematic diagram of an exemplary first ML model providing multiple outputs according to certain implementations of the subject matter of the present disclosure is shown.

[0093] The first ML model in Figure 6 exemplified as U-net in the figure. U-net is a type of CNN. It includes a contraction path (represented as the encoder part in the figure) and an expansion path (represented as the decoder part in the figure), the expansion path being more or less symmetrical to the contraction path, resulting in a U-shaped architecture. A given image (e.g., one of a plurality of captured images) can be provided to an input layer 602 of the network, and after processing, a score map 604 can be output, the score map 604 including scores indicating pixel-level probabilities of defects being present in the given image. As described above, the score map 604 can be processed to provide a segmentation map, where a value of 1 indicates a defective segment to which the corresponding pixel in the given image belongs, and a value of 0 indicates a non-defective segment to which the corresponding pixel in the given image belongs.

[0094] In some cases, in addition to the output of the score map or segmentation map, an additional output 606 may be provided based on a feature map extracted from at least one of the convolutional layers in or after the encoder portion. As illustrated, the feature map extracted from the convolutional layer 608 may be processed via a classification head 610 to obtain an additional output 606 representing an image-level defect presence indication. For a given input image, if both the output of the segmentation map and the additional output indicate the presence of a defect, then a given image marked with the defect indicated by the segmentation map may be selected to be included in the set of images, as described above with respect to block 204.

[0095] In some further embodiments, an output of an uncertainty map of the score map 604 and / or the segmentation map may be generated in addition to or in lieu of the additional output 606. The uncertainty map includes values ​​representing the level of uncertainty of the defect-related segments represented by the segmentation map or the score map, and the uncertainty map may be generated in parallel with the score map, for example, as an additional output from a decoder of the ML model. In such a case, a set of images marked with detected defects may be selected based on both the segmentation map and the uncertainty map. As an example, a given image having pixels where the segmentation map indicates that the pixels are defective with a high degree of certainty indicated by the uncertainty map will be selected, where these pixels are marked as detected defects.

[0096] In some cases, multiple score maps may be generated, and a total score map may be derived based on the multiple score maps (e.g., as an average of the score maps). As an example, defect detection using a first ML model (at the inference stage) may be performed multiple times, and multiple score maps may be obtained, for example, by randomly applying a dropout of some outputs of some layers in inference. Alternatively, several ML models (e.g., segmentation models) may be trained with different subsets of the training set, which may be used to generate corresponding score maps at runtime. Another possible way to obtain several score maps is by having a multi-input-multi-output (MIMO) segmentation network, in which each input image is mapped to a score map, in which some hidden layers are shared when different score maps are generated. During training, different images are input into the network, resulting in different score maps, while in inference, the same image is used in all inputs, resulting in multiple score maps. The total score map may be used to derive a segmentation map. In the case of using uncertainty maps, the uncertainty maps may be generated as the standard deviation or entropy of each pixel of these multiple score maps. In a similar manner as described above, the selection of the set of images can be based on the segmentation map obtained from the total score map and the uncertainty map.

[0097] The first ML model used in the process of block 204 is a trained ML model. Figure 5 A training process for a first ML model according to certain embodiments of the subject matter of the present disclosure is shown. A first training set of images can be obtained (502), the first training set comprising a subset of synthetic defective images and a subset of nominal images, each synthetic defective image containing one or more synthetic defects. In some cases, the first training set can optionally further comprise a subset of real defective images (i.e., actually captured defective images, rather than synthetically generated defective images). The first training set of images can be used to pre-train (504) the first ML model during a training / setup phase.

[0098] Figure 4 A generalized flow chart for generating a subset of synthetic defective images according to certain embodiments of the subject matter of the present disclosure is shown. One or more defects may be injected (402) into design data of a sample, and a synthetic defective image may be simulated (404) based on the design data with the injected defects. The design data may refer to original design layout data of the sample, such as CAD data. In some cases, the design data of the sample may be provided by a customer. In the event that the design data is not available, the design data may be generated based on an actual image of the sample (e.g., a SEM image captured by a SEM tool). For example, the SEM image may be analyzed, and feature extraction may be performed on the image. For example, the extracted features may represent the structure and / or pattern and / or style of any structural features in the image, such as edges, corners, pixel intensities, measurements associated therewith, etc. The design data may be simulated based on the extracted features, such as a design image (e.g., a CAD image) corresponding to the SEM image. Alternatively, defects may be injected directly onto the actual SEM image of the sample by, for example, manipulating features extracted from the SEM image and decoding them into a defective image using an image conversion network.

[0099] One or more candidate regions may be found in the design data based on the type of defect to be injected. As an example, where the defect to be injected is a bridge, a candidate region of a pattern comprising two structures with a gap (e.g., two parallel line structures) may be identified. In some cases, feature-based matching techniques based on comparison of characteristics of the features / structures may be used to identify candidate regions, particularly with respect to patterns of the same type / property that may vary in terms of geometric transformations (e.g., scaling, offset, rotation, etc.), grayscale intensity, and / or contrast changes. For example, such techniques may be used to identify candidate regions comprising parallel line structures having different line widths and / or different distances between lines.

[0100] The defect to be injected may be integrated into the candidate region of the identified design data. According to certain embodiments, one or more image characteristics of the defect feature or a portion thereof may be manipulated before injecting the defect feature into the design. As an example, the manipulation may be based on at least one of: geometric transformation (including translation, rotation, scaling, shear mapping, etc.), grayscale intensity modification, style transfer, etc. The defect feature may be adjusted in one or more of these aspects according to the characteristics of the candidate region, and the adjusted feature may be injected into the candidate region.

[0101] Once defects are injected into the design data, image simulations may be performed based on the design data with the injected defects to generate a synthetic defective image.

[0102] According to some embodiments, image simulations may be performed to simulate one or more physical effects caused by one or more physical processes of a semiconductor sample, thereby generating a set of synthetic defective images. According to some embodiments, these effects may refer to changes caused by one or more of the following physical processes: a production / fabrication process (FP) for manufacturing the sample (e.g., printing a design pattern of the sample on a wafer by a photolithography tool), a scanning process for scanning the manufactured sample, and a signal processing process for processing the scanned signal to generate a synthetic defective image (e.g., in an inspection process by an inspection tool), etc.

[0103] For example, the physical effects caused by the manufacturing process (FP) of the sample can be simulated to represent how the design pattern in the design data will actually appear on the wafer. In other words, the FP simulation transfers the design intent layout to the expected processing pattern on the wafer. As another example, in the case of inspection by an electron beam tool such as a SEM, the physical effects of the scanning process can be simulated by representing the electron beam signal based on the yield of electrons emitted from the sample and detected by the detector. As yet another example, the signal processing simulation reflects the impact of the signal processing path in the inspection tool on the electron beam signal. In some cases, process variations (PV) and grayscale (GL) variations can be considered during such simulations. A simulated image, such as a simulated SEM image, is obtained after performing one or more of the above simulations.

[0104] According to certain embodiments, image simulation can be performed based on machine learning. For example, an image conversion model trained to convert design data into a synthetic image can be used. The image conversion model can be implemented using various ML model architectures, such as CNN, autoencoder, GAN, etc. As an example, a set of training samples can be used to pre-train the image conversion model, each training sample including a design image and a corresponding actual image, such as a SEM image. The model can be trained to learn to map between two image representations. After being trained, when given design data with injected defects as input, the model can output a synthetic defective image corresponding thereto.

[0105] The synthetic defective images generated as described above effectively enrich the training set used to train the first ML model and solve the problem of lack of defective training images and their annotation efforts. When the first ML model is used for runtime inspection to process actual captured images, it is proved that the first ML model trained using such a training set has high detection performance, especially in terms of detection accuracy (i.e., the percentage of detected defects that are real defects, rather than false alarms). As an example, the first ML model trained using a subset of several hundred synthetic defective images (each synthetic defective image contains one or more synthetic bridge defects) and a subset of several hundred nominal images can achieve an accuracy rate of 90% or more, indicating that the defects detected by the first ML model have a very high probability of being real defects, rather than false alarms.

[0106] Therefore, detected defects detected from real images (therefore, maintaining natural image characteristics and various variations of defects occurring in actual production compared to synthetic defective images) and proven to be real defects with a high probability can be directly used as high-quality defect training samples without further review / verification by human reviewers. In some cases, additional filtering mechanisms (such as additional outputs of the classification head, and / or uncertainty maps as described above) can be applied to filter / select the set of images to be used as training samples. Training using such defective training samples is also referred to as "privileged learning". In some cases, training of the second ML model using such defective training samples can also be referred to as the second stage of the learning process relative to the first stage, and the first stage refers to the training of the first ML model using synthetic defective images.

[0107] In some embodiments, the second stage of training can be repeated multiple times, each stage using defective training samples, which are real defect images detected by one or more ML models obtained from one or more previous stages. In such a case, the ML model obtained from the final stage can be referred to as the final ML model, which can be used for runtime defect detection. The present disclosure is not limited by the number of learning stages used to obtain the final ML model. For example, when the number of learning stages is 2, the second ML model can be regarded as the final ML model.

[0108] Return to reference Figure 2Once the set of images labeled with detected defects is obtained, a second ML model (e.g., second ML model 108) can be trained (206) using a second training set (e.g., by training module 106), the second training set including at least a portion of the set of images labeled with detected defects. Regarding the nominal training images, in some cases, the second training set can include a subset of the nominal images (which is part of the first training set) or a portion thereof. In some other cases, the nominal training images can be created based on the set of images labeled with detected defects. For example, when the detected defects are labeled in the set of images, the images can be cropped according to the labeling, and the portion of the image that does not contain any defect features can be used as the nominal training image.

[0109] The second ML model, after being trained, can be used for runtime defect detection, and is proven to have improved detection performance compared to the first ML model. The improved detection performance can be represented by at least one of the following detection measures: precision, recall (also known as capture rate, the percentage of detected defects relative to the total number of defects), filter rate, and false alarm rate (the percentage of false alarms among detected defects). As an example, it is proven that the detection accuracy of the second ML model is further improved compared to the first ML model. For example, the accuracy of the second ML model trained using the same subset of defects and nominal images detected by the first ML model can reach 95% to 99%, compared to the approximately 90% accuracy of the first ML model.

[0110] By way of another example, the detection recall of the second ML model is significantly improved compared to the first ML model. The recall rate (i.e., capture rate) of the first ML model in the above example is approximately in the range of 70% to 80%, indicating that although the first ML model can capture a large percentage of the total number of defects, it still misses approximately 20 to 30% of the defects. This may be due to the fact that the synthetic defects used to train the first ML model are not sufficient to represent the different types of variations of actual defects. On the other hand, the second ML model trained using defects detected by the first ML model (the defects detected by the first ML model are generated from real images and have a high probability of being real defects) can manage to achieve a recall rate of 90% or above, indicating that the second ML model is able to capture most of the defects that exist.

[0111] In some embodiments, the second ML model may be based on the first ML model. As an example, the second ML model may be implemented using the trained first ML model or a portion thereof, thereby benefiting from the previous training results. For example, the trained first ML model may be used as an initial version of the second ML model, and the second ML model will be trained as described with respect to box 206. In such a case, the training of the second ML model is actually a retraining process of the first ML model. The trained second ML model may also be referred to as a retrained first ML model with updated training parameters. In some cases, the second ML model may be initialized based on only a portion of the training parameters of the trained first ML model. In such a case, which portion of the model parameters of the first ML model is used to initialize the second ML model may be further configured.

[0112] In some further implementations, the second ML model may be a different model from the first ML model. For example, a new ML model may be employed, and training with respect to block 206 may be performed for the new ML model.

[0113] As described above, the training process for the second ML model may be performed in a similar manner as that for the first ML model, and the details thereof will not be repeated here for the sake of brevity of description.

[0114] Reference now Figure 7 , showing a schematic diagram of employing a first ML model and a second ML model in a defect inspection system according to certain embodiments of the subject matter of the present disclosure.

[0115] Diagram 701 shows how Figure 7 The training phase and the inference phase of the illustrated first ML model 700. To train the first ML model 700, a first training set of images is obtained, the first training set including a subset of synthetic defective images 704 and a subset of nominal images 702 without defective features, each synthetic defective image containing one or more synthetic defects 706. As shown in reference Figure 4 As described, the synthetic defective image 704 may be generated by injecting one or more defects into the design data of the sample, or into an image of the sample itself, and performing an image simulation based on the design data with the injected defects.

[0116] For each given training image, the first ML model 700 can process it and provide a predicted detection output 708 indicating whether there are any detected defects in the given image. As an example, as described above, the predicted detection output 708 can be in the form of a segmentation map. A loss function 712 can be used to evaluate the predicted detection output 708 relative to the corresponding ground truth 710 of the given training image, which indicates the marked defects of the given training image (represented by ellipses for defective images, while nominal images are not marked). The loss function 712 can represent the difference between the predicted detection output and the ground truth associated with the corresponding training image. The ML model can be optimized by minimizing the value of the loss function. Training can be determined to be complete when the value of the loss function is less than a predetermined value, or when a limited change in performance between iterations is achieved.

[0117] After being trained, the trained first ML model 720 can be deployed in inference for runtime defect inspection. Multiple images 722 of samples actually captured by the inspection tool during runtime inspection can be processed by the trained first ML model 720, and for each given image, a segmentation map 724 can be output, which includes labels indicating defective segments or non-defective segments. As described above, a segmentation map can be generated at the pixel level, where each label corresponds to a pixel in a given image and indicates the segment to which the corresponding pixel belongs. A segmentation map can also be generated at a structure level or a region level, where each label corresponds to a structure or a region in a given image and indicates the segment to which the corresponding structure or region belongs.

[0118] By using the segmentation map, a set 726 of images can be selected from the plurality of images 722, each image in the set 726 of images being associated with a segmentation map including one or more markers indicating the presence of a defect. As described above, in some cases, an additional output 728 can be extracted based on an output feature map extracted from a convolutional layer (e.g., via a classification head). The additional output 728 can represent an image-level indication of the presence of a defect. The set 726 of images can be selected based on both the segmentation map 724 and the additional output 728. For example, for a given input image, if both the output of the segmentation map and the additional output indicate the presence of a defect, a given image marked with a defect indicated by the segmentation map can be selected to be included in the set of images. In some cases, as an addition or alternative to the additional output 728, an uncertainty map can be derived based on the score map and / or the segmentation map, and the uncertainty map can be used to select a set of images, as described above.

[0119] Diagram 703 shows the training process for the second ML model 730. The set 726 of images labeled with defects, together with the set of nominal images (e.g., as part of the nominal images 702, or derived from the set 726 of images), can be used as a second training set to train the second ML model 730. Similarly, for each training image, the second ML model can output a predicted detection output 732, which will be evaluated using a loss function 734 relative to the corresponding ground truth labeling of the training image, and the second ML model can be optimized by minimizing the value of the loss function.

[0120] The second ML model 730 can be used for runtime defect inspection after being trained. As described above, it has been demonstrated that the second ML model has improved defect detection performance compared to the first ML model 720.

[0121] It should be noted that the first ML model and / or the second ML model may be implemented as various types of ML models, and Figure 7 The models are shown for exemplary purposes only.

[0122] According to certain embodiments, the above reference Figure 2 The described process may be included as part of a process for generating an inspection recipe that the inspection system 100 and / or the inspection tool 120 may use for inline inspection at runtime. After being trained, the trained second ML model may be included as part of the inspection recipe. It should be noted that the term "inspection recipe" should be interpreted broadly to encompass any recipe that can be used by an inspection tool for any inspection application (e.g., a defect detection application or a defect review application, including the embodiments described above).

[0123] It should be noted that the examples shown in the present disclosure, such as exemplary ML models and structures, training data and processing, inspection tools, image simulation methods, etc., are shown for exemplary purposes and should not be considered to limit the present disclosure in any way. Other appropriate examples / implementations may be used in addition to or instead of the above.

[0124] One of the advantages of certain embodiments of the defect inspection systems and methods described herein is that using the two-stage learning process as described above (or the multi-stage learning process as described above) can result in a trained ML model (i.e., a trained second ML model) that is capable of performing defect inspection with significantly improved detection performance.

[0125] This is achieved at least in part by the fact that the second ML model is trained using high quality defective training samples that are selected based on the processing of the first ML model and are proven to have a high probability of being real defects. In addition, these defective training samples are selected from real images, thus maintaining natural image characteristics and the variety of defects that occur in actual production compared to synthetic defective images.

[0126] As an example, the precision of a second ML model trained using defects detected by the first ML model is very high, and can reach 95% to 99%. In addition, the recall of the second ML model can reach 90% or higher, which represents an improvement over the recall of the first ML model, indicating that the second ML model is able to capture most of the defects that exist.

[0127] One of the additional advantages of certain embodiments of the process monitoring systems and methods described herein is that a set of images labeled with detected defects for training a second ML model can be selected based on the output of the segmentation map and an additional output representing an image-level indication of defect presence based on features extracted from a given image. The selection and verification based on the consistency of the two outputs further increases the probability that the selected images contain true defects rather than false alarms.

[0128] It is to be understood that the disclosure is not limited in its application to the details set forth in the description contained herein or illustrated in the drawings.

[0129] It should also be understood that the system according to the present disclosure can be implemented at least in part on a suitably programmed computer. Likewise, the present disclosure contemplates a computer program readable by a computer for performing the method of the present disclosure. The present disclosure further contemplates a non-transitory computer readable memory for performing the method of the present disclosure, the non-transitory computer readable memory tangibly containing a program of instructions executable by a computer.

[0130] The present disclosure can have other embodiments and can be practiced and executed in various ways. Therefore, it should be understood that the words and terms used herein are for descriptive purposes and should not be considered as limiting. Therefore, it will be understood by those skilled in the art that the concepts on which the present disclosure is based can be easily used as the basis for designing other structures, methods and systems for several purposes of achieving the subject matter of the present disclosure.

[0131] Those skilled in the art will readily appreciate that various modifications and changes may be made to the embodiments of the present disclosure described above without departing from the scope of the present disclosure as defined by the appended claims.

Claims

1. A computerized system for semiconductor sample inspection, the system comprising processing circuitry configured to: obtaining a plurality of images of a semiconductor sample acquired by an inspection tool; processing the plurality of images using a first machine learning (ML) model for defect detection, thereby obtaining a set of images labeled with detected defects from the plurality of images, wherein the first ML model is pre-trained using a first training set, the first training set comprising a subset of synthetic defective images and a subset of nominal images, each synthetic defective image comprising one or more synthetic defects; and A second ML model is trained using a second training set, the second training set comprising at least a portion of the set of images labeled with detected defects, wherein the second ML model can be used for defect detection after being trained, and the second ML model has improved detection performance relative to the first ML model.

2. The computerized system of claim 1, wherein the second ML model is based on the first ML model.

3. The computerized system of claim 1, wherein the second ML model is a new ML model different from the first ML model.

4. The computerized system of claim 1, wherein the processing include: for each given image in the plurality of images, providing a segmentation map, the segmentation map comprising values ​​representing labels indicating defect-related segments corresponding to the given image; and selecting a set of images from the plurality of images, each image in the set of images being associated with a segmentation map comprising one or more indicia indicative of the presence of a defect.

5. The computerized system of claim 4, wherein the processing further include: For each given image, an additional output is provided based on features extracted from the given image, the additional output representing an image-level defect presence indication, and wherein the selecting comprises selecting a set of the images marked with detected defects based on consistency between the segmentation map and the additional output.

6. The computerized system of claim 4, wherein the processing further include: For each given image, an uncertainty map is provided, the uncertainty map comprising values ​​representing respective uncertainties of the defect-related segments indicated by the segmentation map, and wherein the selecting comprises selecting a set of the images marked with detected defects based on the segmentation map and the uncertainty map.

7. The computerized system of claim 1, wherein the subset of the synthetic defective images is generated by injecting one or more defects in design data of the sample and simulating the synthetic defective images based on the design data with the injected defects.

8. The computerized system of claim 7 , wherein the simulation is performed by simulating physical effects of one or more processes, the one or more processes comprising a manufacturing process for manufacturing a sample based on the design data, a scanning process for scanning the manufactured sample, and a signal processing process for processing the scanned signal to generate the synthetic defective image.

9. The computerized system of claim 7, wherein the simulation is performed using an image conversion model trained to convert design data of a sample into a synthetic image.

10. The computerized system of claim 7, wherein the design data is generated based on an actual image of the sample.

11. The computerized system of claim 1, wherein the first training set further comprises a subset of real defective images.

12. The computerized system of claim 1, wherein the improved detection performance is represented by at least one of the following detection measures: precision, recall, filter rate, and false alarm rate.

13. A computerized method for inspecting a semiconductor sample, the method include: obtaining a plurality of images of a semiconductor sample acquired by an inspection tool; processing the plurality of images using a first machine learning (ML) model for defect detection, thereby obtaining a set of images labeled with detected defects from the plurality of images, wherein the first ML model is pre-trained using a first training set, the first training set comprising a subset of synthetic defective images and a subset of nominal images, each synthetic defective image comprising one or more synthetic defects; as well as A second ML model is trained using a second training set, the second training set comprising at least a portion of the set of images labeled with detected defects, wherein the second ML model can be used for defect detection after being trained, and the second ML model has improved detection performance relative to the first ML model.

14. The computerized method of claim 13, wherein the second ML model is based on the first ML model.

15. The computerized method of claim 13, wherein said processing include: for each given image in the plurality of images, providing a segmentation map, the segmentation map comprising values ​​representing labels indicating defect-related segments corresponding to the given image; and selecting a set of images from the plurality of images, each image in the set of images being associated with a segmentation map comprising one or more indicia indicative of the presence of a defect.

16. The computerized method of claim 15, wherein said processing further include: For each given image, an additional output is provided based on features extracted from the given image, the additional output representing an image-level defect presence indication, and wherein the selecting comprises selecting a set of the images marked with detected defects based on consistency between the segmentation map and the additional output.

17. The computerized method of claim 15, wherein said processing further include: For each given image, an uncertainty map is provided, the uncertainty map comprising values ​​representing respective uncertainties of the defect-related segments indicated by the segmentation map, and wherein the selecting comprises selecting a set of the images marked with detected defects based on the segmentation map and the uncertainty map.

18. The computerized method of claim 13, wherein the subset of the synthetic defective images is generated by injecting one or more defects in design data of the sample and simulating the synthetic defective images based on the design data with the injected defects.

19. The computerized method of claim 13, wherein the improved detection performance is represented by at least one of the following detection measures: precision, recall, filter rate, and false alarm rate.

20. A non-transitory computer-readable storage medium tangibly embodying a program of instructions which, when executed by a computer, causes the computer to perform a method of inspecting a semiconductor sample, the method include: obtaining a plurality of images of a semiconductor sample acquired by an inspection tool; processing the plurality of images using a first machine learning (ML) model for defect detection, thereby obtaining a set of images labeled with detected defects from the plurality of images, wherein the first ML model is pre-trained using a first training set, the first training set comprising a subset of synthetic defective images and a subset of nominal images, each synthetic defective image comprising one or more synthetic defects; as well as A second ML model is trained using a second training set, the second training set comprising at least a portion of the set of images labeled with detected defects, wherein the second ML model can be used for defect detection after being trained, and the second ML model has improved detection performance relative to the first ML model.

Images