Method and system for classifying defects in wafer by using wafer defect image based on deep learning

Abstract

The invention provides a method and a system 100 for classifying defects in a wafer by using wafer defect images based on a deep learning network. Embodiments herein use collaboration among multiple modes of wafer defect images to make classification decisions. Furthermore, information, such as a color image, an ICI image, a black-and-white image and the like, can be obtained from different sources by mixing the additive modes, so that defect images can be classified. In addition to the mixed mode, a reference image may be used for each mode. The reference image of each pattern image is provided to a deep learning model to focus on the defect itself instead of the associated underlying lithography of the defect image. Furthermore, the reference image can be provided to a training process of the deep learning model, which can significantly reduce the number of marked images and the training time for the deep learning model to converge.

Description

[0001] Case division information

[0002] This application is a divisional application of the invention patent application with application number 202010831725.0 filed on August 18, 2020, and invention name: “Method and system for classifying defects in wafers using wafer defect images based on deep learning”.

[0003] Related applications

[0004] This application claims the benefit of and priority to U.S. Provisional Application No. 63 / 015,101, filed on April 24, 2020. Technical Field

[0005] The present invention generally relates to neural networks for semiconductor applications. Specifically, but not limited to, the present invention relates to a method and system for classifying defects in wafers using wafer defect images based on a deep learning network. Background Art

[0006] Generally speaking, semiconductor substrate (i.e., wafer) manufacturing technology has been continuously improved to incorporate an ever-increasing number of features and multiple layers of semiconductor elements into a smaller surface area of a semiconductor wafer. As a result, photolithography (lithography) processes can be used in semiconductor wafer manufacturing, which are more complex, thereby allowing more and more features to be incorporated into a smaller area of the semiconductor wafer (i.e., to achieve higher performance of the semiconductor wafer). As a result, the size of potential defects on a semiconductor wafer can range from microns to submicrons due to the addition of more and more features. In addition, defects in a wafer can be, for example, defects caused by real and physical phenomena of the wafer, as well as false events / nuisance defects (i.e., nuisances can be irregularities or false defects on the wafer, but are not defects of interest).

[0007] Typically, defects in semiconductor wafers can be detected based on obtaining a higher resolution image of the wafer using at least one of a high magnification optical system or a scanning electron microscope (SEM). In order to determine parameters such as the thickness, roughness, size, etc. of the defects, a high-resolution image of the silicon wafer defect can be generated. In addition, a conventional system discloses an imaging system that can be configured to scan a multimode energy source (e.g., light or electrons) on a physical version of the wafer, and thereby generate an actual image of the physical version of the wafer. In addition, the defect area can be determined by comparing the defect image with a reference image for anomaly detection and defect classification. Conventional systems can use a single deep learning model to detect and classify defects in wafers. However, conventional systems may not be able to accurately determine defects in wafers by considering different aspects / modes of the defect image corresponding to the wafer. Summary of the Invention

[0008] The present invention provides a method and system for classifying defects in a wafer using wafer defect images based on a deep learning network. The embodiments herein use synergy between multiple modes of wafer defect images to make classification decisions. In addition, by adding a mixture of modes, information can be obtained from different sources, such as: color images, internal crack imaging (ICI) images, black and white images, etc., to classify the defect images. In addition to the mixture of modes, a reference image (e.g., a golden die image) can be used for each mode. The advantage of providing a reference image for each mode image is that it focuses on the defect itself rather than the associated underlying lithography of the defect image. In addition, the reference image can be provided to the training process of the deep learning model, which can significantly reduce the number of labeled images and the training time required for the deep learning model to converge (i.e., when the entire data set is passed forward and backward through the deep learning neural network).

[0009] The embodiments herein can use a directed acyclic graph (DAG) as a combination of deep learning models, and each deep learning can use defective wafer images to process different aspects of the problem or different forms of defects in the wafer. In addition, the created DAG can have any number of models and multiple different images for each deep learning model. In addition, the post-processing decision module can be configured to combine parameters, such as two aspects of the defect survey image and the result label of the defect survey image, the value of each deep learning model from the DAG, and the metrology information (metadata) of the defect or the parameters previously collected in the scanner machine. Based on the deep learning network, the DAG including the deep learning model can be used to accurately classify wafer defects using wafer defect images.

[0010] The features disclosed in this invention facilitate accurate defect detection and classification of defects in wafers during manufacturing by analyzing multiple patterns in wafer defect images.

[0011] In one aspect, a computer-implemented method for classifying and inspecting defects in a semiconductor wafer includes: providing one or more imaging units; providing a computing unit; receiving a plurality of images taken from one or more die on a semiconductor wafer inspected by the one or more imaging units, wherein the plurality of images are captured using a plurality of imaging modes; providing one or more machine learning units; a method for detecting defects in a semiconductor wafer by imaging the one or more wafers; and a method for detecting defects in the semiconductor wafer by imaging the one or more wafers. The method further comprises: providing the one or more ML models with a plurality of labeled images and a plurality of reference images of the semiconductor wafer stored in the database to the one or more ML models; configuring each ML model from the plurality of ML models to classify the labeled images into one or more defect categories using a corresponding reference image from the plurality of reference images; storing the one or more defect categories; inspecting the one or more wafers on the semiconductor wafer for defects by imaging the one or more wafers; attempting to match the images of the one or more wafers with any one or more of the one or more defect categories; and classifying the one or more matching wafers as defects if a match exists between the one or more wafers and the one or more defect categories, and transmitting identifications of the one or more defective wafers and rejecting them as defects.

[0012] In another aspect, the one or more ML models have a plurality of images from a plurality of imaging modalities and a plurality of labeled images belonging to the imaging modalities. In addition, each of the plurality of ML models is one of a supervised model, a semi-supervised model, and an unsupervised model.

[0013] In a further aspect, the plurality of modalities includes at least one of: X-ray imaging, internal crack imaging (ICI), grayscale imaging, black and white imaging, and color imaging. In addition, the plurality of ML models are deep learning models.

[0014] In another aspect, the plurality of labeled images include labels associated with the one or more defect categories, wherein the plurality of labeled images are generated using a labeling model.

[0015] In an additional aspect, the computing unit described herein includes one or more processors and memory configured to perform the above-described method steps.

[0016] In one aspect, a method for classifying defects in a semiconductor wafer includes: capturing multiple images of a semiconductor wafer inspected by one or more imaging units, wherein the multiple images are captured using multiple imaging modes; providing the multiple images from multiple ML models to one or more machine learning (ML) models thereof to identify one or more defects in the semiconductor wafer and classify them into one or more defect categories, wherein the multiple ML models are configured in a directed acyclic graph (DAG) architecture, wherein each node in the DAG architecture represents an ML model, wherein the one or more ML models are configured as root nodes in the DAG architecture; wherein the multiple ML models are trained to classify the one or more defects in the semiconductor wafer, and wherein the training includes: providing the one or more ML models from the multiple ML models with multiple labeled images and multiple reference images of the semiconductor wafer stored in a database; and, configuring each ML model from the multiple ML models to classify the multiple labeled images into one or more defect categories using a corresponding reference image from the multiple reference images.

[0017] In another aspect, the one or more ML models are fed with a plurality of images from a plurality of imaging modalities and a plurality of labeled images belonging to a single imaging modality. Furthermore, each of the plurality of ML models is one of a supervised model, a semi-supervised model, and an unsupervised model. The plurality of modalities includes at least one of: X-ray imaging, internal crack imaging (ICI), grayscale imaging, black and white imaging, and color imaging. Each of the plurality of ML models is a deep learning model.

[0018] In another aspect, the plurality of labeled images include labels associated with the one or more defect categories, wherein the plurality of labeled images are generated using historical images of the semiconductor wafer. Features extracted from the plurality of modalities are combined using one of a late fusion technique, an early fusion technique, or a hybrid fusion technique. Post-processing is further included, wherein the post-processing includes accurately classifying the plurality of images into the one or more defect categories using classification information from each of the plurality of ML models.

[0019] In one aspect, a system for classifying and inspecting defects in semiconductor wafers includes: one or more imaging units configured to capture a plurality of images of one or more wafers on a semiconductor wafer inspected by the one or more imaging units, wherein the plurality of images are captured using a plurality of imaging modes; a computing unit comprising at least a computer processor, a database, and a memory, and configured to: provide a plurality of images from a plurality of ML models to one or more machine learning (ML) models to identify and classify more defects in the one or more wafers on the one or more machine learning (ML) semiconductor wafers into one or more defect categories, wherein the plurality of ML models are configured in a directed acyclic graph (DAG) architecture, wherein each node in the DAG architecture represents an ML model, wherein the one or more ML models are configured as root nodes in the DAG architecture, and the plurality of ML models are configured to be trained to classify one or more defects on the one or more wafers in the semiconductor wafer, wherein the computing unit is configured to: provide a plurality of labeled images and a plurality of reference images of the semiconductor wafer stored in the database to the one or more ML models from the plurality of ML models;

[0020] Each ML model from the plurality of ML models is configured to classify the plurality of labeled images into one or more defect classes using corresponding reference images from the plurality of reference images; and then, based on the presence of a match between the one or more wafers and the one or more defect classes, respectively, storing the defect classes and accepting or rejecting the one or more wafers upon inspection.

[0021] In another aspect, the one or more imaging units include automated optical inspection (AOI) equipment, automated X-ray inspection (AXI) equipment, joint test group (JTAG) equipment, and in-circuit test (ICT) equipment. Furthermore, the computing unit receives a plurality of labeled images including labels associated with one or more defect categories from a labeling model, wherein the labeling model generates the plurality of labeled images using historical images of the semiconductor wafer.

[0022] In yet another aspect, features extracted from the multiple modalities are combined using one of a late fusion technique, an early fusion technique, or a hybrid fusion technique. The computing unit is further configured to post-process outputs of the multiple ML models, wherein the computing unit uses classification information from each of the multiple ML models to accurately classify the multiple images into one or more defect categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The invention itself, as well as the preferred mode of use, further objects and advantages thereof, will be best understood by reference to the following detailed description of illustrative embodiments when read in conjunction with the accompanying drawings. One or more embodiments will now be described with reference to the accompanying drawings, in which:

[0024] Figure 1 A block diagram of a system for classifying defects in a wafer using wafer defect images based on a deep learning network according to some embodiments of the present invention is shown;

[0025] Figure 2 A block diagram of a multimodal late fusion deep learning model according to some embodiments of the present invention is shown, which can be used as one of the deep learning models for classifying defects in a wafer using wafer defect images;

[0026] Figure 3 A block diagram of a multi-modal hybrid fusion deep learning model according to some embodiments of the present invention is shown, which can be used as one of the deep learning models for classifying defects in a wafer using wafer defect images;

[0027] Figure 4 A block diagram of a multimodal early fusion deep learning model according to some embodiments of the present invention is shown, which can be used as one of the deep learning models for classifying defects in a wafer using wafer defect images;

[0028] Figure 5a A schematic diagram illustrating a DAG topology using a series of deep learning models according to some embodiments of the present invention;

[0029] Figure 5b A schematic diagram showing exemplary result labels from each deep learning model defining a flow path in a DAG according to some embodiments of the present application is shown;

[0030] Figure 6a is a process describing a method for classifying defects in a wafer using wafer defect images based on a deep learning network according to some embodiments of the present invention; and

[0031] Figure 6b is a flow chart describing a method for calculating features representing defect metadata if defect metadata of a wafer defect image is not stored in an electronic device, according to some embodiments of the present invention.

[0032] The drawings depict embodiments of the present invention for purposes of illustration only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods shown herein may be used without departing from the principles of the disclosure described herein. DETAILED DESCRIPTION

[0033] The foregoing generally summarizes the features and technical advantages of the present invention in order to better understand the following detailed description of the present invention. It should be understood by those skilled in the art that the disclosed concepts and specific embodiments can be easily used as the basis for modifying or designing other structures for achieving the same purpose of the present invention.

[0034] The technical features of the present invention which are believed to be novel, including their organization and method of operation, as well as further objects and advantages, will be better understood from a consideration of the accompanying drawings hereinafter. However, it is to be expressly understood that each of the figures is provided for illustration and description purposes only and is not to be construed as a definition of the limits of the present invention.

[0035] Figure 1 A block diagram of a system 100 for classifying defects in a wafer using wafer defect images based on a deep learning network is shown, according to some embodiments of the present invention.

[0036] Throughout the present invention, the term "wafer" generally refers to a substrate formed of semiconductor or non-semiconductor materials. For example, the semiconductor or non-semiconductor materials may include, but are not limited to, single crystal silicon, gallium arsenide, indium phosphide, etc. A wafer may include one or more layers, and the layers may include, for example, but are not limited to, resists, node materials, conductive materials, semiconductor materials, etc. For example, one or more layers formed on a wafer may be patterned or unpatterned. For example, a wafer may include multiple chips, each chip having repeatable pattern features. The formation and processing of these material layers may result in a complete device. In addition, as used herein, the term "surface defect" or "defect" refers to defects that are completely above the upper surface of the wafer (e.g., particles) and defects that are partially below the upper surface of the wafer or completely below the upper surface of the wafer. Therefore, the classification of defects is particularly useful for semiconductor materials such as wafers and materials formed on wafers. Furthermore, distinguishing between surface and subsurface defects can be particularly important for bare silicon wafers, silicon-on-insulator (SOI) films, strained silicon films, and dielectric films. The embodiments herein can be used to inspect wafers containing silicon or silicon-containing layers formed thereon, such as silicon carbide, carbon-doped silicon dioxide, silicon-on-insulator (SOI), strained silicon, silicon-containing dielectric films, and the like.

[0037] exist Figure 1In one embodiment, the system 100 includes an imaging device 102 and an electronic device 104. The imaging device 102 is associated with the electronic device 104 via a communication network 106. The communication network 106 can be a wired or wireless network. In one embodiment, the imaging device 102 can be, but is not limited to, at least one of an automated optical inspection (AOI) device, an automated X-ray inspection (AXI) device, a joint test group (JTAG) device, an in-circuit test (ICT) device, and the like. The imaging device 102 includes, but is not limited to, at least one of a light source 108, a camera lens 110, a defect detection module 112, and an image storage unit 126. For example, the defect detection module 112 associated with the imaging device 102 can detect multiple surface feature defects on a wafer, such as, but not limited to, at least one of silicon junctions (i.e., bumps), scratches, stains, and dimensional defects (e.g., opens, shorts, and solder thinning). Furthermore, the defect detection module 112 can also detect incorrect, missing, and misplaced components, as the imaging device 102 is capable of performing all visual inspections.

[0038] Furthermore, the electronic device 104 may be, but is not limited to, at least one of a mobile phone, a smartphone, a tablet, a handheld device, a phablet, a laptop, a computer, a personal digital assistant (PDA), a wearable computing device, a virtual / augmented reality device, an Internet of Things (IoT) device, and the like. The electronic device 104 further includes a storage unit 116, a processor 118, and an input / output (I / O) interface 120. Furthermore, the electronic device 104 includes a deep learning module 122. The deep learning module 122 enables the electronic device 104 to classify defects in a wafer using the wafer defect images obtained from the imaging device 102. The electronic device 104 may also include an application management framework for classifying defects in a wafer using a deep learning network. The application management framework may include various modules and sub-modules to perform operations for classifying wafer defects using wafer defect images based on the deep learning network. Furthermore, the modules and sub-modules may include at least one or both of software modules and hardware modules.

[0039] Thus, embodiments described herein are configured for image-based wafer process control and yield improvement. For example, one embodiment herein relates to a system and method for classifying defects in a wafer using wafer defect images based on a deep learning network.

[0040] In one embodiment, the imaging device 102 can be configured to capture images of a wafer placed therein. For example, the images can include at least one of inspection images, optical or electron beam images, wafer inspection images, optical and SEM-based defect inspection images, simulation images, clips from a design layout, and the like. Furthermore, the imaging device 102 can then be configured to store the captured images in an image storage unit 126 associated with the imaging device 102. In one embodiment, the electronics 104 communicatively coupled to the imaging device 102 can be configured to retrieve images stored in the image storage unit 126 associated with the imaging device 102. For example, the images can include black and white images, color images, internal crack imaging (ICI) images, images previously scanned using the imaging device 102 (e.g., an AOI machine, etc.), images from the imaging storage unit 126 or several storage units (not shown), images obtained in real time from the imaging device 102 (e.g., an AOI machine, etc.), and the like. The electronic device 104 is then configured to load, from an external database (not shown) or from a storage unit 116 associated with the electronic device 104, the wafer area of the defect-free inspection image of at least one wafer in the reference image corresponding to at least one of the black and white reference image, the color reference image, and the ICI reference image representing the same scan. Furthermore, the electronic device 104 is configured to provide the reference image and the wafer image with the associated pattern to the deep learning module 122. In one aspect of the present invention, multiple deep learning models or deep learning classifiers can be trained using different types of defect classifications on the wafer. The multiple deep learning models or deep learning classifiers can be, but are not limited to, at least one of a convolutional neural network (CNN) (e.g., LeNet, AlexNet, VGGNet, GoogleNet, ResNet, etc.), a recurrent neural network (RNN), a generative adversarial network (GAN), a random forest algorithm, an autoencoder, etc. The purpose of training multiple deep learning models is to create a synergistic effect of each model to address the concentrated patterns of defects. Therefore, several deep learning models can be established to stratify all defects based on similarity and dissimilarity, distributing the training and classification process. In another embodiment, to shorten the training process and the number of images to be classified, a reference image for each pattern image can be added to the architecture of each deep learning model. The reference image during the training process can provide the deep learning module 122 with information about the internal relationship between the inspected image and the reference image, allowing for faster adjustment or training of the deep learning internal parameters. In addition, if defects appear on different wafers, the deep learning module 122, which has been trained to classify specific defects for a single wafer, can also dynamically classify the training defects. As a result, the training process can discard common events, such as underlying lithography, and can focus on actual defects.

[0041] In one embodiment, the deep learning model can be connected as a parallel architecture or a serial architecture. Further, the electronic device 104 can be configured to use a directed acyclic graph (DAG) architecture of multiple deep learning models to generate classification decisions for wafer images. For example, multiple trained deep learning models can be called from the deep learning module 122 and then connected in a directed acyclic graph (DAG) architecture for classification processing of wafer defects. In addition, the electronic device 104 can be configured to save the classified wafer images including the relevant catalog and metadata results (i.e., defect metadata) in an external database or storage unit 116 associated with the electronic device 104.

[0042] In another embodiment, the electronic device 104 may be configured to load previously calculated and stored defect metadata from an external database or storage unit 116 associated with the electronic device 104. For example, the metadata includes different features, but is not limited to, the size of the defect, the histogram of the defect, the maximum color or grayscale value of the defect, the minimum color or grayscale value of the defect, etc. If the defect metadata is not stored, then the electronic device 104 may be configured to calculate features representing the defect metadata. The electronic device 104 may then be configured to provide the inspected image, the reference image, and the defect metadata (i.e., metadata features of the defect) to the trained deep learning model. Thus, the electronic device 104 may be configured to generate classification decisions for wafer images using a directed acyclic graph (DAG) architecture of multiple deep learning models. In addition, the electronic device 104 may be configured to store the classified wafer images including the associated metadata results (i.e., defect metadata) in the storage unit 116 or external database associated with the electronic device 104.

[0043] Additionally, images and defect metadata may also be stored in an external database (not shown). For example, the external storage may be used for training deep learning models / classifiers. As an example, the images stored in the external database (or the image storage unit 126) may include black and white images, color images, ICI images, images previously scanned by the AOI equipment, images containing wafer defects, false events, and nuisance defects, etc. The images may be labeled before being stored in the external database (or image storage unit 126). For each defect found in an image stored in the external database (or image storage unit 126), a set of metadata features extracted from the defect image is stored in the external database (or image storage unit 126). The metadata defect features may be provided by the user or from AOI scanner results (or the metadata defect features may be created for data retrieval by the deep learning classifier). Furthermore, reference images (e.g., gold wafers), including color reference images, black and white reference images, and / or ICI reference images, may also be stored in the external database. These reference images are images of the same wafer. The external database may also be used to train the deep learning model.

[0044] The embodiments herein use synergy between concentrated patterns of wafer defect images to make classification decisions. In addition, by adding a mixture of patterns, information can be obtained from different sources such as color images, ICI, black and white images, etc. to classify the defect images. In addition to the mixture of patterns, a reference image (e.g., a gold wafer image) can be used for each pattern. The advantage of providing a reference image for each pattern image is that it focuses on the defect itself rather than the associated underlying lithography of the defect image. This approach saves processing power, memory utilization, and time. In addition, the reference images provided to the training process of the deep learning model can significantly reduce the number of labeled images and training time (i.e., when a complete data set is passed forward and backward through the deep learning neural network) required for the fusion of the deep learning model.

[0045] Figure 2 A block diagram of a multimodal late fusion deep learning model according to some embodiments of the present invention is shown, which can be used as one of the deep learning models for classifying defects in wafers using wafer defect images.

[0046] In one embodiment, the electronic device 104 includes a multimodal convolutional neural network (CNN) configured to integrate images acquired by different image sensors in a single forward pass. A deep learning model such as a multimodal late fusion deep learning model can be trained on two sensor images, for example, an ICI image using a first deep learning model and a color image using a second deep learning model. Figure 2As shown, the multimodal CNN model includes CNN models for encoding the color image and the ICI image separately and combining the decisions for both. A trained multimodal post-fusion deep learning model is then used to process each modality, allowing for separate decisions to be made for each. Finally, a central classification layer provides a joint decision based on the different modalities.

[0047] Figure 3 A block diagram of a multimodal hybrid fusion deep learning model according to some embodiments of the present invention is shown, which can be used as one of the deep learning models for classifying defects in wafers using wafer defect images.

[0048] A multimodal CNN model, such as a multimodal hybrid fusion deep learning model, may include a first CNN model for encoding color images, a second CNN model for encoding ICI images, and a third CNN model for jointly representing color and ICI defect images. The third / final CNN model can learn the inter-model relationship between color images and ICI images before making a classification decision.

[0049] Figure 4 Shown is a block diagram of a multimodal early fusion deep learning model according to some embodiments of the present invention, which can be used as one of the deep learning models for classifying defects in wafers using wafer defect images.

[0050] The multimodal early fusion deep learning model may include a CNN model for jointly representing the color defect image and the ICI defect image by simultaneously processing joint feature points in a single multimodal image.

[0051] Figure 5a A schematic diagram illustrating a DAG topology using a series of deep learning models according to some embodiments of the present invention.

[0052] like Figure 5a As shown, multiple deep learning models can be connected into a polytree, which can be a directed acyclic graph (DAG) of the deep learning model, and its underlying undirected graph can be a tree, such as Figure 5a As shown, a multi-modal hybrid fusion deep learning model, a multi-modal early fusion deep learning model, a single input image deep learning model, an autoencoder with one or two input images, and / or a generative adversarial network (GAN) deep learning model. The DAG may include a unique topological order, and each deep learning model may be located on a node of the DAG. In addition, each node may be directly connected to one or more previous nodes, and then connected to one or more nodes. In addition, the result label of each deep learning model defines a flow path in the DAG. For example, Figure 5aAs shown, the result image "label 1" in "model 1" will continue to be evaluated in "model 3".

[0053] As an example, consider the resulting labels of each model in Figure 5b . The resulting label "Label 1: A" from "Model 1" may have a probability value of 0.9, while the probability value of "Label 1: B" may be 0.1. Similarly, the resulting label "Label 2: B" from "Model 3" may have a probability value of 0.2, the probability value of "Label 3: B" may be 0.7, and the probability value of "Label 3: C" may be 0.1. Furthermore, the resulting label "Label 5: A" from "Model 5" has a probability value of 0.1, the probability value of "Label 5: B" is 0.1, the probability value of "Label 5: C" is 0.2, and the probability value of "Label 5: D" is 0.6. Each deep learning model in a DAG can be unique and can be designed to handle a specific part of the classification problem. For example, one deep learning model in a DAG can be a ResNet model, another can be a GoogleNet model, and another can be a multimodal deep learning model. At the end of the DAG path, each image (such as Figure 5a ), where decisions can be made based on the results of a deep learning model interacting with the image.

[0054] Figure 6a Described is a flow chart of a method 600a for classifying defects in a wafer using wafer defect images based on a deep learning network according to some embodiments of the present application.

[0055] At block 601, an image of a wafer is captured by an imaging device 102. At block 602, the captured image is captured by the imaging device 102 ( Figure 1 ) is stored in the image storage unit 126 ( Figure 1 At block 603, the image stored in the image storage unit 126 associated with the imaging device 102 is generated by the electronic device 104 ( Figure 1) for retrieval. In box 604, the electronic device 104 receives at least one reference image corresponding to at least one black and white reference image, a color reference image, and an ICI reference image, which reference image represents the same area of the wafer scanned with the inspected image having no defects in the wafer. In box 605, the electronic device 104 uses a plurality of trained deep learning models / classifiers with relevant expected pattern images from the deep learning module 122 of the electronic device 104. In box 606, the trained multiple deep learning models are connected by the electronic device 104 in a directed acyclic graph (DAG) architecture for classification processing of wafer image defects. In box 607, the electronic device 104 uses the directed acyclic graph (DAG) architecture of the multiple deep learning models to generate a classification decision for the wafer image. Finally, in box 608, the electronic device 104 stores the classified wafer image including the associated metadata results (i.e., defect metadata) in an external database or storage unit 116 of the electronic device 104.

[0056] Figure 6b The flowchart of the method 600 b for calculating features representing defect metadata when the defect metadata of the wafer defect image is not stored in the electronic device 104 according to some embodiments of the present application is described.

[0057] At block 611, the electronic device 104 receives previously calculated and stored defect metadata from an external database or storage unit 116 of the electronic device 104. For example, the metadata includes, but is not limited to, various characteristics of the defect, such as the size of the defect, a histogram of the defect, a maximum color or grayscale value of the defect, a minimum color or grayscale value of the defect, etc. At block 612, if the defect metadata is not stored, the electronic device 104 calculates features representing the defect metadata.

[0058] The embodiments of the present invention can utilize a directed acyclic graph (DAG) as a combination of deep learning models, and each deep learning can use defective wafer images to process different aspects of the problem or different forms of defects in the wafer. In addition, the DAG can create multiple different images (e.g., six images) with any number of models and each deep learning model. In addition, the post-processing decision module can be configured to combine parameters, such as two aspects of the defect survey image and the result label of the defect survey image, the value of each deep learning model from the DAG, and the defect or defect measurement information (metadata) previously collected in the scanner. Based on the deep learning network, the DAG including the deep learning model can be used to accurately classify wafer defects using wafer defect images.

[0059] For the use of substantially any plural and / or singular terms herein, those skilled in the art can shift from the plural to the singular and / or from the singular to the plural, depending on the context and / or application. For clarity, various singular / plural arrangements may be explicitly set forth herein.

[0060] Those skilled in the art will understand that, in general, the terms used herein are generally “open-ended” terms (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “including but not limited to,” etc.). Those skilled in the art will further understand that the specific number of the claims cited is intended. For example, as an aid to understanding, the detailed description may include the use of the introductory phrases “at least one” and “one or more” to introduce the claims. However, the use of such phrases should not be construed as limiting any particular claim containing the introduced claim recitation by introducing the claim with the indefinite article “a” or “an” to an invention containing only one such recitation, even if the same claim includes the introductory phrases “one or more” or “at least one” and an indefinite article, such as “a” or “an” (e.g., “a” or “an” should generally be interpreted as “one or more” or “at least one”); the same applies to the use of definite articles to introduce claims. Furthermore, even if a claim is explicitly recited to a specific number, those skilled in the art will recognize that such a reference should generally be interpreted to mean at least the number of references (e.g., "two references" without other modifiers generally means at least two references, or two or more references).

[0061] Although various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are intended to be illustrative rather than restrictive, with the true scope and spirit being represented by the following detailed description.

[0062] Reference Numbers

[0063]

[0064]

Claims

1. A method comprising: obtaining one or more images of defects located on a die of a semiconductor wafer; applying a plurality of prediction models to the one or more images to obtain a plurality of classification decisions for the defect, each of the plurality of prediction models being configured to classify a defect into one or more defect classes; obtaining metrological information of the defect; utilizing the metrology information along with the plurality of classification decisions to determine a combined classification decision for the defect; as well as The combined classification decision is output. The method of claim 1 , wherein the metrology information is collected by a scanner.

3. The method of claim 1 , wherein the metrology information is collected by an automated optical inspection scanner.

4. The method of claim 1 , wherein the metrology information includes a size measurement of the defect, whereby the size measurement of the defect is used along with the classification decisions of the plurality of predictive models to determine the combined classification decision.

5. The method according to claim 1, wherein the metering information includes at least one of the following: a histogram of the defects; The maximum color or grayscale value of the defect; and The minimum color or grayscale value of the defect.

6. The method of claim 1, wherein at least some of the predictive models are deep learning models.

7. The method of claim 1, wherein the one or more images include at least two images of the defect.

8. The method according to claim 7, The at least two images of the defect include at least two images of two different imaging modes, and at least one prediction model is a fusion model based on features extracted from the at least two images of the two different imaging modes.

9. The method of claim 1, wherein the one or more images are obtained from a plurality of imaging units.

10. The method according to claim 9, wherein the plurality of imaging units include a first imaging unit providing an image in a first imaging mode and a second imaging unit providing an image in a second imaging mode, the first imaging mode being different from the second imaging mode, At least one of the prediction models is based on a fusion model of features extracted from two images of two different imaging modalities.

11. The method of claim 1 , wherein at least one prediction model is configured to provide a classification prediction based on an image of the one or more images and based on a reference image. The method of claim 11 , wherein the reference image is a gold wafer image.

13. A system for classifying defects in a semiconductor wafer, the system comprising: processor; and one or more imaging units, wherein the one or more imaging units are used to obtain one or more images of defects located on a die of a semiconductor wafer; wherein the processor is configured to: applying a plurality of prediction models to the one or more images to obtain a plurality of classification decisions for the defect, each of the plurality of prediction models being configured to classify a defect into one or more defect classes; obtaining metrological information of the defect; as well as The metrology information is utilized along with the plurality of classification decisions to determine a combined classification decision for the defect.

14. The system of claim 13, further comprising a scanner, wherein the metrology information is collected by the scanner.

15. The system of claim 13, wherein the metrology information includes a size measurement of the defect, whereby the size measurement of the defect is used along with the classification decisions of the plurality of predictive models to determine the combined classification decision.

16. The system of claim 13, wherein the metering information includes at least one of the following: a histogram of the defects; The maximum color or grayscale value of the defect; and The minimum color or grayscale value of the defect.

17. The system of claim 13, wherein at least some of the predictive models are deep learning models.

18. The system of claim 13, wherein the one or more images include at least two images of the defect.

19. The system according to claim 18, The at least two images of the defect include at least two images of two different imaging modes, and at least one prediction model is a fusion model based on features extracted from the at least two images of the two different imaging modes.

20. The system according to claim 13, wherein the one or more imaging units include a first imaging unit providing an image in a first imaging mode and a second imaging unit providing an image in a second imaging mode, the first imaging mode being different from the second imaging mode, At least one of the prediction models is based on a fusion model of features extracted from two images of two different imaging modalities.

Images