Automated method for defect characterization for hybrid bonding applications

Abstract

A method and apparatus for training a machine learning model for automatic detection and classification of defects on a wafer includes receiving labeled wafer defect images having multiple defect classifications, creating a first training set containing the received labeled wafer defect images, training a machine learning model to automatically detect and classify wafer defects in a first stage using the first training set, mixing at least one set of at least two labeled images having different classifications to generate additional labeled image data, creating a second training set containing the mixed additional labeled image data, and training a machine learning model to automatically detect and classify wafer defects in a second stage using the second training set. The trained machine learning model can then be applied to at least one unlabeled wafer image to determine at least one defect classification for that at least one unlabeled wafer image.

Description

Technical Field

[0001] Embodiments of this principle generally relate to detecting defects on wafers, and more particularly to automatically detecting and classifying defects on wafers.

Background Art

[0002] Wafer defects may be caused by the processes through which the wafers are processed. Currently, there are many manual processes for detecting and classifying such defects. For example, in semiconductor wafer metrology, packaging processes, plasma processes, wet cleaning, or wafer singulation processes, such as in the hybrid bonding of wafers, it is currently necessary to implement various metrology tools and imaging tools that can help to manually identify, case by case, the failures present on the wafers between process steps. Current state-of-the-art defect analysis techniques include those that combine optical inspection tools and scanning tools, such as scanning electron microscopes (SEM) and automated optical inspection (AOI) review tools. In this technique, the location of defects on the wafer is first identified using inspection tools, and then inspected using SEM review tools for further defect characterization. Such techniques are based on manual characterization of vast amounts of data.

[0003] That is, in all current defect analysis techniques, manual work is required to inspect images and classify defects into various categories. For example, currently, defects can be captured from sample images using optical inspection tools, and the images are labeled into various defect categories for further review and analysis using manual techniques. In such current defect analysis processes, manual defect review and classification are time-consuming. For example, the process of manual review and classification can take 5 to 30 minutes per wafer depending on the amount of defects on the wafer. Moreover, manual review, being a human judgment, can be inaccurate.

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, accurate and efficient processes are needed to automate defect detection and classification during various processes related to wafer hybrid bonding.

[0005] For example, methods and apparatus for automated detection and automated classification of defects on a wafer during various processes related to hybrid bonding are provided herein. [Means for solving the problem]

[0006] In some embodiments, a method for training a machine learning model for automatic detection and classification of defects on a wafer includes receiving labeled images of wafer defects having multiple defect classifications; creating a first training set comprising the received labeled images of wafer defects having multiple defect classifications; training the machine learning model to automatically detect and classify wafer defects in a first stage using the first training set; mixing at least one set of at least two labeled images having different classifications to generate additional labeled image data; creating a second training set comprising the generated blended, additional labeled image data; and training the machine learning model to automatically detect and classify wafer defects in a second stage using the second training set.

[0007] In some embodiments, the method further includes mixing at least one set of at least two labeled images having different classifications using at least one weighting component.

[0008] In some embodiments, a method for automated detection and classification of defects on a wafer using a trained machine learning model includes receiving at least one unlabeled image of the surface of a wafer, and applying a trained machine learning (ML) model to at least one unlabeled wafer image, wherein the machine learning model is trained to detect and classify defects on a wafer using a first set of labeled wafer defect images and a second set of additional wafer defect images generated from mixing at least two labeled images having different classifications, and using the trained machine learning model to determine at least one defect classification for at least one unlabeled wafer image. In some embodiments, the trained ML model includes at least one of a visual transducer model, a convolutional neural network model, or a recurrent neural network model.

[0009] In some embodiments, the method further includes determining whether the wafer contains significant defects from at least one determined defect classification.

[0010] In some embodiments, the apparatus for training a machine learning model for automatic detection and classification of defects on a wafer includes a processor and memory. In some embodiments, the memory stores at least one program, the at least one program including instructions, which, when executed by the processor, cause the apparatus to perform a method including receiving labeled wafer defect images having multiple defect classifications; creating a first training set including the received labeled wafer defect images having multiple defect classifications; using the first training set to train a machine learning model to automatically detect and classify wafer defects in a first stage; mixing at least one set of at least two labeled images having different classifications to generate additional labeled image data; creating a second training set including the generated mixed additional labeled image data; and using the second training set to train a machine learning model to automatically detect and classify wafer defects in a second stage.

[0011] In some embodiments, the apparatus method is further configured to mix at least one set of at least two labeled images having different classifications using at least one weighting component.

[0012] In some embodiments, an apparatus for automated detection and classification of defects on a wafer, using a trained machine learning model, includes a processor and memory. In some embodiments, the memory stores at least one program, the at least one program including instructions, the instructions which, when executed by the processor, cause the apparatus to receive at least one unlabeled image of the surface of a wafer, and to apply a trained machine learning (ML) model to at least one unlabeled wafer image, the machine learning model being trained to detect and classify defects on a wafer using a first set of labeled wafer defect images and a second set of additional wafer defect images generated from a mixture of at least two labeled images having different classifications, and to use the trained machine learning model to determine at least one defect classification for at least one unlabeled wafer image.

[0013] In some embodiments, the apparatus method is further configured to determine whether a wafer contains significant defects from at least one determined defect classification.

[0014] Further embodiments of this disclosure are described below.

[0015] The embodiments of this disclosure, which are briefly summarized above and discussed in more detail below, can be understood by referring to the explanatory embodiments of this disclosure depicted in the accompanying drawings. However, the accompanying drawings illustrate only typical embodiments of this disclosure and should not be considered to limit the scope of this disclosure, as this disclosure may allow for other similarly effective embodiments. [Brief explanation of the drawing]

[0016] [Figure 1] This is a high-level block diagram of a wafer defect detection and classification system based on one embodiment of the present principle. [Figure 2]This figure illustrates a mix-up augmentation that can be performed by a training and data generation module for generating training data to train a deep learning model, according to one embodiment of this principle. [Figure 3A] This figure shows an image of a stain defect on a wafer and a corresponding attention map image of the area of ​​focus of the machine learning model / algorithm based on this principle when processing that image. [Figure 3B] This figure shows an image of particle defects on a wafer and a corresponding attention map image of the region of focus of the learning model / algorithm of this principle when processing that image. [Figure 3C] This figure shows an image of fiber defects on a wafer and a corresponding attention map image of the region of focus of the learning model / algorithm of this principle when processing that image. [Figure 3D] This figure shows an image of a defect-free wafer and the corresponding attention map image of the area of ​​focus of the machine learning model / algorithm based on this principle when processing that image. [Figure 4] This figure shows a functional training architecture for a wafer defect detection and classification system based on this principle, according to one embodiment. [Figure 5] This figure shows the functional architecture of a wafer defect detection and classification system based on this principle, according to one embodiment. [Figure 6] This is a flowchart illustrating a method for training a machine learning model for automatic detection and classification of defects on a wafer, based on one embodiment of this principle. [Figure 7] This is a flowchart of a method for automatic detection and classification of defects on a wafer, based on one embodiment of this principle. [Figure 8] This is a high-level block diagram of a computing device suitable for use with an embodiment of a wafer defect detection and classification system based on this principle. [Figure 9] This is a high-level block diagram of a network to which an embodiment of a wafer defect detection and classification system based on this principle can be applied, according to one embodiment.

Best Mode for Carrying Out the Invention

[0017] For ease of understanding, whenever possible, the same reference numbers are used to designate the same elements common to each figure. The figures are not to scale and may be simplified for clarity. Elements and features of one embodiment can be advantageously incorporated into other embodiments even without further elaboration.

[0018] In the following detailed description, for example, techniques (such as methods, apparatuses, and systems) for the automatic detection and automatic classification of defects on a wafer during various processes related to hybrid bonding will be described. The concepts of the present principle are capable of various modifications and alternative forms, and specific embodiments thereof are shown in the figures as examples and will be described in detail below. It should be understood that there is no intention to limit the concepts of the present principle to the specific forms disclosed. Rather, the intention is to cover all modifications, equivalents, and alternative forms that are consistent with the present principle and the appended claims. For example, embodiments of the present principle are described herein with respect to specific wafer defects that may occur during a hybrid bonding process and classification categories related to the defects, but embodiments of the present principle can be applied to automatically detect substantially any wafer defect that occurs during any process related to a wafer and classify it into substantially any classification category.

[0019] Throughout the present disclosure, the terms learning model, machine learning (ML) model, ML algorithm, and ML classifier are used interchangeably to describe an ML process that can be trained to recognize / detect and distinguish various types of defects occurring on a wafer and classify those defects into categories.

[0020] Embodiments of this principle enable the automated detection of wafer defects and the automated classification of defects into their respective categories with consistency, reproducibility, and efficiency. Specifically, embodiments of this principle provide the ability to automate the detection and classification of defects on wafers during various processes associated with hybrid bonding, such as plasma, wet cleaning, and / or wafer fragmentation. In embodiments of this principle, a learning model is trained through a novel training process so that it can detect / recognize defects on the wafer, distinguish between various types of defects occurring on the wafer, and classify those defects into categories. In some embodiments, defects can be classified into categories including, but not limited to, particle defects, fiber defects, fouling defects, and / or defect-free defects.

[0021] Figure 1 shows a high-level block diagram of a wafer defect detection and classification system 100 according to one embodiment of the present principle. In the embodiment of Figure 1, the wafer defect detection and classification system 100 exemplary includes a training data generation module 110 and a training and defect detection / classification module 120. In the embodiment of Figure 1, the training and defect detection / classification module 120 includes a training model 122 (described in more detail below). The wafer defect detection and classification system 100 of Figure 1 further exemplary includes an optional storage device 130.

[0022] As shown in Figure 1, embodiments of wafer defect detection and classification systems based on this principle, such as wafer defect detection and classification system 100, can be implemented by a computing device 800 based on this principle (described in more detail below).

[0023] In the wafer defect detection and classification system 100 of Figure 1, the training data generation module 110 can receive data including labeled wafer defect images, which include at least defect classification of the imaged defects. In some embodiments, such labeled image data can be received / acquired from an optional storage device. Alternatively, or in addition, the training data generation module 110 can receive labeled image data from a user of a wafer defect detection and classification system of the present principle, such as the wafer defect detection and classification system 100 of Figure 1. The labeled image data received by the training data generation module 110 can be used to train a learning model 122 (described in more detail below).

[0024] However, the data available for training the learning model 122 may be limited. Therefore, due to a lack of available training data for training the learning model 122, the training data generation module 110 in Figure 1 may implement specific data augmentation techniques to compensate for the limited availability of training data. For example, in some embodiments, the training data generation module 110 may implement a confusion augmentation process to generate additional training data for training the learning model 122. For example, Figure 2 shows a diagram of a confusion augmentation process 200 that can be implemented by the training data generation module 110 in Figure 1 to generate training data for training the learning model 122 according to an embodiment of this principle.

[0025] According to this principle, a confusion expansion process may include mixing images of wafer defects, each containing a label that identifies any defect on the wafer. For example, in the embodiment of Figure 2, the confusion expansion process 200 is represented as including mixing an image 202 containing particle defects 203, each having a particle defect label 204, with an image 206 containing fiber defects 207, each having a fiber defect label 208. In the embodiment of the confusion expansion process 200 in Figure 2, the images and labels can be mixed, for example, by equations (1) and (2) below, using a weighting component λ (0.5 in the example of Figure 2). miximage=λ×image1+(1-λ)×image2 (1) mixlabel=λ×label1+(1-λ)×label2 (2)

[0026] The confusion extension process of this principle functions like a regularization method, clarifying the resulting model along with each class boundary. That is, the classifier / model generally learns a hard decision boundary to distinguish classes based on the model. In some embodiments of this principle, the confusion extension process of this principle takes intermediate images into account during the learning process, which allows the model to learn diffuse boundaries. This makes the model more regular or robust so that it predicts the intermediate images with a certain amount of class confusion. For example, in the embodiment of Figure 2, the resulting mixed image 210 can be used as additional data to train a learning model of this principle to recognize / detect wafer defects and to categorize the detected wafer defects according to this principle.

[0027] In the embodiment shown in Figure 2, the confusion augmentation process 200 performed by the training data generation module 110 is represented as mixing two images, an image of a labeled particle defect and an image of a labeled fiber defect. Alternatively, or additionally, in other embodiments of the principle, the training data generation module of the principle may perform the confusion augmentation process to mix other numbers and types of labeled defect images to generate training data for a learning model 122 according to the principle.

[0028] In some embodiments, the training data received and / or generated by the training data generation module 110 is transmitted to the training and defect detection / classification module 120 of a wafer defect detection and classification system of the present principle, such as the wafer defect detection and classification system 100 in Figure 1. In some embodiments, the training and defect detection / classification module 120 uses the received training data to train a learning model 122 to recognize / detect and distinguish various types of defects occurring on the wafer and to classify the defects into categories. In some embodiments, defects can be classified into categories including, but not limited to, particle defects, fiber defects, dirt defects and / or defect-free defects.

[0029] In some embodiments, the model / algorithm of this principle, such as the learning model / algorithm 122, may include a multilayer neural network having nodes that are trained to have specific weights and biases. In some embodiments, the learning model / algorithm 122 employs artificial intelligence or machine learning techniques to analyze received data images containing wafer defects. In some embodiments of this principle, appropriate machine learning techniques may be applied to learn commonalities in sequential application programs and to determine from the machine learning techniques to what level sequential application programs can be normalized. In some embodiments, machine learning techniques that can be applied to learn commonalities in sequential application programs may include, but are not limited to, regression methods, set methods, or neural networks such as "Se2oSeq" recurrent neural networks (RNNs) / Long Short-Term Memory (LSTM) networks, convolutional neural networks (CNNs), graph neural networks applied to abstract syntax trees corresponding to sequential program applications, and similar, as well as deep learning. In some embodiments, however, supervised machine learning (ML) classifiers / algorithms such as multilayer perceptrons, random forests, naive Bayes, support vector machines, logistic regression, and similar methods may be used. In addition, in some embodiments, the ML classifiers / algorithms of this principle may employ at least one of sliding window or sequence-based techniques to analyze the data.

[0030] The learning model / algorithm 122 can be trained using multiple (e.g., hundreds, thousands, etc.) specific examples of labeled image data, and its training data includes multiple images of labeled wafer defects to train the learning model / algorithm of this principle to recognize / detect and distinguish various types of defects on the wafer and categorize the defects.

[0031] For example, in one training example, a training dataset containing a total of 21,000 images of labeled wafer defects was used to train the learning model / algorithm of this principle. In this training example, relevant images within the training dataset were selected in a similar ratio to correct for any distortion in the training dataset. In this training example, confusion augmentation was applied to new images in the dataset with each iteration, thereby subjecting the learning model / algorithm of this principle to 21,000 variations of the training data.

[0032] For example, in some embodiments, a transformer such as a ViT (Vision Transducer) may be used to transform each image into multiple patches, and position coding may be applied to each patch. The result can then be input to an encoder along with multi-head attention. In some embodiments, different learning rates can be applied to each layer so that the resulting model can be optimized without disturbing its respective pre-trained weights.

[0033] After training a learning model of this principle, such as learning model 122 in Figure 1, the accuracy of the learning model / algorithm of this principle was inspected / tested using 105 wafer defect images with four different classifications, including 27 dirt defect images, 33 particle defect images, 25 fiber defect images, and 20 miscount (defect-free) images.

[0034] In the inspection / testing process described above, during testing using inspection images, the region of interest of the learning model / algorithm of this principle was analyzed using an attention map to verify whether the learning model / algorithm of this principle was focusing on the correct region of the inspection image containing wafer defects when detecting and / or classifying wafer defects using this principle. Figures 3A to 3D show the inspection image of the wafer and the respective attention map representing the focus of the learning model / algorithm of this principle when the inspection image is taken into consideration.

[0035] More specifically, Figure 3A shows an image 302 of a stain defect on a wafer and a corresponding attention map image 304 of the area of ​​focus of the machine learning model / algorithm of this principle when processing that image. As can be seen in Figure 3A, the learning model / algorithm appropriately focuses on the area containing the stain defect on the wafer image.

[0036] Figure 3B shows an image 306 of particle defects on a wafer and a corresponding attention map image 308 of the area of ​​focus of the machine learning model / algorithm based on this principle when processing that image. As can be seen in Figure 3B, the learning model / algorithm appropriately focuses on the area containing particle defects on the wafer image.

[0037] Figure 3C shows an image 310 of fiber defects on a wafer and a corresponding attention map image 312 of the area of ​​focus of the machine learning model / algorithm based on this principle when processing that image. As can be seen in Figure 3C, the learning model / algorithm appropriately focuses on the area containing fiber defects on the wafer image.

[0038] Figure 3D shows an image 314 of a defect-free wafer and a corresponding attention map image 316 of the area of ​​focus of the machine learning model / algorithm of this principle when processing that image. In the embodiment of Figure 3D, the learning model / algorithm focuses on the background of the wafer image because no other class of defects can be detected in the wafer image.

[0039] Figure 4 shows a diagram of the functional training architecture of a wafer defect detection and classification system of this principle, such as the wafer defect detection and classification system 100 in Figure 1. As shown in Figure 4, a labeled image (SEM image as an example) 402 is received by the wafer defect detection and classification system 100. The received image 402 is complemented using a confusion expansion process 410 according to the pre-configured principle described above to generate an expanded labeled image. The received labeled image 402 and the expanded labeled image are used to train a learning model (ViT model as an example) 422 of this principle to recognize / detect and distinguish various types of defects occurring on the wafer, and to classify defects into categories from the received labeled image 402 and the expanded labeled image.

[0040] In the embodiment of Figure 4, the ViT model 422 implements an attention mechanism to determine which part of the image to focus on (region of interest (ROI)). That is, in the embodiment of Figure 4, the region of interest (ROI) determined by the ViT model 422 is analyzed using the attention map 415 to verify whether the ViT model 422 of this principle was focusing on the correct region of the image. The trained ViT model 422 can then be applied, for example, as a classifier to recognize / detect and distinguish various types of defects occurring on a wafer, and to categorize defects from images such as SEM images. In the embodiment of Figure 4, the learning model of this principle is represented as a ViT model, but in alternative embodiments of this principle, the ML model of this principle may include, as an alternative or additional, a neural network such as a convolutional neural network or a recurrent neural network and similar.

[0041] Figure 5 shows a diagram of the functional architecture of a wafer defect detection and classification system of the present principle, such as the wafer defect detection and classification system 100 of Figure 1, according to one embodiment of the present principle. As shown in Figure 5, at least one SEM image 502 can be received by the wafer defect detection and classification system 100. An ML model (ViT model as an example) 522 trained according to the present principle identifies wafer defects in at least one SEM image 502 and classifies the defects into defect classes.

[0042] The defect class of wafer defects determined by the ML classifier of this principle can be used, for example, to determine the throughput of a wafer system. For example, the defect class of wafer defects determined according to this principle can be used to determine whether a defect on a wafer is critical and whether a wafer with a particular defect must be discarded or removed from the wafer processing system.

[0043] Figure 6 shows a flowchart of method 600 for training a machine learning model for automatic detection and classification of defects on a wafer, according to one embodiment of the present principle. The method can begin at 602, where a labeled wafer defect image having multiple defect classifications is received. Method 600 can then proceed to 604.

[0044] In 604, a first training set is created, which includes received labeled wafer defect images with multiple defect classifications. Method 600 can then proceed to 606.

[0045] In 606, the machine learning model is trained using a first training set to automatically detect and classify wafer defects in the first stage. Method 600 can then proceed to 608.

[0046] In 608, at least one set of at least two labeled images having different classifications is mixed to generate additional labeled image data. Method 600 can then proceed to 610.

[0047] In step 610, a second training set is created containing the generated mixed additionally labeled image data. Method 600 can then proceed to step 612.

[0048] In step 612, the machine learning model is trained to automatically detect and classify wafer defects in the second stage using a second training set. Method 600 can then be derived.

[0049] Figure 7 shows a flowchart of a method for automated detection and classification of defects on a wafer using a trained machine learning model, according to an embodiment of the present principle. The method can be started from 702, where at least one unlabeled image of the wafer surface is received. The method can then proceed from 700 to 704.

[0050] In 704, a machine learning model is applied to at least one unlabeled wafer image, which is trained using a first set of labeled wafer defect images and a second set of additional wafer defect images generated from a mixture of at least two labeled images having different classifications. Method 700 can proceed to 706.

[0051] In method 706, a trained machine learning model is used to determine the defect classification of at least one unlabeled wafer image. Method 700 can then be derived.

[0052] As shown in Figure 1, embodiments of the wafer defect detection and classification system of this principle, such as the wafer defect detection and classification system 100 in Figure 1, can be implemented in a computing device 800 based on this principle. That is, in some embodiments, wafer defect image data and similar data can be transmitted to the wafer defect detection and classification system of this principle using the computing device 800, for example, via any input / output means associated with the computing device 800. The classification data associated with the wafer defect detection and classification system of this principle can be presented to the user using an output device of the computing device 800, such as a display, printer, or any other form of output device.

[0053] For example, Figure 8 shows a high-level block diagram of a computing device 800 suitable for use with embodiments of wafer defect detection and classification systems based on this principle, such as the wafer defect detection and classification system 100 of Figure 1. In some embodiments, the computing device 800 can be configured to implement the method of this principle in various embodiments as a processor-executable program instruction 822 (e.g., a program instruction executable by processor 810).

[0054] In the embodiment shown in Figure 8, the computing device 800 includes one or more processors 810a to 810n coupled to system memory 820 via an input / output (I / O) interface 830. The computing device 800 further includes a network interface 840 coupled to the I / O interface 830, and one or more input / output devices 850, such as a cursor control device 860, a keyboard 870, and a display 880. In various embodiments, a user interface may be generated and displayed on the display 880. In some cases, the embodiment can be carried out using a single specific example of the computing device 800, while in other embodiments, it is intended that multiple such systems, or multiple nodes constituting the computing device 800, may be configured to host different parts or specific examples of various embodiments. For example, in one embodiment, several elements may be carried out through one or more nodes of the computing device 800, separate from the nodes that carry out the other elements. In another example, multiple nodes may carry out the computing device 800 in a distributed manner.

[0055] In different embodiments, the computing device 800 may be any of various types of devices, including but not limited to personal computer systems, desktop computers, laptops, notebooks, tablet or netbook computers, mainframe computer systems, handheld computers, workstations, network computers, cameras, set-top boxes, mobile devices, consumer devices, video game consoles, handheld video game devices, application servers, storage devices, peripheral devices (such as switches, modems, and routers), or any type of computing device or electronic device in general.

[0056] In various embodiments, the computing device 800 may be a uniprocessor system including one processor 810, or a multiprocessor system including several processors 810 (e.g., two, four, eight, or another appropriate number). The processors 810 can be any suitable processor capable of executing instructions. For example, in various embodiments, the processors 810 may be general-purpose processors or embedded processors implementing any of various instruction set architectures (ISAs). In a multiprocessor system, each of the processors 810 may, but does not necessarily, implement the same ISA.

[0057] The system memory 820 can be configured to store program instructions 822 and / or data 832 accessible by the processor 810. In various embodiments, the system memory 820 can be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory. In the exemplary embodiment, program instructions and data that implement any of the elements of the embodiments described above can be stored in the system memory 820. In other embodiments, program instructions and / or data can be received, transmitted, or stored by a different type of computer-accessible medium or similar medium separate from the system memory 820 or computing device 800.

[0058] In one embodiment, the I / O interface 830 can be configured to coordinate I / O traffic between the processor 810, the system memory 820, and any peripheral devices in the device, including other peripheral interfaces such as the network interface 840 or input / output device 850. In some embodiments, the I / O interface 830 can perform any necessary protocol conversion, timing conversion, or other data conversion to convert data signals from one component (e.g., the system memory 820) into a format suitable for use by another component (e.g., the processor 810). In some embodiments, the I / O interface 830 may include supports for devices attached via various types of peripheral buses, such as the Peripheral Component Interconnect (PCI) bus standard or a variant of the Universal Serial Bus (USB) standard. In some embodiments, the functionality of the I / O interface 830 can be divided into two or more separate components, such as a northbridge and a southbridge. Also, in some embodiments, some or all of the functionality of the I / O interface 830, such as the interface with the system memory 820, may be directly integrated into the processor 810.

[0059] The network interface 840 can be configured to allow data to be exchanged between the computing device 800 and other devices attached to a network (e.g., network 890), such as one or more external systems, or between nodes of the computing device 800. In various embodiments, network 890 can include, but is not limited to, one or more networks, including, a local area network (LAN) (e.g., Ethernet or an enterprise network), a wide area network (WAN) (e.g., the Internet), a wireless data network, any other electronic data network, or some combination thereof. In various embodiments, the network interface 840 can support communication over, for example, a wired or wireless general-purpose data network, such as any suitable type of Ethernet network; over a digital fiber optic communication network; over a storage area network such as a Fibre Channel SAN; or over any other suitable type of network and / or protocol.

[0060] In some embodiments, the input / output device 850 may include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other devices suitable for inputting or accessing data by one or more computer systems. Multiple input / output devices 850 may reside within a computer system or be distributed across various nodes of the computing device 800. In some embodiments, similar input / output devices may be isolated from the computing device 800 and may interact with one or more nodes of the computing device 800 via wired or wireless connections, such as through a network interface 840.

[0061] Those skilled in the art will understand that computing device 800 is merely illustrative and does not limit the scope of the embodiments. In particular, computer systems and devices may include any combination of hardware or software capable of performing the functions shown in various embodiments, including computers, network devices, internet equipment, PDAs, wireless telephones, pagers, and the like. Computing device 800 may also be connected to other devices not illustrated, or it may instead operate as a standalone system. In addition, the functions provided by the illustrated components may, in some embodiments, be integrated into fewer components or distributed among additional components. Similarly, in some embodiments, some functions of the illustrated components may not be provided, and / or other additional functions may be available.

[0062] The computing device 800 can communicate with other computing devices based on various computer communication protocols, such as Wi-Fi, Bluetooth® (and / or other standards for exchanging data over short distances, including protocols that use short-wavelength wireless transmission), USB, Ethernet, cellular, and ultrasonic local area communication protocols. The computing device 800 may also include a web browser.

[0063] Although the computing device 800 is represented as a general-purpose computer, it is programmed to perform various specialized control functions and is configured to operate as a specialized, dedicated computer according to the present principle. Embodiments can be implemented in hardware, for example, as an application-specific integrated circuit (ASIC). Therefore, the process steps described herein are intended to be broadly interpreted as being equivalently implemented by software, hardware, or a combination thereof.

[0064] Figure 9 is a high-level block diagram of a network to which embodiments of the wafer defect detection and classification system of the present principle can be applied, such as the wafer defect detection and classification system 100 of Figure 1. The network environment 900 in Figure 9 exemplifies a user domain 902 including a user domain server / computing device 904. The network environment 900 in Figure 9 further includes a computer network 906 and a cloud environment 910 including a cloud server / computing device 912.

[0065] In the network environment 900 of Figure 9, a wafer defect detection and classification system based on this principle, such as the wafer defect detection and classification system of Figure 1, may be included in at least one of the user domain server / computing device 904, the computer network 906, and the cloud server / computing device 912. That is, in some embodiments, a user can use a local server / computing device (e.g., user domain server / computing device 904) to detect and classify wafer defects using this principle.

[0066] In some embodiments, a user can implement a system for detecting and classifying wafer defects according to this principle on a computer network 906. Alternatively, or additionally, in some embodiments, a user can implement a system for detecting and classifying wafer defects on a cloud server / computing device 912 in a cloud environment 910 to provide container forensics according to this principle. For example, in some embodiments, it may be advantageous to implement the processing functions of this principle in the cloud environment 910 in order to utilize the processing and storage capabilities of the cloud environment 910. In some embodiments of this principle, a system for detecting and classifying wafer defects can be located in one and / or more locations / servers / computers to perform all or part of the functions of the system according to this principle as described herein. For example, a wafer defect detection and classification system according to this principle can be located in one or more of the user domain 902, the computer network environment 906, and the cloud environment 910 to detect and classify wafer defects according to this principle.

[0067] Those skilled in the art will understand that, although various items are illustrated as being stored in memory or storage while in use, these items or parts thereof may be transferred between memory and other storage devices for the purposes of memory management and data integrity. Alternatively, in other embodiments, some or all of the software components in memory may be executed in the memory of another device and communicate with the illustrated computer system via intercomputer communication. Some or all of the system components or data structures may also be stored in computer-accessible media or portable devices read by a suitable drive (e.g., as instructions or structured data), various examples of which are described above. In some embodiments, instructions stored in computer-accessible media separate from the computing device 600 may be transmitted to the computing device 600 via a transmission medium or via signals such as electrical, electromagnetic, or digital signals transmitted by a communication medium such as a network and / or wireless link. Various embodiments may further include storing instructions and / or data to be performed in accordance with the foregoing description in computer-accessible media, or receiving and transmitting them via a communication medium. Generally, computer-accessible media may include storage or memory media such as magnetic or optical media (e.g., disks or DVD / CD-ROMs) and volatile or non-volatile media such as RAM (e.g., SDRAM, DDR, RDRAM, SRAM, etc.), ROM, etc.

[0068] The methods and processes described herein can be implemented in software, hardware, or a combination thereof in different embodiments. In addition, the order of the methods can be changed, and various elements can be added, rearranged, combined, omitted, or otherwise modified. All examples described herein are presented non-limitingly. Various modifications and changes are possible, as will be apparent to those skilled in the art who benefit from this disclosure. Embodiments according to embodiments are described in relation to specific embodiments. These embodiments are illustrative and not limiting. Many variations, modifications, additions, and improvements are possible. Thus, multiple specific examples may be provided with respect to the components described herein as a single specific example. The boundaries between various components, operations, and data storage devices are somewhat arbitrary, and certain operations are illustrated in relation to specific exemplary configurations. Other assignments of functions are assumed and may fall within the scope of the appended claims. Structures and functions presented as individual components of exemplary configurations can be implemented as combined structures or components. These, and other variations, modifications, additions, and improvements may fall within the scope of embodiments as defined in the appended claims.

[0069] The preceding description includes numerous specific details, examples, and scenarios to provide a more complete understanding of the disclosure. However, it should be understood that embodiments of the disclosure can be practiced without such specific details. Furthermore, such examples and scenarios are presented for illustrative purposes only and are not intended to limit the disclosure. Those skilled in the art should be able to perform the appropriate functions without unnecessary experimentation based on the included description.

[0070] In this specification, references to "one embodiment," etc., indicate that the described embodiment may include a particular feature, structure, or characteristic, but not all embodiments necessarily include that particular feature, structure, or characteristic. Such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in relation to an embodiment, it is considered within the knowledge of those skilled in the art that such feature, structure, or characteristic will be affected in relation to other embodiments, whether or not it is explicitly shown.

[0071] Embodiments provided in this disclosure can be implemented in hardware, firmware, software, or any combination thereof. Embodiments can also be implemented as instructions stored using one or more machine-readable media, which can be read and executed by one or more processors. The machine-readable media can include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing device, or a “virtual machine” operating on one or more computing devices). For example, the machine-readable media can include any suitable form of volatile or non-volatile memory.

[0072] The modules, data structures, and similar entities defined herein are defined so for the sake of ease of discussion and do not imply that details of any particular embodiment are required. For example, any of the modules and / or data structures described may be combined or divided into submodules, subprocesses, or other units of computer code or data, as may be required by a particular design or embodiment.

[0073] Drawings may show a specific arrangement or order of diagrammatic elements for ease of explanation. However, a specific order or arrangement of such elements does not imply that a specific order or sequence of operations, or separation of processes, is required in all embodiments. Generally, diagrammatic elements used to represent instruction blocks or modules can be implemented using any suitable form of machine-readable instruction, each such instruction can be implemented using any suitable programming language, library, application programming interface (API), and / or other software development tools or frameworks. Similarly, diagrammatic elements used to represent data or information can be implemented using any suitable electronic arrangement or data structure. Furthermore, some connections, relationships, or combinations between elements may be simplified or omitted in the drawings so as not to obscure this disclosure.

[0074] While the above applies to embodiments of the Disclosure, other further embodiments of the Disclosure can be conceived without departing from the basic scope of the Disclosure.

Claims

1. A method for training a machine learning model for automatic detection and classification of defects on a wafer, Receiving labeled wafer defect images with multiple defect classifications, Creating a first training set including the received labeled wafer defect images having the plurality of defect classifications, The machine learning model is trained to automatically detect and classify wafer defects in the first stage using the first training set, Mixing at least one set of at least two labeled images having different classifications to generate additional labeled image data, Creating a second training set that includes the generated mixed additional label image data, Using the second training set, the machine learning model is trained to automatically detect and classify wafer defects in the second stage. Methods that include...

2. The method according to claim 1, wherein the plurality of defect classifications include at least two of particle defects, fiber defects, stain defects, or no defects.

3. The method according to claim 1 or 2, wherein the ML model includes at least one of a visual transducer model, a convolutional neural network model, or a recurrent neural network model.

4. The method according to any one of claims 1 to 3, further comprising mixing the at least one set of the at least two labeled images having different classifications using at least one weighting component.

5. The method according to any one of claims 1 to 4, wherein the at least one set of the at least two labeled images having different classifications is mixed using a confusion expansion process.

6. A method for automated detection and classification of defects on a wafer using a trained machine learning model, Receiving at least one unlabeled image of the wafer surface, The application of the trained machine learning (ML) model to the at least one unlabeled wafer image, wherein the machine learning model is trained to detect and classify defects on the wafer using a first set of labeled wafer defect images and a second set of additional wafer defect images generated from a mixture of at least two labeled images having different classifications. Using the trained machine learning model, determine at least one defect classification for the at least one unlabeled wafer image. Methods that include...

7. The method according to claim 6, wherein the at least one defect classification includes at least one of particle defects, fiber defects, stain defects, or no defects.

8. The method according to claim 6 or 7, further comprising determining whether the wafer contains a significant defect from the at least one determined defect classification.

9. The method according to claim 6, wherein the trained ML model includes at least one of a visual transducer model, a convolutional neural network model, or a recurrent neural network model.

10. The method according to any one of claims 6 to 9, wherein the second set of additional wafer defect images is generated using at least one weighting component.

11. Apparatus for training a machine learning model for automatic detection and classification of defects on a wafer, Processor and Memory that internally stores at least one program and The device comprises, wherein the at least one program includes instructions, and the instructions, when executed by the processor, Receiving labeled wafer defect images with multiple defect classifications, Creating a first training set including the received labeled wafer defect images having the plurality of defect classifications, The machine learning model is trained to automatically detect and classify wafer defects in the first stage using the first training set, Mixing at least one set of at least two labeled images having different classifications to generate additional labeled image data, Creating a second training set that includes the generated mixed additional label image data, Using the second training set, the machine learning model is trained to automatically detect and classify wafer defects in the second stage. A device that causes a method including the execution of a procedure.

12. The apparatus according to claim 11, wherein the plurality of defect classifications include at least two of particle defects, fiber defects, stain defects, or no defects.

13. The apparatus according to claim 11, wherein the ML model includes at least one of a visual transducer model, a convolutional neural network model, or a recurrent neural network model.

14. The method described above is The apparatus according to any one of claims 11 to 13, further comprising mixing the at least one set of the at least two labeled images having different classifications using at least one weighting component.

15. The apparatus according to any one of claims 11 to 14, wherein at least one set of at least two labeled images having different classifications is mixed using a confusion expansion process.

16. An apparatus for automatic detection and classification of defects on a wafer, using a trained machine learning model, Processor and Memory that internally stores at least one program and The device comprises, wherein the at least one program includes instructions, and the instructions, when executed by the processor, Receiving at least one unlabeled image of the wafer surface, The application of the trained machine learning (ML) model to the at least one unlabeled wafer image, wherein the machine learning model is trained to detect and classify defects on the wafer using a first set of labeled wafer defect images and a second set of additional wafer defect images generated from a mixture of at least two labeled images having different classifications. Using the trained machine learning model, determine at least one defect classification for the at least one unlabeled wafer image. A device that causes a method including the execution of a procedure.

17. The apparatus according to claim 16, wherein the at least one defect classification includes at least one of particle defects, fiber defects, stain defects, or no defects.

18. The method described above is The apparatus according to claim 16 or 17, further comprising determining whether the wafer contains a significant defect from the at least one determined defect classification.

19. The apparatus according to claim 16, wherein the trained ML model includes at least one of a visual transducer model, a convolutional neural network model, or a recurrent neural network model.

20. The apparatus according to any one of claims 16 to 19, wherein the second set of additional wafer defect images is generated using at least one weighting component.

Images