Wafer defect identification method and device, equipment, storage medium and program product

Abstract

The invention relates to a wafer defect identification method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring wafer image data of a target wafer; inputting the wafer image data into a trained defect identification model to obtain a defect identification result of the target wafer; a loss function of the trained defect identification model in training comprises a loss item of a dichotomy head and a loss item of a multi-classification head of the defect identification model, the dichotomy head is used for outputting a defect existence probability, and the multi-classification head is used for outputting probability distribution of defect types. By adopting the method, the wafer defect identification accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of industrial quality inspection technology, and in particular to a wafer defect identification method, apparatus, equipment, storage medium, and program product. Background Technology

[0002] With the development of Automated Optical Inspection (AOI) technology for defect detection on wafers, the detection algorithms used in AOI technology are mainly divided into two categories: one is based on traditional digital image processing methods, and the other is based on emerging technologies such as deep learning.

[0003] While emerging technologies based on deep learning have certain advantages, the diverse types of wafer defects present significant challenges. Deep learning, particularly for multi-class classification, demands a high degree of data distribution balance. However, collecting defect data in the wafer environment is extremely difficult; some types of defects occur with very low probabilities, while others occur with extremely high probabilities. This poses a significant challenge to data balance, resulting in poor accuracy for the trained defect recognition models in identifying wafer defects. Summary of the Invention

[0004] Therefore, it is necessary to provide a wafer defect identification method, apparatus, device, storage medium, and program product that can improve the accuracy of wafer defect identification in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a method for identifying wafer defects, including:

[0006] Acquire wafer image data of the target wafer;

[0007] The wafer image data is input into the trained defect recognition model to obtain the defect recognition result of the target wafer. The loss function of the trained defect recognition model during training includes the loss term of the binary classification head and the loss term of the multi-classification head. The binary classification head is used to output the probability of the existence of defects, and the multi-classification head is used to output the probability distribution of defect types.

[0008] In one embodiment, before inputting wafer image data into a trained defect recognition model to obtain the defect recognition result of the target wafer, the method further includes:

[0009] Obtain a wafer sample dataset, which includes wafer image samples and label information for the wafer image samples, including defect labels and defect type labels;

[0010] Input wafer image samples into the initial recognition model to obtain defect probability and defect type probability distribution;

[0011] The binary classification loss value is determined based on the difference between the defect probability and the defect label.

[0012] The multi-class loss value is determined based on the difference between the defect type probability distribution and the defect type label;

[0013] Based on the binary classification loss value and the multi-class classification loss value, determine the function value of the loss function of the initial recognition model;

[0014] Based on the value of the loss function, the model parameters of the initial recognition model are updated until the loss function of the initial recognition model converges, thus obtaining the trained defect recognition model.

[0015] In one embodiment, the step of obtaining a trained defect recognition model until the loss function of the initial recognition model converges includes:

[0016] If the loss function of the defect recognition model converges, discard the binary classification head of the initial recognition model to obtain the trained defect recognition model.

[0017] In one embodiment, obtaining a wafer sample dataset includes:

[0018] Acquire images of defective wafers and normal wafers, with the defective wafer images marked with defective regions;

[0019] Generate an augmented wafer image based on a normal wafer image and a defective wafer image with marked defect areas;

[0020] Defective wafer images, normal wafer images, and expanded wafer images are identified as wafer image samples, and the label information of the wafer image samples is determined to obtain a wafer sample dataset.

[0021] In one embodiment, the trained defect recognition model includes an input layer, a feature extraction layer, and a classification layer, wherein the classification layer includes at least a multi-classifier head; inputting wafer image data into the trained defect recognition model to obtain the defect recognition result of the target wafer includes:

[0022] Wafer image data is input into the input layer to obtain a predetermined number of image sub-regions, wherein each image sub-region has the same specifications and some overlap.

[0023] After stitching together the image sub-regions, the images are input into the feature extraction layer to obtain the wafer image features.

[0024] By inputting wafer image features into the classification layer, at least the probability distribution of each wafer defect type output by the multi-classification head can be obtained;

[0025] If there are wafer defect types with a probability higher than a predetermined confidence threshold in the probability distribution, the wafer defect type with the highest probability is determined as the defect identification result.

[0026] In one embodiment, the method further includes:

[0027] Detection evaluation indicators for obtaining defect identification results;

[0028] If the detected evaluation index is less than or equal to the predetermined index threshold, the wafer sample dataset is updated based on the wafer image data of the target wafer to obtain a new wafer sample training set.

[0029] Based on the new wafer sample dataset, the initial identification model is trained until the loss function of the initial identification model converges, resulting in a well-trained defect identification model.

[0030] Secondly, this application also provides a wafer defect identification device, comprising:

[0031] The data acquisition module is used to acquire wafer image data of the target wafer;

[0032] The defect recognition module is used to input wafer image data into the trained defect recognition model to obtain the defect recognition result of the target wafer. The loss function of the trained defect recognition model during training includes the loss term of the binary classification head and the loss term of the multi-classification head of the defect recognition model. The binary classification head is used to output the probability of the existence of defects, and the multi-classification head is used to output the probability distribution of defect types.

[0033] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0034] Acquire wafer image data of the target wafer;

[0035] The wafer image data is input into the trained defect recognition model to obtain the defect recognition result of the target wafer. The loss function of the trained defect recognition model during training includes the loss term of the binary classification head and the loss term of the multi-classification head. The binary classification head is used to output the probability of the existence of defects, and the multi-classification head is used to output the probability distribution of defect types.

[0036] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0037] Acquire wafer image data of the target wafer;

[0038] The wafer image data is input into the trained defect recognition model to obtain the defect recognition result of the target wafer. The loss function of the trained defect recognition model during training includes the loss term of the binary classification head and the loss term of the multi-classification head. The binary classification head is used to output the probability of the existence of defects, and the multi-classification head is used to output the probability distribution of defect types.

[0039] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0040] Acquire wafer image data of the target wafer;

[0041] The wafer image data is input into the trained defect recognition model to obtain the defect recognition result of the target wafer. The loss function of the trained defect recognition model during training includes the loss term of the binary classification head and the loss term of the multi-classification head. The binary classification head is used to output the probability of the existence of defects, and the multi-classification head is used to output the probability distribution of defect types.

[0042] The aforementioned wafer defect identification method, apparatus, device, storage medium, and program products acquire wafer image data of the target wafer, input the wafer image data into a trained defect identification model, and obtain the defect identification result of the target wafer. The loss function of the trained defect identification model includes a loss term for the binary classification head and a loss term for the multi-class classification head. The binary classification head outputs the probability of the presence of a defect, and the multi-class classification head outputs the probability distribution of the defect type. This application sets loss terms for both the binary and multi-class classification heads in the loss function of the defect identification model. This allows the binary classification loss term, which has a coarser granularity (more coarse but higher identification accuracy due to a relatively balanced sample distribution) and the multi-class classification loss term, which has a finer granularity (lower identification accuracy due to an imbalanced sample distribution), to collaboratively determine the update gradient of the model parameters. This makes the model parameter updates more stable, avoids the defect identification model being biased by imbalanced multi-class data, and helps improve the accuracy of wafer defect identification. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is an application environment diagram of a wafer defect identification method in one embodiment;

[0045] Figure 2 This is a flowchart illustrating a wafer defect identification method in one embodiment;

[0046] Figure 3 This is a flowchart illustrating a wafer defect identification method in another embodiment;

[0047] Figure 4 This is a schematic diagram of a defect generation scenario in one embodiment of this application;

[0048] Figure 5 This is a schematic diagram of another scenario of defect generation in one embodiment of this application;

[0049] Figure 6 This is a schematic diagram of another defect generation scenario in one embodiment of this application;

[0050] Figure 7 This is a flowchart illustrating the wafer defect identification method in yet another embodiment;

[0051] Figure 8 This is a schematic diagram of a defect identification scenario in one embodiment of this application;

[0052] Figure 9 This is a flowchart illustrating the wafer defect identification method in another embodiment;

[0053] Figure 10 This is an application scenario diagram related to an embodiment of this application;

[0054] Figure 11 This is a structural block diagram of a wafer defect identification device in one embodiment;

[0055] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0058] The wafer defect identification method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. In this embodiment, the trained defect recognition model can be configured on server 104 or terminal 102. Taking the trained defect recognition model configured on server 104 as an example, terminal 102 can acquire wafer image data of the target wafer, and then send the wafer image data to server 104. Server 104 inputs the wafer image data into the trained defect recognition model, outputs the defect recognition result of the target wafer, and feeds back the defect recognition result of the target wafer to terminal 102. Terminal 102 can be, but is not limited to, various shooting devices, personal computers, laptops, smartphones, tablets, IoT devices, etc. Server 104 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0059] In one exemplary embodiment, such as Figure 2 As shown, a wafer defect identification method is provided, which can be applied to... Figure 1 Taking the server in the example, the explanation includes the following steps S110 to S120. Wherein:

[0060] Step S110: Obtain wafer image data of the target wafer;

[0061] The target wafer is the wafer for which defect identification is desired, and the wafer image data is the image data obtained by photographing the target wafer.

[0062] For example, this embodiment can establish a communication connection with an imaging device on a wafer production line to obtain wafer image data obtained by the imaging device capturing images of a target wafer on the wafer production line. Alternatively, this embodiment can also obtain wafer image data of the target wafer from a local device or other communication-connected terminal devices.

[0063] Step S120: Input the wafer image data into the trained defect recognition model to obtain the defect recognition result of the target wafer; The loss function of the trained defect recognition model during training includes the loss term of the binary classification head and the loss term of the multi-classification head of the defect recognition model. The binary classification head is used to output the probability of the existence of defects, and the multi-classification head is used to output the probability distribution of defect types.

[0064] The training loss function of the defect recognition model includes loss terms for both the binary classification head and the multi-class classification head. The binary classification head outputs the probability of a defect, while the multi-class classification head outputs the probability distribution of defect types. The trained defect recognition model comprises an input layer, a feature extraction layer, and a classification layer connected in sequence. The input layer processes the input image, such as image normalization (subtracting the mean and dividing by the standard deviation). Further, in this embodiment, the input layer can also be used to segment the input image, obtaining a predetermined number of image sub-regions. These sub-regions have consistent dimensions and partial overlap. After stitching together the sub-regions, they are input to the feature extraction layer. The feature extraction layer extracts features from the image passed from the input layer, obtaining image feature vectors. The classification layer includes at least a multi-class classification head; that is, both a binary classification head and a multi-class classification head exist during training. However, if the loss function converges, the binary classification head can be discarded or retained to obtain the trained defect recognition model. The input to the multi-class classification head is the image feature vector passed from the feature extraction layer, and the output is the probability distribution of K defect types. The classification layer can also include a binary classification head, which takes the image feature vector passed from the feature extraction layer as input and outputs the probability of the presence of defects.

[0065] In this embodiment, wafer image data is input into a pre-trained defect recognition model. The defect recognition model, based on the wafer image data and according to the trained model parameters, outputs the defect recognition result for the target wafer. Taking a pre-trained defect recognition model comprising an input layer, a feature extraction layer, and a classification layer connected in sequence as an example, wafer image data is input into the input layer to obtain a pre-processed image. The pre-processed image is then input into the feature extraction layer to obtain wafer image features. These wafer image features are input into the classification layer to obtain at least the probability distribution of each wafer defect type output by the multi-classification head. If a wafer defect type with a probability higher than a predetermined confidence threshold exists in the probability distribution, the wafer defect type with the highest probability is determined as the defect recognition result.

[0066] Furthermore, to enable the defect recognition model to adapt to wafer image data of different sizes and channels (RGB channels, RGBW channels, etc.), this embodiment can set a segmentation unit and a stitching unit in the input layer. The segmentation unit is used to divide the input image into a predetermined number of image sub-regions. Each image sub-region has the same specifications and partially overlaps. For example, the overlap ratio between each image sub-region is 10%, 20%, 30% of the length or width, etc., which can be selected according to specific needs. The stitching unit is used to stitch the image sub-regions together according to channels. It should be noted that the stitching is not a spatial stitching of the image sub-regions, but a superposition along the channel dimension, so as not to change the internal pixel arrangement of each image sub-region and maintain the original shape of the image features. Thus, this embodiment can input wafer image data into the input layer to obtain a predetermined number of image sub-regions, wherein each image sub-region has the same specifications and partially overlaps. After stitching the image sub-regions, they are input into the feature extraction layer to obtain wafer image features. Then, the wafer image features are input into the classification layer to obtain the defect recognition result of the target wafer.

[0067] In the aforementioned wafer defect identification method, wafer image data of the target wafer is acquired and input into a trained defect identification model to obtain the defect identification result of the target wafer. The loss function of the trained defect identification model during training includes a loss term for the binary classification head and a loss term for the multi-classification head. The binary classification head outputs the probability of the presence of a defect, and the multi-classification head outputs the probability distribution of the defect type. This embodiment sets loss terms for both the binary and multi-classification heads in the loss function of the defect identification model. The binary classification head, with its coarser granularity (whether a defect exists or not, but whose sample distribution is relatively balanced, resulting in higher identification accuracy), and the multi-classification head, with its finer granularity (but whose multi-class sample distribution is prone to imbalance, resulting in lower identification accuracy), collaboratively determine the update gradient of the model parameters. This makes the update of the model parameters more stable, avoids the defect identification model being biased by imbalanced multi-class data, and helps improve the accuracy of wafer defect identification.

[0068] In one exemplary embodiment, such as Figure 3 As shown, before inputting the wafer image data into the trained defect recognition model to obtain the defect recognition result of the target wafer, the method further includes steps S210 to S260. Wherein:

[0069] Step S210: Obtain a wafer sample dataset. The wafer sample dataset includes wafer image samples and label information for the wafer image samples. The label information includes defect labels and defect type labels.

[0070] Step S220: Input the wafer image sample into the initial recognition model to obtain the defect probability and defect type probability distribution;

[0071] Step S230: Determine the binary classification loss value based on the difference between the defect probability and the defect label;

[0072] Step S240: Determine the multi-classification loss value based on the difference between the defect type probability distribution and the defect type label;

[0073] Step S250: Based on the binary classification loss value and the multi-class classification loss value, determine the function value of the loss function of the initial recognition model;

[0074] Step S260: Based on the function value of the loss function, update the model parameters of the initial recognition model until the loss function of the initial recognition model converges, and obtain the trained defect recognition model.

[0075] The wafer sample dataset includes wafer image samples and label information for the wafer image samples. The label information includes defect labels and defect type labels. The wafer image samples include normal wafer images and defective wafer images with defects. The defect labels are labels that characterize whether the wafer image sample has defects, and the defect type labels are labels that characterize the type of wafer defect present in the wafer image sample, such as grain twin cell defects, grain breakage defects, and conventional grain defects (i.e., irregular and differently colored connected regions on the grain). The initial identification model is the defect identification model in the initial state. For example, the initial identification model can be a pre-trained neural network model such as ResNet (Residual Network), Swin Transformer (Shifted windows Transformer), ConvNeXt V1 (Convolutional Network for the 2020s (Next) Version 1), or ConvNeXt V2 (Convolutional Network for the 2020s (Next) Version 2).

[0076] This embodiment inputs wafer image samples into an initial recognition model to obtain the defect probability output by the binary classification head of the initial recognition model, and the defect type probability distribution output by the multi-classification head of the initial recognition model, i.e., the probability corresponding to each wafer defect type. This embodiment can determine the binary classification loss value based on the difference between the defect probability and the defect label. For example, this embodiment can determine the difference between the defect probability and the probability value represented by the defect label as the difference between the defect probability and the defect label. Alternatively, this embodiment can determine the difference between the defect probability and the probability value represented by the defect label as the difference between the defect probability and the defect label. Furthermore, this embodiment can determine the multi-classification loss value based on the difference between the defect type probability distribution and the defect type label. For example, this embodiment can determine the difference between the defect type probability distribution and the probability distribution represented by the defect type label as the difference between the defect type probability distribution and the defect type label. Alternatively, this embodiment can determine the difference between the defect type probability distribution and the probability distribution represented by the defect type label as the difference between the defect type probability distribution and the defect type label. In this embodiment, the sum of the binary classification loss value and the multi-class classification loss value can be used as the function value of the loss function of the initial recognition model. Alternatively, the binary classification loss value and the multi-class classification loss value can be weighted to obtain the function value of the loss function of the initial recognition model. For example, the loss function can be expressed as follows:

[0077] Loss = w1×loss1+w2×loss2;

[0078] Where loss1 is the binary classification loss value, loss2 is the multi-class classification loss value, w1 is the weight of the binary classification loss, and w2 is the weight of the multi-class classification loss.

[0079] Therefore, in this embodiment, the model parameters of the initial identification model can be updated based on the function value of the loss function until the loss function of the initial identification model converges, thus obtaining a trained defect identification model. For example, the convergence of the loss function of the initial identification model is determined when the number of updates to the model parameters of the initial identification model reaches the maximum number of iterations, or when the function value of the loss function is minimized. Thus, in this embodiment, the initial identification model can be used as the trained defect identification model. Alternatively, this embodiment can discard the binary classification head of the initial identification model to obtain the trained defect identification model.

[0080] This embodiment utilizes a loss term from a binary head (which has a coarser granularity, indicating the presence or absence of defects, but whose sample distribution is relatively balanced, resulting in higher recognition accuracy) and a loss term from a multi-class head (which has a finer granularity, but whose multi-class sample distribution is prone to imbalance, resulting in lower recognition accuracy). Together, they determine the update gradient of the model parameters, making the update of the model parameters more stable and preventing the defect recognition model from being biased by the imbalanced data of the multi-class classification, thus helping to improve the recognition accuracy of the defect recognition model.

[0081] In some embodiments, the step of obtaining a trained defect recognition model until the loss function of the initial recognition model converges includes:

[0082] Step S261: If the loss function of the defect recognition model converges, discard the binary classification head of the initial recognition model to obtain the trained defect recognition model.

[0083] Since the multi-classifier head of the initial identification model already possesses the ability to identify the types of wafer defects when the loss function of the initial identification model converges, this embodiment can discard the binary classifier head of the initial identification model to obtain the trained defect identification model. Discarding the binary classifier head of the initial identification model simplifies the model structure of the trained defect identification model, saves runtime resources, and also improves runtime efficiency.

[0084] In some embodiments, obtaining a wafer sample dataset includes:

[0085] Step S310: Obtain images of the defective wafer and normal wafers, wherein the defective wafer image is marked with defective regions;

[0086] Step S320: Generate an expanded wafer image based on the normal wafer image and the defective wafer image with marked defective areas;

[0087] Step S330: Defective wafer images, normal wafer images, and expanded wafer images are identified as wafer image samples, and the label information of the wafer image samples is determined to obtain a wafer sample dataset.

[0088] Since the number of defective wafer images is small and the distribution of defective wafer images of different wafer defect types is not uniform, this embodiment can expand the defective wafer images based on normal wafer images and defective wafer images with marked defective areas.

[0089] This embodiment can acquire defective wafer images and normal wafer images, wherein the defective wafer image is marked with defective regions. This embodiment can generate an augmented wafer image based on the normal wafer image and the defective wafer image with marked defective regions. For example, this embodiment can construct a defect generation algorithm based on the defective wafer image and the normal wafer image. The defect generation algorithm can be a function or model built based on open-source algorithm libraries such as OpenCV, such as a defect generation function, generative adversarial network, or diffusion model, which can be used to augment image samples. Then, the normal wafer image can be input into the defect generation algorithm to obtain an augmented wafer image containing wafer defects.

[0090] For example, this embodiment can use a defect generation function built based on open-source algorithm libraries such as OpenCV as the defect generation algorithm, such as... Figure 4 As shown, taking the generation of a twin-cell defect as an example, the morphology of a twin-cell defect is that one side is connected to another grain. In this embodiment, a normal wafer image can be input into a defect generation algorithm to generate twin-cell defects, thus obtaining an expanded wafer image containing twin-cell defects, achieving the expansion of the wafer image containing twin-cell defects. For example... Figure 5 As shown, taking a defective wafer image with a grain chipping defect as an example, the grain chipping defect is characterized by chipping along the edge of the grain, but the gap extends into the internal region of the grain. This embodiment can utilize open-source algorithm libraries such as OpenCV to generate a corresponding defect generation function based on the defective wafer image, serving as the defect generation algorithm for the grain chipping defect. Then, a normal wafer image is input into the defect generation algorithm corresponding to this grain chipping defect, resulting in an expanded wafer image showing the grain chipping defect. For example... Figure 6 As shown, in addition to twin cells and grain breakage, conventional defects also exist. Taking a conventional defect as an example, in a defective wafer image, a conventional defect is an irregular, connected region of varying colors on the grain. This embodiment can utilize open-source algorithm libraries such as OpenCV to generate a corresponding defect generation function based on the defective wafer image, which serves as the defect generation algorithm for the conventional defect. Then, a normal wafer image is input into the defect generation algorithm corresponding to this conventional defect, resulting in an expanded wafer image containing the conventional defect.

[0091] Therefore, in this embodiment, defective wafer images, normal wafer images, and augmented wafer images can be identified as wafer image samples, and corresponding label information can be generated for each wafer image sample. Each wafer image sample, along with its corresponding label information, is used as a wafer sample dataset. To improve the sample quality of the wafer dataset, this embodiment can also perform undersampling processing on the wafer sample with the higher proportion when the difference in the proportion of normal wafer samples to defective wafer samples in the wafer image samples is greater than or equal to a predetermined difference threshold, until the difference in the proportion of normal wafer samples to defective wafer samples is less than the predetermined difference threshold.

[0092] Furthermore, to further improve sample diversity, this embodiment can also acquire raw wafer images under at least two light sources, divide the raw wafer images into defective wafer images and normal wafer images, and mark the defective areas on the defective wafer images. The light sources can include at least two of the following: ring light sources, coaxial light sources, strip light sources, dome light sources, polarized light sources (adding polarizers to conventional light sources), and spectral light sources (emitting multiple specific wavelengths of light).

[0093] In this embodiment, defective wafer images and normal wafer images are acquired, with the defective wafer images marked with defective regions. Based on the normal wafer images and the defective wafer images with marked defective regions, expanded wafer images are generated. The defective wafer images, normal wafer images, and expanded wafer images are identified as wafer image samples, and the label information of these wafer image samples is determined, resulting in a wafer sample dataset. Therefore, this embodiment expands the samples based on normal wafer images and defective wafer images with marked defective regions, which helps increase the number of samples and, to some extent, balances the proportion of samples that would be unbalanced under natural distribution.

[0094] In one exemplary embodiment, such as Figure 7 As shown, the trained defect recognition model includes an input layer, a feature extraction layer, and a classification layer, with the classification layer including at least a multi-classifier head. Inputting wafer image data into the trained defect recognition model to obtain the defect recognition result of the target wafer includes steps S410 to S440. Wherein:

[0095] Step S410: Input wafer image data into the input layer to obtain a predetermined number of image sub-regions, wherein each image sub-region has the same specifications and some overlap.

[0096] Step S420: After stitching together the image sub-regions, input them into the feature extraction layer to obtain the wafer image features;

[0097] Step S430: Input the wafer image features into the classification layer to obtain at least the probability distribution of each wafer defect type output by the multi-classification head;

[0098] Step S440: If there is a wafer defect type with a probability higher than a predetermined confidence threshold in the probability distribution, the wafer defect type with the highest probability is determined as the defect identification result.

[0099] The trained defect recognition model includes an input layer, a feature extraction layer, and a classification layer. The classification layer includes at least a multi-classifier head. The input layer, feature extraction layer, and classification layer are connected sequentially. The input layer contains a segmentation unit and a stitching unit. The segmentation unit divides the input image into a predetermined number of image sub-regions. The stitching unit stitches these sub-regions together along the channel dimension. It's important to note that stitching does not spatially merge the sub-regions; rather, it superimposes them along the channel dimension, thus preserving the internal pixel arrangement of each sub-region and maintaining the original shape of the image features. The segmentation and stitching units can utilize splitting and stitching tools from deep learning frameworks.

[0100] In this embodiment, wafer image data can be input into an input layer. The input layer can segment the image region corresponding to the wafer image data according to a predetermined number (e.g., 2×2, 3×3, 4×4), obtaining a predetermined number of image sub-regions. Each image sub-region has the same dimensions and some overlap. The dimensions (length, width, aspect ratio) of each image sub-region are consistent, and the edges of each image sub-region overlap with adjacent image sub-regions. For example, the overlap ratio between an image sub-region and its adjacent image sub-regions is 10%, 20%, 30%, etc., and the specific overlap ratio can be selected according to requirements. It is understood that since common images are mostly rectangular, each image sub-region can be rectangular (e.g., rectangular or oriented in a positive direction) to achieve accurate segmentation of the original wafer image data. Therefore, the partial overlap of each image sub-region not only ensures the preservation of the structural integrity across sub-regions and enhances the correlation between corresponding features of each image sub-region, but also reduces the problem of boundary information loss to a certain extent.

[0101] Therefore, in this embodiment, the various image sub-regions can be stitched together to obtain a stitched image, which is then input into the feature extraction layer to obtain wafer image features. It should be noted that the stitching does not involve spatially stitching the image sub-regions, but rather superimposing them along the channel dimension, thus preserving the internal pixel arrangement of each image sub-region and maintaining the original shape of the image features. For example... Figure 8As shown, the wafer image data is divided into 3×3=9 image sub-regions. These 9 sub-regions are then stacked and stitched along the channel dimension to obtain a 9-channel stitched image. This ensures that regardless of the wafer image data's size or channel type (RGB, RGBW, etc.), the resulting image is always a 9-channel stitched image after the above segmentation and stitching process. This maintains consistency in the input to the feature extraction layer while preserving the image features, thus improving the model's generalization ability. Furthermore, in this embodiment, the stitched image can be input into the feature extraction layer to obtain wafer image features. Taking ConvNeXt V2 as an example, the feature extraction layer includes a stem layer, a four-stage feature extraction network, and a global processing layer. After inputting the stitched image into the stem layer, it undergoes standard convolution and channel dimension normalization to obtain initial image features. These initial image features are then input into the four-stage feature extraction network. The four-stage feature extraction network is designed with a pyramid structure of "gradually halving the spatial size and gradually doubling the number of channels," abstracting the initial image features layer by layer and finally outputting high-dimensional image features. The high-dimensional image features are then processed through a global processing layer, undergoing global average pooling and vector normalization to obtain wafer image features. The multi-classification head in the classification layer can include at least one fully connected network layer. After the wafer image features are input into the fully connected network, the feature vector output by the fully connected network is processed by a normalized exponential function to obtain the probability distribution of wafer defect types. Similarly, the binary classification head can also include at least one fully connected network layer. After the wafer image features are input into the fully connected network, the feature vector output by the fully connected network is processed by a normalized exponential function to obtain the probability of defect presence. Therefore, in this embodiment, by inputting wafer image features into the classification layer, at least the probability distribution of each wafer defect type output by the multi-classification head is obtained. If a wafer defect type with a probability higher than a predetermined confidence threshold exists in the probability distribution, the wafer defect type with the highest probability is determined as the defect identification result. If the trained defect identification model also has a binary classification head, in this embodiment, the wafer image features can be input into the classification layer in the binary classification head to obtain the probability of defect presence output by the binary classification head, as well as the probability distribution of each wafer defect type output by the multi-classification head. If the probability of a defect is less than or equal to a specified probability threshold, the target wafer can be identified as a normal wafer as the defect identification result. If the probability of a defect is greater than the specified probability threshold, and there are wafer defect types with a probability higher than a predetermined confidence threshold in the probability distribution, the wafer defect type with the highest probability can be identified as the defect identification result.

[0102] In this embodiment, wafer image data is input into the input layer to obtain a predetermined number of image sub-regions. These sub-regions have consistent specifications and partial overlap. After stitching together the sub-regions, the images are input into the feature extraction layer to obtain wafer image features. These features are then input into the classification layer to obtain the probability distribution of each wafer defect type output by the multi-classification head. If a wafer defect type with a probability higher than a predetermined confidence threshold exists in the probability distribution, the wafer defect type with the highest probability is identified as the defect recognition result. This embodiment, by segmenting and re-stitching the original wafer image data, ensures consistency in the input to the feature extraction layer for wafer image data of different specifications and channels, while maintaining the integrity of the image features, thus improving the model's generalization ability. Furthermore, the partial overlap of the sub-regions not only preserves the structural integrity across sub-regions and enhances the correlation between corresponding features in each sub-region, but also reduces the loss of boundary information to some extent, helping to ensure the model's recognition accuracy.

[0103] In one exemplary embodiment, such as Figure 9 As shown, the method further includes steps S510 to S530. Wherein:

[0104] Step S510: Obtain the detection evaluation index of the defect identification results;

[0105] Step S520: If the detected evaluation index is less than or equal to the predetermined index threshold, the wafer sample dataset is updated based on the wafer image data of the target wafer to obtain a new wafer sample training set.

[0106] Step S530: Based on the new wafer sample dataset, train the initial identification model until the loss function of the initial identification model converges, and obtain the trained defect identification model.

[0107] Among them, the detection evaluation index is an indicator for evaluating the accuracy of defect identification results, such as detection accuracy rate, number of accurate detections, and detection completion rate.

[0108] This embodiment can obtain a detection evaluation index for defect identification results. If the detection evaluation index is less than or equal to a predetermined threshold, it indicates that the current defect identification model has significant over-detection, under-detection, or false detection. In this case, this embodiment can add at least some of the target wafer image data to the wafer sample dataset to update the wafer sample dataset and obtain a new wafer sample training set. For example, this embodiment can use the wafer image data of the target wafers that have over-detected, under-detected, or falsely detected as new wafer image samples, and then add the new wafer image samples and label information to the wafer sample dataset to obtain a new wafer sample training set. Then, based on the new wafer sample dataset, the initial identification model is trained until the loss function of the initial identification model converges, resulting in a trained defect identification model.

[0109] In this embodiment, the defect identification results are monitored by the detection evaluation index. If there are obvious over-detections, under-detections, or false detections in the defect identification model, the defect identification model is updated to ensure the accuracy of the defect identification model.

[0110] like Figure 10 As shown, Figure 10 This is an application scenario diagram related to an embodiment of this application. In this embodiment, by acquiring normal wafer images and defective wafer images, and using random masks for data synthesis, a wafer sample dataset is obtained. Thus, a large number of wafer image samples can be generated as a wafer sample dataset using only a small number of wafer images. Then, an initial recognition model (such as an optimized ConvNeXt-V2 model, i.e., a ConvNeXt-V2 model with input layers containing segmentation and stitching units) can be trained using the wafer sample dataset. The trained initial recognition model is then converted to a relevant format and input into the detection system. Online wafer data from online operations is input into the detection system, and the detection system uses the trained initial recognition model to output defect recognition results. A detection evaluation index for the defect recognition results is obtained. If the detection evaluation index is less than or equal to a predetermined index threshold, a supplementary dataset can be formed based on the online wafer data. This supplementary dataset is then merged with the original wafer sample dataset to obtain a new wafer sample dataset. Then, based on the new wafer sample dataset, the steps of training the initial recognition model based on the wafer sample dataset and converting the trained initial recognition model to a relevant format before inputting it into the detection system are returned. If the detected evaluation index exceeds the predetermined threshold, the trained initial identification model will be updated to the detection system as the defect detection model and officially used in production.

[0111] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0112] Based on the same inventive concept, this application also provides a wafer defect identification device for implementing the wafer defect identification method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more wafer defect identification device embodiments provided below can be found in the limitations of the wafer defect identification method described above, and will not be repeated here.

[0113] In one exemplary embodiment, such as Figure 11 As shown, a wafer defect identification device 600 is provided, including: a data acquisition module 610 and a defect identification module 620, wherein:

[0114] The data acquisition module 610 is used to acquire wafer image data of the target wafer;

[0115] The defect recognition module 620 is used to input wafer image data into the trained defect recognition model to obtain the defect recognition result of the target wafer. The loss function of the trained defect recognition model during training includes the loss term of the binary classification head and the loss term of the multi-classification head of the defect recognition model. The binary classification head is used to output the probability of the existence of defects, and the multi-classification head is used to output the probability distribution of defect types.

[0116] In some embodiments, the wafer defect identification device 600 further includes a model training module for:

[0117] Obtain a wafer sample dataset, which includes wafer image samples and label information for the wafer image samples. The label information includes defect labels and defect type labels. Input the wafer image samples into the initial recognition model to obtain the defect probability and defect type probability distribution. Determine the binary classification loss value based on the difference between the defect probability and the defect label. Determine the multi-class classification loss value based on the difference between the defect type probability distribution and the defect type label. Determine the function value of the loss function of the initial recognition model based on the binary classification loss value and the multi-class classification loss value. Update the model parameters of the initial recognition model based on the function value of the loss function until the loss function of the initial recognition model converges, and obtain the trained defect recognition model.

[0118] In some embodiments, the model training module is further configured to:

[0119] If the loss function of the defect recognition model converges, discard the binary classification head of the initial recognition model to obtain the trained defect recognition model.

[0120] In some embodiments, the model training module is further configured to:

[0121] Obtain defective wafer images and normal wafer images, where defective wafer images are marked with defective regions. Based on normal wafer images and defective wafer images with marked defective regions, generate augmented wafer images. Determine the defective wafer images, normal wafer images, and augmented wafer images as wafer image samples, and determine the label information of the wafer image samples to obtain a wafer sample dataset.

[0122] In some embodiments, the trained defect recognition model includes an input layer, a feature extraction layer, and a classification layer, wherein the classification layer includes at least a multi-classifier head; the defect recognition module 620 is further configured to:

[0123] Wafer image data is input into the input layer to obtain a predetermined number of image sub-regions, where each image sub-region has the same specifications and some overlap. After the image sub-regions are stitched together, they are input into the feature extraction layer to obtain wafer image features. The wafer image features are input into the classification layer to obtain at least the probability distribution of each wafer defect type output by the multi-classification head. If there is a wafer defect type with a probability higher than a predetermined confidence threshold in the probability distribution, the wafer defect type with the highest probability is determined as the defect identification result.

[0124] In some embodiments, the wafer defect identification device 600 further includes a model update module for:

[0125] The detection evaluation index of the defect identification result is obtained. When the detection evaluation index is less than or equal to the predetermined index threshold, the wafer sample dataset is updated based on the wafer image data of the target wafer to obtain a new wafer sample training set. The initial identification model is trained based on the new wafer sample dataset until the loss function of the initial identification model converges, and the trained defect identification model is obtained.

[0126] Each module in the aforementioned wafer defect identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0127] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores model data for a trained defect identification model. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When executed by the processor, the computer program implements a wafer defect identification method.

[0128] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0129] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the embodiments described above.

[0130] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above embodiments.

[0131] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps described in the above embodiments.

[0132] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0133] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0134] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for identifying wafer defects, characterized in that, The method includes: Acquire wafer image data of the target wafer; The wafer image data is input into the trained defect recognition model to obtain the defect recognition result of the target wafer; the loss function of the trained defect recognition model during training includes the loss term of the binary classification head and the loss term of the multi-classification head of the defect recognition model. The binary classification head is used to output the probability of the existence of defects, and the multi-classification head is used to output the probability distribution of defect types.

2. The method according to claim 1, characterized in that, Before inputting the wafer image data into the trained defect recognition model to obtain the defect recognition result of the target wafer, the method further includes: Obtain a wafer sample dataset, which includes wafer image samples and label information for the wafer image samples, including defect labels and defect type labels; The wafer image samples are input into the initial recognition model to obtain the defect probability and defect type probability distribution; The binary classification loss value is determined based on the difference between the defect probability and the defect label; The multi-class loss value is determined based on the difference between the defect type probability distribution and the defect type label; Based on the binary classification loss value and the multi-class classification loss value, determine the function value of the loss function of the initial recognition model; Based on the value of the loss function, the model parameters of the initial identification model are updated until the loss function of the initial identification model converges, thus obtaining a trained defect identification model.

3. The method according to claim 2, characterized in that, The step of obtaining a trained defect recognition model until the loss function of the initial recognition model converges includes: If the loss function of the defect recognition model converges, discard the binary classification head of the initial recognition model to obtain the trained defect recognition model.

4. The method according to claim 2, characterized in that, The acquisition of the wafer sample dataset includes: Acquire images of defective wafers and normal wafers, wherein the defective wafer images are marked with defective regions; An expanded wafer image is generated based on the normal wafer image and the defective wafer image with marked defective areas; The defective wafer image, the normal wafer image, and the augmented wafer image are identified as wafer image samples, and the label information of the wafer image samples is determined to obtain a wafer sample dataset.

5. The method according to claim 1, characterized in that, The trained defect recognition model includes an input layer, a feature extraction layer, and a classification layer, wherein the classification layer includes at least a multi-classifier head; the step of inputting the wafer image data into the trained defect recognition model to obtain the defect recognition result of the target wafer includes: The wafer image data is input into the input layer to obtain a predetermined number of image sub-regions, wherein each image sub-region has the same specifications and some overlap. After stitching together the image sub-regions, the images are input into the feature extraction layer to obtain wafer image features. By inputting the wafer image features into the classification layer, at least the probability distribution of each wafer defect type output by the multi-classification head can be obtained; If there is a wafer defect type with a probability higher than a predetermined confidence threshold in the probability distribution, the wafer defect type with the highest probability is determined as the defect identification result.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain the detection evaluation index of the defect identification results; If the detection evaluation index is less than or equal to the predetermined index threshold, the wafer sample dataset is updated based on the wafer image data of the target wafer to obtain a new wafer sample training set. Based on the new wafer sample dataset, the initial identification model is trained until the loss function of the initial identification model converges, thus obtaining the trained defect identification model.

7. A wafer defect identification device, characterized in that, The device includes: The data acquisition module is used to acquire wafer image data of the target wafer; The defect identification module is used to input the wafer image data into the trained defect identification model to obtain the defect identification result of the target wafer; the loss function of the trained defect identification model during training includes the loss term of the binary classification head and the loss term of the multi-classification head of the defect identification model, the binary classification head is used to output the probability of the existence of defects, and the multi-classification head is used to output the probability distribution of defect types.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images