Unsupervised wafer graph anomaly detection and clustering method based on automatic encoder

Abstract

The invention relates to an unsupervised wafer graph anomaly detection and clustering method based on an automatic encoder, and the method comprises the steps: carrying out the filtering preprocessing of a test image containing a normal wafer graph and an abnormal wafer graph, building an automatic encoder model, inputting the normal wafer graph after the filtering preprocessing into the model, and carrying out the multi-round training, calculating a judgment threshold according to the reconstruction error, inputting the test image after filtering preprocessing into a trained automatic encoder model, comparing the reconstruction error with the judgment threshold, if the reconstruction error is smaller than the judgment threshold, determining that the test image is a normal wafer image, and otherwise, determining that the test image is an abnormal wafer image. And finally, reconstructing the multi-term statistics of the error graph by using the abnormal image to serve as features, and completing abnormal clustering. According to the method, the abnormal wafer graph can be effectively and automatically identified and classified under the condition that the label does not need to be manually labeled.

Description

Technical Field

[0001] The present invention belongs to the field of integrated circuit testing and relates to the field of deep learning. Specifically, it relates to an unsupervised wafer map anomaly detection and clustering method based on an autoencoder. Background Art

[0002] Semiconductors form the core of the electronic information industry and are hailed as the "heart" of electronic information products. With the progress and wide application of semiconductor technology in multiple fields, it has promoted the development of multiple industries such as new energy, communication technology, medical health, logistics, artificial intelligence, and autonomous driving. Wafer manufacturing is a key step in the semiconductor industry, and its manufacturing quality directly affects the qualification rate of the final product. In this process, any process flaw may cause defects in the wafer, thereby reducing the overall production efficiency. The wafer defect distribution can be described by the wafer map generated during wafer testing. As an important visualization tool for electrical performance data in the testing stage, the wafer map can be used for tasks such as wafer map failure detection and wafer map failure mode recognition. The most common binary wafer map includes the background, normal devices, and abnormal devices, and normal and abnormal devices are represented by pixel points of different colors. By analyzing the defect patterns in the wafer map, the causes of these defects can be revealed, providing key clues for engineers. This analysis helps to identify and solve problems in the production process, thereby optimizing the production process and improving the qualification rate of wafers.

[0003] In recent years, deep learning has played an important role in image processing, and models based on convolutional neural networks have become powerful tools for solving image classification. Many existing wafer defect detection methods are supervised learning methods. Supervised learning has a high cost of label annotation. Especially for semantic segmentation networks, more precise pixel-level segmentation and annotation are required, which will consume a lot of time, increase the testing cost, and the annotation process will be affected by subjectivity. Unsupervised methods usually do not require label annotation, which can reduce the workload of testers and improve the testing efficiency. In addition, in response to the current development trend of the increasing complexity of semiconductor device design and process manufacturing, compared with the limitations of supervised learning on failure categories, using unsupervised methods can assist professionals in discovering more unknown and new types of wafer failure patterns, which has high industrial application value. Summary of the Invention

[0004] The purpose of the present invention is to provide an unsupervised wafer map anomaly detection and clustering method based on an autoencoder. By training normal images with the designed autoencoder model, the reconstruction error of the normal images is output. Combining with the Gaussian distribution, the judgment threshold is calculated. Based on the comparison result between this threshold and the reconstruction error of the abnormal images, anomaly detection is completed. Finally, by extracting statistical indicators of multiple abnormal image reconstruction error maps as clustering features, unsupervised clustering is completed.

[0005] To achieve the above object, the technical solution of the present invention is: an unsupervised wafer map anomaly detection method based on an autoencoder, including the following steps:

[0006] Step 1: Perform filtering preprocessing on test images containing normal wafer maps and abnormal wafer maps;

[0007] Step 2: Build an autoencoder model, input the preprocessed normal wafer maps into the model for multiple rounds of training, and calculate the judgment threshold according to the reconstruction error;

[0008] Step 3: Input the preprocessed test images into the trained autoencoder model, compare the size of the reconstruction error and the judgment threshold. If the reconstruction error is less than the judgment threshold, it is a normal wafer image; otherwise, it is an abnormal wafer image.

[0009] Further technology of the present invention:

[0010] Preferably, the algorithm used for filtering preprocessing in Step 1 is the median filtering algorithm;

[0011] The median filtering algorithm defines a neighborhood of size r×r for each pixel of the image, where r is the neighborhood radius. The pixel values are sorted within the neighborhood, and then each pixel value is replaced with the median of its neighborhood values.

[0012] Preferably, the specific implementation process of Step 2 is as follows:

[0013] (a) Adjust the normal wafer map to a size of m×n according to the wafer particles, where m and n are the number of rows and columns of the wafer particles;

[0014] (b) Build an autoencoder model, which includes an encoder, a decoder, and a channel spatial attention module added to both the encoder and the decoder to enhance the feature capture ability, and compress and reconstruct the normal wafer map;

[0015] (c) Calculate the reconstruction error between the reconstructed image output by the normal wafer map through the autoencoder model and the original image. The reconstruction error is composed of the sum of the mean absolute error and the mean square error;

[0016] (d) Calculate the judgment threshold according to the reconstruction error. The size of the judgment threshold is the average value of the reconstruction error plus twice the standard deviation of the reconstruction error, following a Gaussian distribution.

[0017] Preferably, for the autoencoder model, the specific establishment method and the parameters of each layer of the model are:

[0018] (a) Add an input layer in the Encoder part, with an input size of m×n and 3 channels;

[0019] (b) Add a convolutional block to the Encoder part, which consists of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of E f1 ×E f1 and a number of channels of E c1 and a stride of E s1 ;

[0020] (c) Add a convolutional block to the Encoder part, which consists of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of E f2 ×E f2 and a number of channels of E c2 and a stride of E s2 ;

[0021] (d) Add a convolutional block to the Encoder part, which consists of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of E f3 ×E f3 and a number of channels of E c3 and a stride of E s3 ;

[0022] (e) Add a convolutional block to the Encoder part, which consists of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of E f4 ×E f4 and a number of channels of E c4 and a stride of E s4 ;

[0023] (f) Add a transposed convolutional block to the Decoder part, which consists of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of D f1 ×D f1 and a number of channels of D c1 and a stride of D s1 ;

[0024] (g) Add a transposed convolutional block to the Decoder part, which consists of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of D f2 ×D f2 and a number of channels of D c2 and a stride of D s2 ;

[0025] (h) Add a transposed convolutional block to the Decoder part, which consists of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of D f3 ×D f3 and a number of channels of D c3 and a stride of D s3 ;

[0026] (i) Add a transposed convolution block to the Decoder part, which consists of a convolutional layer, a ReLU layer, and a CBAM layer. The convolutional kernel size is D f4 ×D f4 , the number of channels is D c4 , and the stride is D s4 ;

[0027] (j) Add an activation layer with a Sigmoid activation function to the Decoder part as the output layer.

[0028] Preferably, the specific implementation process of step three is as follows:

[0029] (a) Adjust the test image to the size of m×n, where m and n are the number of rows and columns of the wafer particles;

[0030] (b) Calculate the reconstruction error between the reconstructed image output by the test image passing through the autoencoder model and the original image, and compare the size relationship between the reconstruction error and the judgment threshold. If the reconstruction error is less than the judgment threshold, it is a normal wafer image; otherwise, it is an abnormal wafer image.

[0031] The present invention also provides an unsupervised wafer map anomaly clustering method based on an autoencoder. According to the pixel value difference between the input abnormal image and the model reconstructed image, an abnormal image reconstruction error map is obtained. Multiple statistical parameter values of the abnormal image reconstruction error map are selected and calculated, and used as clustering features to cluster unsupervised abnormal wafer maps.

[0032] The further clustering specific implementation process is as follows:

[0033] (a) Stack the channels of the abnormal image reconstruction error map to obtain a single-channel two-dimensional error map;

[0034] (b) Calculate multiple statistics of the abnormal image reconstruction error map, including the average value of pixel values, the standard deviation of pixel value distribution, the size of the abnormal pixel area, the x coordinate of the centroid of the abnormal area, the y coordinate of the centroid of the abnormal area, the dispersion of abnormal pixels, the edge gradient size of the abnormal area, and the edge curvature of the abnormal area;

[0035] (c) Use the statistics as the feature basis for clustering. After min-max normalization and weighted operation processing, input them into the K-Means clustering algorithm to cluster unsupervised abnormal wafer maps.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] In the method of the present invention, median filtering preprocessing is performed on the wafer image, reducing the influence of noise in the original picture and retaining the main features of the wafer map;

[0038] In the method of the present invention, when training the autoencoder model, only normal wafer map samples need to be input, without inputting abnormal samples or labeling abnormal samples, realizing unsupervised anomaly detection and improving the detection efficiency;

[0039] In the method of the present invention, a combination of a deep convolutional network and channel spatial attention is used to construct an autoencoder model, enhancing the model's feature capture ability, making the compression and reconstruction of normal images during model training more accurate, and better highlighting the difference in the reconstruction error sizes between the reconstructed abnormal images and the reconstructed normal images during testing, thus accurately and efficiently completing anomaly detection;

[0040] In the method of the present invention, statistical indicators of the reconstruction error map of abnormal samples detected by the autoencoder model are calculated, and these indicators are weighted as clustering features of the K-Means algorithm, realizing unsupervised anomaly clustering with a relatively high silhouette coefficient of the clustering result and good clustering effect;

[0041] Using the method of the present invention, compared with the limitations of supervised learning on failure categories, it can, to a certain extent, assist integrated circuit test engineers in discovering more unknown and novel-featured wafer failure modes. Brief Description of the Drawings

[0042] Figure 1 It is a schematic diagram of the steps of the present invention.

[0043] Figure 2 It is a comparison diagram of median filtering before and after in part of the wafer map.

[0044] Figure 3 It is a structural diagram of the autoencoder used by the present invention to realize unsupervised wafer map anomaly detection and clustering.

[0045] Figure 4 It is a visualization diagram of the reconstruction effect of part of the wafer map through the autoencoder model.

[0046] Figure 5 It is an effect diagram of t-SNE clustering of abnormal wafer maps. Detailed Embodiments

[0047] The present invention will be further described below in conjunction with the drawings and embodiments, and the present invention includes but is not limited to the following embodiments.

[0048] The wafer map dataset used in this embodiment is from a public dataset, including one type of normal sample, namely Normal (N), and eight types of abnormal samples, namely Center (C), Donut (D), Edge-Local (EL), Edge-Ring (ER), Local (L), Near-Full (NF), Scratch (S), and Random (R). The number of rows m of the wafer particles is 52, and the number of columns n is 52. When training the model, all normal samples are used, a total of 1000 images. When testing, 10 samples are randomly selected from each category to form a test set, a total of 90 images, which are used to test the unsupervised anomaly detection and clustering effects of this method.

[0049] As Figure 1 shown, the unsupervised wafer map anomaly detection method based on an autoencoder provided by the present invention includes the following steps:

[0050] Step 1: Perform median filtering preprocessing on the original wafer images in the embodiment.

[0051] First, set the neighborhood radius r = 3, that is, define a neighborhood range of 3×3 for each pixel in the image. Then, sort the pixel values within this range, and replace each pixel value with the median of the pixel values within its neighborhood range. The comparison of some wafer maps before and after filtering is as Figure 2 shown.

[0052] Step 2: Build an autoencoder model, input the normal wafer maps into the model for multiple rounds of training, and calculate the judgment threshold according to the reconstruction error.

[0053] In the embodiment, the autoencoder model is as Figure 3 shown. The model is built in the Pytorch environment. The specific building method and the parameters of each layer of the model are:

[0054] (a) Add an input layer in the Encoder part, with an input size of 52×52 and 3 channels;

[0055] (b) Add a convolutional block in the Encoder part, consisting of a convolutional layer, a ReLU layer, and a CBAM layer. The convolutional kernel size E f1 ×E f1 is 3×3, the number of channels E c1 is 16, and the stride E s1 is 1;

[0056] (c) Add a convolutional block in the Encoder part, consisting of a convolutional layer, a ReLU layer, and a CBAM layer. The convolutional kernel size E f2 ×E f2 is 3×3, the number of channels Ec2 is 32, with a step size of E s2 is 1;

[0057] (d) Add a convolutional block to the Encoder part, consisting of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of E f3 ×E f3 is 1×1, and the number of channels is E c3 is 64, with a step size of E s3 is 1;

[0058] (e) Add a convolutional block to the Encoder part, consisting of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of E f4 ×E f4 is 1×1, and the number of channels is E c4 is 128, with a step size of E s4 is 1;

[0059] (f) Add a transposed convolutional block to the Decoder part, consisting of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of D f1 ×D f1 is 1×1, and the number of channels is D c1 is 64, with a step size of D s1 is 1;

[0060] (g) Add a transposed convolutional block to the Decoder part, consisting of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of D f2 ×D f2 is 1×1, and the number of channels is D c2 is 32, with a step size of D s2 is 1;

[0061] (h) Add a transposed convolutional block to the Decoder part, consisting of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of D f3 ×D f3 is 3×3, and the number of channels is D c3 is 16, with a step size of D s3 is 1;

[0062] (i) Add a transposed convolutional block to the Decoder part, consisting of a convolutional layer, a ReLU layer, and a CBAM layer, with a kernel size of D f4 ×D f4 is 3×3, and the number of channels is D c4 is 3, with a step size of D s4 is 1;

[0063] (j) Add an activation layer with a Sigmoid activation function to the Decoder part as the output layer.

[0064] By introducing the CBAM module into the model, the multi-scale feature capture ability of the model is enhanced.

[0065] The filtered normal images are used as the training set and input into the autoencoder model. It is trained for 50 Epochs with the Adam optimizer, and the learning rate is set to 0.001. After feature compression and reconstruction, the difference between the output reconstructed image and the original image is quantified to obtain the reconstruction error recon_error. The mean reconstruction error mean_recon_error and the standard deviation of the reconstruction error std_recon_error of all normal samples are calculated. Then, according to the Gaussian distribution, the judgment threshold is calculated as thresho l d = mean_recon_error + 2×std_recon_error. The judgment threshold obtained in this embodiment is 0.0029.

[0066] Step 3: Input the test set into the autoencoder model, compare the size of the reconstruction error with the judgment threshold, and realize the detection of unlabeled abnormal wafer images.

[0067] After experiments, the reconstruction error values output by the 80 abnormal samples in the test set of this embodiment after passing through the autoencoder model are all greater than 0.0029, and the value range distribution interval is [0.017, 0.52]. The reconstruction error values output by the 10 normal samples after passing through the autoencoder model are all less than 0.0029, and the value range distribution interval is [0.00012, 0.0018]. Figure 4 For the visualization results of the reconstruction of some samples, it can be seen that the reconstruction effect of normal samples is better, so the reconstruction error value is smaller, while the reconstruction effect of abnormal samples is worse, so the reconstruction error value is larger. Abnormal detection is realized under the unsupervised condition of only training normal samples without labeling abnormal labels.

[0068] In this embodiment, further: according to the pixel value difference between the input abnormal image and the model reconstructed image when the abnormal image is input, an abnormal image reconstruction error map is obtained. Multiple statistical parameter values of the abnormal image reconstruction error map are selected and calculated, and they are used as clustering features to cluster unlabeled abnormal wafer images.

[0069] In this embodiment, 90 abnormal samples in the test set are detected in step three. The average value of the pixel values of the reconstruction error map obtained by these samples through the autoencoder model, the standard deviation of the pixel value distribution, the size of the abnormal pixel region, the x coordinate of the centroid of the abnormal region, the y coordinate of the centroid of the abnormal region, the dispersion degree of abnormal pixels, the edge gradient size of the abnormal region, and the edge curvature of the abnormal region are calculated. These 8 statistical indicators are used as features and are input into the K-Means clustering algorithm after weighting. The feature weights in this embodiment are set to [2, 2.5, 1, 2, 0.5, 1, 0.5, 0.5], the number of clustering clusters is set to 8, the number of times of randomly running with different centroid seeds is set to 10 times, and the maximum number of iterations for a single run of the K-Means algorithm is set to 300. The silhouette coefficient obtained from the experiment is 0.7038, and the t-SNE clustering effect diagram is as shown in Figure 5 shown. Each clustering cluster is relatively distinct. By comparing the results of each clustering with the self-labels of the samples within the class, the sample labels in each clustering in this embodiment basically belong to the same class, indicating that the clustering effect is relatively ideal.

[0070] The above content is only an example and explanation of the structure of the present invention. Those skilled in the art of this technology can make various modifications or supplements to the described specific embodiments or use similar methods for substitution. As long as they do not deviate from the structure of the invention or exceed the scope defined by this claim book, they should fall within the protection scope of the present invention.

Claims

1. An unsupervised wafer image anomaly detection method based on an autoencoder, characterized in that: The steps include: Step 1: Perform filtering preprocessing on the test image including the normal wafer image and the abnormal wafer image; Step 2: Build an autoencoder model, input the normal wafer image after filtering preprocessing into the model for multiple rounds of training, and calculate the judgment threshold based on the reconstruction error; Step 3: Input the test image after filtering preprocessing into the trained autoencoder model, and compare the reconstruction error with the judgment threshold. If the reconstruction error is less than the judgment threshold, it is a normal wafer image, otherwise it is an abnormal wafer image.

2. The unsupervised wafer image anomaly detection method based on an autoencoder according to claim 1, characterized in that: The algorithm used in step 1 filtering preprocessing is the median filtering algorithm; The median filtering algorithm defines a neighborhood of size r×r for each pixel of the image, where r is the radius of the neighborhood, sorts the pixel values ​​within the neighborhood, and then replaces each pixel value with the median value of its neighborhood value.

3. The unsupervised wafer image anomaly detection method based on an autoencoder according to claim 1, characterized in that: The specific implementation process of step 2 is as follows: (a) According to the wafer particles, the normal wafer image is adjusted to a size of m×n, where m and n are the number of rows and columns of wafer particles; (b) constructing an autoencoder model, wherein the autoencoder model includes an encoder and a decoder, and a channel space attention module added to both the encoder and the decoder for enhancing feature capture capability, and compressing and reconstructing a normal wafer image; (c) calculating a reconstruction error between a reconstructed image output by the autoencoder model of a normal wafer image and the original image, wherein the reconstruction error is formed by the sum of a mean absolute error and a mean square error; (d) Calculating a judgment threshold according to the reconstruction error. The judgment threshold is equal to the average value of the reconstruction error plus twice the standard deviation of the reconstruction error and obeys Gaussian distribution.

4. The unsupervised wafer image anomaly detection method based on an autoencoder according to claim 3, characterized in that: The specific establishment method and parameters of each layer of the autoencoder model are: (a) Add an input layer in the Encoder part with an input size of m×n and 3 channels; (b) Add a convolution block to the encoder, which consists of a convolution layer, a ReLU layer, and a CBAM layer. The convolution kernel size is E. f1 ×E f1 , the number of channels is E c1 , the step length is E s1 ; (c) Add a convolution block to the encoder, which consists of a convolution layer, a ReLU layer, and a CBAM layer. The convolution kernel size is E. f2 ×E f2 , the number of channels is E c2 , the step length is E s2 ; (d) Add a convolution block to the encoder, which consists of a convolution layer, a ReLU layer, and a CBAM layer. The convolution kernel size is E. f3 ×E f3 , the number of channels is E c3 , the step length is E s3 ; (e) Add a convolution block to the encoder, which consists of a convolution layer, a ReLU layer, and a CBAM layer. The convolution kernel size is E. f4 ×E f4 , the number of channels is E c4 , the step length is E s4 ; (f) Add a deconvolution block in the decoder part, which consists of a convolution layer, a ReLU layer, and a CBAM layer, and the convolution kernel size is D f1 ×D f1 , the number of channels is D c1 , the step length is D s1 ; (g) Add a deconvolution block in the Decoder part, which consists of a convolution layer, a ReLU layer, and a CBAM layer, and the convolution kernel size is D f2 ×D f2 , the number of channels is D c2 , the step length is D s2 ; (h) Add a deconvolution block in the Decoder part, which consists of a convolution layer, a ReLU layer, and a CBAM layer, and the convolution kernel size is D f3 ×D f3 , the number of channels is D c3 , the step length is D s3 ; (i) Add a deconvolution block in the decoder part, which consists of a convolutional layer, a ReLU layer, and a CBAM layer, with a convolution kernel size of D. f4 ×D f4 , the number of channels is D c4 , the step length is D s4 ; (j) Add an activation layer using Sigmoid activation function as the output layer in the Decoder part.

5. The unsupervised wafer image anomaly detection method based on an autoencoder according to claim 1, characterized in that: The specific implementation process of step three is as follows: (a) The test image is adjusted to a size of m×n, where m and n are the number of rows and columns of wafer particles; (b) calculating the reconstruction error between the reconstructed image output by the autoencoder model and the original image of the test image, and comparing the reconstruction error with the judgment threshold. If the reconstruction error is less than the judgment threshold, it is a normal wafer image, otherwise it is an abnormal wafer image.

6. An unsupervised wafer image anomaly clustering method based on an autoencoder, characterized in that: When an abnormal image is input, the pixel value difference between the input image and the model reconstructed image is used to obtain an abnormal image reconstruction error map, and multiple statistical parameters of the abnormal image reconstruction error map are selected and calculated as clustering features. Clustering unsupervised anomaly wafer graphs.

7. The unsupervised wafer image anomaly clustering method based on an autoencoder according to claim 6, characterized in that: The specific implementation process is as follows: (a) Superimpose the channels of the abnormal image reconstruction error map to obtain a single-channel two-dimensional error map; (b) calculating a number of statistics of the abnormal image reconstruction error map, including the average value of pixel values, the standard deviation of pixel value distribution, the size of abnormal pixel region, the x-coordinate of the centroid of the abnormal region, the y-coordinate of the centroid of the abnormal region, the dispersion of abnormal pixels, the size of the gradient at the edge of the abnormal region, and the curvature at the edge of the abnormal region; (c) The statistic is used as a feature basis for clustering, and after minimum-maximum normalization and weighted operation processing, it is input into a K-Means clustering algorithm to cluster unsupervised abnormal wafer images.

Images