Wafer defect positioning method based on image block guided generative adversarial network
By using an improved image patch-guided generative adversarial network, the problem of high-precision detection of minute defects during wafer coating is solved, realizing a low-cost and efficient defect localization method that does not require labeled samples and is suitable for wafer inspection in complex backgrounds.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-01
- Publication Date
- 2026-03-27
AI Technical Summary
Existing technologies lack high-precision, low-cost defect detection methods suitable for complex backgrounds during wafer coating processes, especially for detecting minute defects. Furthermore, traditional methods rely on manual experience or require a large amount of labeled data.
An improved method based on image patch-guided generative adversarial networks (GANomaly) is adopted. Through training with defect-free samples, combined with a lightweight encoder-decoder-discriminator structure, post-reconstruction consistency constraints, and optimized loss function, accurate detection and localization of wafer background is achieved.
It enables low-cost inspection without the need for labeled samples, improves the sensitivity and accuracy of inspection for minute defects, reduces false positives and false negatives, and is suitable for online inspection of large-size wafers.
Smart Images
Figure CN121746307A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of semiconductor manufacturing detection, and particularly relates to a wafer glue coating defect positioning method based on an image block guided generative adversarial network, which can be applied to automatic detection and process control of a wafer glue coating process. BACKGROUND
[0002] In a photolithography process, the glue coating process is one of the key steps to determine the pattern quality and yield of the finished product. If the tiny defects (such as bubbles, particles, stripes, etc.) on the wafer surface are not detected in time, they will be amplified step by step in the subsequent exposure and development links, eventually leading to circuit failure or yield reduction. Therefore, it is of great significance to achieve high-precision and automated defect detection in the glue coating stage.
[0003] Current detection methods mainly fall into two categories: (1) Traditional machine vision method: relying on edge detection, texture analysis, threshold segmentation and other algorithms to identify abnormal areas, but it is easily affected by background interference and heavily dependent on human experience, with poor robustness.
[0004] (2) Supervised method based on deep learning: identifying defects through convolutional neural networks (CNN) or target detection networks (such as YOLO, Faster R-CNN), with high detection accuracy, but training relies on a large amount of labeled defect data. In actual production, there are many types of wafer defects and sample scarcity, with high labeling cost, making it difficult to meet real-time applications.
[0005] Therefore, researchers have proposed unsupervised learning methods such as AE, VAE, AnoGAN and GANomaly, which achieve image reconstruction and anomaly detection by training on defect-free samples. However, these methods have the following shortcomings when dealing with large-size wafer images: (1) unstable training process, with the model easily falling into fuzzy reconstruction; (2) not sensitive to subtle defects; (3) prone to missing boundary defects after image segmentation; (4) unable to achieve accurate positioning in complex wafer backgrounds.
[0006] Therefore, the prior art still lacks a wafer glue coating defect positioning method that takes into account detection accuracy, generalization and computational efficiency. SUMMARY
[0007] To overcome the above-mentioned shortcomings of the prior art, the present application proposes a wafer defect positioning method based on an image block guided generative adversarial network, which only relies on defect-free samples for training to achieve accurate detection and positioning of subtle glue coating defects in complex wafer backgrounds.
[0008] (I) Technical solution In a first aspect, the present application provides a wafer defect positioning method based on an image block guided generative adversarial network, comprising: Collecting gray-scale wafer images before and after the gluing process, and performing sliding segmentation on each original image according to a fixed size window and a set step to obtain a plurality of local image blocks; performing preliminary screening on the local image blocks based on statistical characteristics, eliminating background, noise and invalid areas, and then performing secondary screening on the image blocks screened in the preliminary screening to obtain defect-free image blocks, which are used as a training data set; An improved GANomaly network is constructed and trained using the training data set; In the test phase, the gray-scale wafer images after the gluing process are not screened, and are segmented into a plurality of local image blocks, which are input into the trained improved GANomaly network and the corresponding reconstructed image blocks are generated by the decoder; After the residual mapping of each reconstructed image block, the image is spliced into a defect heat map of the whole wafer, and the boundary positioning and labeling of the defect area are realized by binarizing the heat map through the threshold segmentation method.
[0009] In one embodiment, the implementation process of the preliminary screening of the image blocks based on statistical characteristics is as follows: if the pixel mean of the image block is lower than the threshold, it is eliminated, otherwise it is continuously judged; if the variance of the adjacent pixels of the image block is lower than the threshold, it is eliminated, otherwise it is not processed.
[0010] In one embodiment, the improved GANomaly network is composed of an encoder, a decoder and a discriminator; The input image block x is input into the encoder, and the encoder outputs the encoder output latent feature vector ; The encoder output latent feature vector is input into the decoder, and the decoder outputs the reconstructed image block ; The reconstructed image block is returned to the encoder again; The discriminator is used to predict the class label of the input image, and the input image block x and the reconstructed image block are classified.
[0011] In one embodiment, the loss function of the improved GANomaly network is composed of a reconstruction loss , a latent consistency loss and an adversarial loss , wherein the reconstruction loss is used to constrain the pixel consistency of the encoder input image block and the reconstructed image block , the latent consistency loss ensures that the representation of the input image block and the reconstructed image block in the latent space remains consistent, and the adversarial loss guides the generator to improve the authenticity of the reconstructed image block, so that it can deceive the discriminator.
[0012] In one implementation, the image block is reconstructed based on the reconstructed image block and the reconstructed image block The residual map is calculated As follows: (1)
[0013] In a second aspect, the application provides a wafer defect positioning method, comprising: A data acquisition module is responsible for acquiring a gray-scale wafer image after a gluing process, and performing sliding segmentation on the image according to a fixed-size window and a set step to obtain a plurality of local image blocks; A reconstruction module is responsible for inputting the local image blocks into the improved GANomaly network trained and generating corresponding reconstructed image blocks by the decoder; A defect positioning module is responsible for reassembling the reconstructed image blocks after the residual mapping into a defect heat map of the entire wafer, combining a threshold segmentation method, and realizing the boundary positioning and labeling of the defect area by binarizing the heat map.
[0014] In a third aspect, the application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed in a computer, the computer executes the method.
[0015] In a fourth aspect, the application provides a computing device comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method is implemented.
[0016] (II) Advantages Compared with the prior art, the application has the following advantages: 1. No need to label samples, reduce data cost: the improved GANomaly network model trained by the application only relies on defect-free sample training, avoiding the high cost and privacy risk of a large number of defect sample labeling.
[0017] 2. More sensitive detection: the "latent consistency loss" strengthens the learning of latent features, maintains the consistency of input and reconstruction features in the latent space, and makes the improved GANomaly network model more sensitive to low-contrast and subtle defects.
[0018] 3. Better reconstruction effect: after optimizing the loss ratio of the improved GANomaly network, the problem of over-smoothing reconstruction of the traditional GANomaly is avoided, and the residual distribution has more physical meaning.
[0019] 4. Lightweight structure, fast detection speed: each of the decoder and the discriminator has three convolutional layers, effectively reducing the computational complexity, and suitable for online detection of large-size wafers.
[0020] 5. Robustness: The patch strategy combining pixel statistics screening and manual review significantly reduces false positives and false negatives, and improves model generalization. BRIEF DESCRIPTION OF DRAWINGS
[0021] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.
[0022] Figure 1 are wafer images before and after coating captured by a line scan camera, where (a) is before coating, and (b) is after coating.
[0023] Figure 2 is a structural diagram of the improved GANomaly network.
[0024] Figure 3 is a defect heat map and a binary image obtained by splicing the reconstructed image blocks of the improved GANomaly network, where (a) is the defect heat map, and (b) is the corresponding binary image.
[0025] Figure 4 is a wafer defect positioning method flowchart based on the image block guided generative adversarial network provided by the embodiment of the present application.
[0026] Figure 5 is a pixel-level labeling result image of the coating image defect provided by the embodiment of the present application. DETAILED DESCRIPTION
[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.
[0028] At least one embodiment provides a wafer defect positioning method based on an image block guided generative adversarial network, referring to the flowchart of the wafer defect positioning method based on the image block guided generative adversarial network provided by the embodiment of the present application. Figure 4 , comprising: Step 1: Data acquisition and preprocessing The gray-scale wafer images before and after the coating process are collected by a line scan camera in a wafer detection system, as shown in Figure 1 (a) in the figure. Figure 1As shown in (b), each original image is divided into several local image patches by sliding a window of a fixed size (e.g., 128×128 pixels) and a set step size (e.g., 64 pixels). These patches are then used as the images to be detected. Subsequently, the image patches are initially screened based on statistical features: if the pixel mean is below a threshold, it is judged as background or noise and removed; otherwise, further judgment is made. If the variance of adjacent pixels in an image patch is below a threshold, it is judged as too smooth and lacks effective structural information and removed. If necessary, image patches with excessively high standard deviations are further filtered to avoid introducing noise or defective regions. Finally, after manual review, only defect-free image patches are retained as the training dataset.
[0029] Step Two: Model Improvement and Training This invention improves the GANomaly network by modifying its framework, introducing a collaborative structure of encoder, decoder, and discriminator, such as... Figure 2 As shown.
[0030] The process of improving the implementation of GANomaly networks is as follows: The input image patch x is fed into the encoder, and the encoder outputs a latent feature vector. The encoder outputs latent feature vectors. The input is fed into the decoder, and the decoder outputs the reconstructed image patch. Reconstructing image patches The input is then returned to the encoder; the discriminator is used to predict the class label of the input image, comparing the input image patch x with the reconstructed image patch x. The classification is performed. The encoder, decoder, and discriminator each have three convolutional layers.
[0031] To balance reconstruction quality and potential consistency, the loss function of the improved GANomaly network is modified by replacing the reconstruction loss. Potential consistency loss Combating losses Composition, namely:
[0032] in These are the reconstruction losses Potential consistency loss Combating losses The weights, and All are greater than 0; The reconstruction loss is used to constrain the encoder input image blocks. With reconstructed image patches Pixel consistency is represented as follows:
[0033] The potential consistency loss ensures that the input image block and the reconstructed image block are consistent in the representation of the latent space, represented as follows:
[0034] wherein represents the latent feature representation of the input image block x after the encoder (Encoder), represents the reconstructed image block latent feature representation after re-encoding; The adversarial loss guides the generator to improve the authenticity of the reconstructed image block, so that it can deceive the discriminator, represented as follows:
[0035] wherein represents the feature extraction function of the input image block x after the encoder (Encoder), represents the reconstructed image block feature extraction function after re-encoding; Preferably, the reconstruction error, the potential consistency and the adversarial constraint are considered in the loss function, and the weights of the three losses are set to =1:1:1 to enhance the learning ability of the model for image features and the training stability.
[0036] The improved network is trained using the aforementioned defect-free image blocks as training data, so that it can accurately represent the latent distribution of normal samples and has high fidelity reconstruction capability.
[0037] Step three: defect positioning and visualization In the test phase, the gray wafer image after the gluing process is cut into several local image blocks of fixed size steps, and the unselected local image blocks are input into the improved GANomaly network trained in step two and the corresponding reconstructed image blocks are generated by the decoder ; According to the image block and the reconstructed image block , the residual mapping error is calculated :
[0038] If the residual mapping error is significantly large, it is determined that the image block has defects.
[0039] Subsequently, each image block is reassembled into a defect heat map of the whole wafer after residual mapping of the image, and the boundary positioning and labeling of the defect area are realized by binarizing the heat map through the threshold segmentation method, such as Figure 3Fig. 2 shows the defect detection results of the improved GANomaly network. Figure 3 Fig. 2 shows the defect detection results of the improved GANomaly network.
[0040] Step four: model defect detection data comparison To visually show the improvement effect, the operation of step three is performed on the improved GANomaly network without potential consistency constraints, the original GANomaly network with a loss weight ratio of 50:1:1, and the original GANomaly network with a loss weight ratio of 1:1:1 to obtain binary images, respectively. At the same time, the pixel-level labels of the defect areas in the wafer images after gluing shown in Fig. 2b are obtained, as shown in Fig. 2c. Figure 1 Fig. 2 shows the defect detection results of the improved GANomaly network. Figure 5
[0041] Subsequently, the white pixels in the four obtained binary images are compared and analyzed with the labeled defect areas, and the corresponding quantitative evaluation results are shown in Table 1.
[0042] Table 1 Performance comparison of different GANomaly network variants
[0043] In addition, the prior art mainly relies on two methods: one is a traditional visual algorithm based on manual features, which judges defects by extracting gray, texture or morphological features; the other is a supervised detection network based on deep learning, which needs a large number of labeled defect samples for training. The former is limited by human experience and is difficult to cope with complex wafer backgrounds; the latter is limited by the confidentiality of semiconductor manufacturing data and the cost of labeling, and is difficult to promote in actual production.
[0044] Existing unsupervised detection methods (such as AE, VAE, AnoGAN, GANomaly) can be trained based on only defect-free samples, but generally have problems such as insensitivity to subtle defects and overly smooth reconstruction results, resulting in insufficient detection accuracy.
[0045] The present application proposes three key improvements based on the GANomaly framework to solve the above problems: First, a lightweight encoder-decoder-discriminator structure is proposed. Compared with the deeper convolution layer design in the original GANomaly, the present application only uses three layers of convolution and deconvolution structure in the encoder and decoder part, reducing the network parameter quantity and computational complexity. This lightweight structure not only significantly improves the inference speed and deployability of the model, but also avoids the problems of gradient disappearance and overfitting that are prone to occur in deep networks.
[0046] Secondly, the "reconstruction-then-encoding consistency constraint" is introduced. This constraint not only focuses on the reconstruction error of the pixel layer, but also enforces the structural consistency between the input image and the reconstructed image in the latent feature space, enabling the model to accurately capture the local structural differences of the image. This improvement significantly improves the model's sensitivity to subtle glue defects in complex wafer backgrounds, achieving more accurate defect positioning.
[0047] Thirdly, the weights of the three types of loss functions in the original GANomaly are optimized and adjusted, adjusting the ratio of reconstruction loss, latent consistency loss, and adversarial loss to 1:1:1, thereby balancing the reconstruction quality and defect sensitivity. This improvement avoids the problem of image over-smoothing caused by excessive weight of the reconstruction term, making the reconstructed image closer to the true defect-free structure, and the residual signal clearer and the defect positioning more accurate.
[0048] In addition, the present application combines a patch-level data screening mechanism based on pixel mean, standard deviation, and manual review to ensure that the training data set only contains pure defect-free samples. Through this mechanism, background noise and unstable areas are effectively excluded, improving the robustness and generalization ability of the model.
[0049] In summary, the present application not only strengthens the latent space consistency learning mechanism in the network structure, but also optimizes the loss weight and data screening strategy, thereby achieving high-precision glue defect detection and positioning under complex wafer texture conditions.
[0050] The present embodiment also provides a wafer defect positioning method, comprising: A data acquisition module is responsible for acquiring a gray-scale wafer image after the glue application process, and performing sliding segmentation on the image according to a fixed size window and a set step size to obtain a plurality of local image blocks; A reconstruction module is responsible for inputting the local image blocks into the improved GANomaly network trained and generating corresponding reconstructed image blocks by the decoder; A defect positioning module is responsible for reassembling the reconstructed image blocks after residual mapping into a defect heat map of the entire wafer, and combining a threshold segmentation method to realize the boundary positioning and labeling of the defect area by binarizing the heat map.
[0051] The above is a preferred embodiment of the present application. It should be noted that for ordinary skilled persons in the technical field, without departing from the principles of the present application, several improvements and refinements can be made, which are also considered within the scope of protection of the present application.
Claims
1. A wafer defect localization method based on image patch-guided generative adversarial networks, characterized in that, The method includes: Grayscale wafer images before and after the coating process are acquired, and each original image is slidably segmented according to a fixed-size window and a set step size to obtain several local image blocks. Based on statistical features, the local image blocks are initially screened to remove background, noise, and invalid areas. The image blocks after the initial screening are then screened a second time to obtain defect-free image blocks, which are used as the training dataset. An improved GANomaly network was constructed and trained using the training dataset; During the testing phase, the unscreened grayscale wafer images after the coating process are divided into several local image blocks. The local image blocks are input into the trained improved GANomaly network and the decoder generates the corresponding reconstructed image blocks. After residual mapping, the reconstructed image blocks are stitched together to form a complete defect heat map of the wafer. By combining threshold segmentation, the boundary location and labeling of the defect region are achieved by binarizing the heat map.
2. The method according to claim 1, characterized in that, The process of preliminary screening of image blocks based on statistical features is as follows: if the pixel mean of an image block is lower than a threshold, it is removed; otherwise, the judgment continues. If the variance of adjacent pixels of an image block is lower than a threshold, it is removed; otherwise, no processing is performed.
3. The method according to claim 1, characterized in that, The improved GANomaly network consists of an encoder, a decoder, and a discriminator; The input image patch x is fed into the encoder, and the encoder outputs a latent feature vector. encoder The latent feature vector is output and input to the decoder, which outputs a reconstructed image patch; the reconstructed image patch is then returned to the encoder as input. The discriminator is used to predict the class label of the input image and classify the input image patch and the reconstructed image patch.
4. The method according to claim 1 or 3, characterized in that, The loss function of the improved GANomaly network consists of reconstruction loss. Potential consistency loss Combating losses The structure includes a reconstruction loss used to constrain the pixel consistency between the encoder input image block and the reconstructed image block, a latent consistency loss to ensure that the representations of the input image block and the reconstructed image block are consistent in the latent space, and an adversarial loss to guide the generator to improve the realism of the reconstructed image block so that it can deceive the discriminator.
5. The method according to claim 1, characterized in that, According to image blocks With reconstructed image patches Calculate residual mapping ,as follows: （1）。 6. A wafer defect location method based on any one of claims 1-5, characterized in that, include: The data acquisition module is responsible for acquiring grayscale wafer images after the coating process and sliding the images according to a fixed-size window and a set step size to obtain several local image blocks; The reconstruction module is responsible for inputting local image patches into the trained improved GANomaly network and having the decoder generate the corresponding reconstructed image patches. The defect localization module is responsible for re-stitching the reconstructed image blocks into a complete defect heat map of the wafer after residual mapping. Combined with threshold segmentation, the module achieves the boundary localization and labeling of defect areas by binarizing the heat map.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed in the computer, it causes the computer to perform the method of any one of claims 1-5.
8. A computing device, comprising a memory and a processor, characterized in that, The memory stores executable code, and when the processor executes the executable code, it implements the method of any one of claims 1-5.