Wafer defect detection method based on double twin networks

Abstract

The invention discloses a wafer defect detection method based on a dual twin network, and the method carries out the training and feature extraction of a defect-free wafer image and a defect wafer image on the surface of a wafer through a dual neural network, can obtain more feature information, and improves the wafer intermode detection capability based on deep learning. The method comprises the following specific steps: acquiring an image of a wafer to be detected by a camera, and preprocessing the acquired image; generating a pseudo-defect image on the defect-free wafer image by using an adversarial network, and performing various transformations on original data through data enhancement to generate a new twin network training sample; and reconstructing and repairing the discriminant features to finally obtain a predicted image. A data set of a twin network is manufactured for training, and a trained model is used for detecting wafer defects; and inputting a wafer image to be detected into the twin network, and generating a prediction image by calculating a difference value between an input feature set Fm1 and an output feature set FAE1 of the twin network.

Description

Technical Field

[0001] The present invention belongs to the technical field of wafer defect detection, and in particular relates to a wafer defect detection method based on a dual twin network. Background Art

[0002] Wafers are the fundamental materials of semiconductor devices. During the production of integrated circuits (ICs), various types of defects may appear on the wafer surface. If these defects are not discovered and addressed in a timely manner, they will have a serious impact on the functionality and reliability of the chip.

[0003] To promptly detect and control surface defects on wafer products to avoid adverse effects and ensure the normal function of subsequent chips, deep learning-based methods using convolutional neural networks are often used to improve the accuracy of chip wafer defect detection and enhance the quality of chip devices. However, these methods only use test images to inspect wafers, without obtaining information from standard defect-free images. Furthermore, previous methods may not be able to detect some smaller defects. Summary of the Invention

[0004] To address these shortcomings, the present invention proposes a wafer defect detection method based on a dual twin network, which performs feature fusion through the MTUF (Multi-scale Top-down Unified Fusion) module. The twin network inputs defect-free mold images and defective mold images, extracts normal features and abnormal features through the dual twin network, and reconstructs and repairs these features, which can effectively process normal and abnormal information.

[0005] The technical problem to be solved by the present invention can be achieved through the following technical solutions:

[0006] The wafer defect detection method based on the dual twin network specifically comprises the following steps:

[0007] S10. Image acquisition: The camera acquires images of the wafer to be inspected and preprocesses the acquired images. The wafer images include images of defect-free wafers and images of defective wafers.

[0008] S20. Generate defective images using an adversarial network: Use an adversarial network to generate pseudo-defective images on defect-free wafer images. Transform the original data through data augmentation to generate new Siamese network training samples.

[0009] S30. Constructing a twin network: This includes two twin networks that reconstruct and repair the discriminant features to ultimately obtain a predicted image.

[0010] S40. Network Training: Use defect-free and defective images as datasets for the twin network training. The network is trained by minimizing the loss function, enabling it to learn the distribution of defect-free areas. The trained model is then used to detect wafer defects and distinguish between defective and non-defective areas.

[0011] S50. Reasoning: Input the image to be tested into the twin network, and calculate the input feature set F of a sub-network of the second twin network. m1 And the output feature set F AE1 The difference between generates a predicted image;

[0012] The adversarial network consists of two key components: a generator for creating new samples and a discriminator for evaluating sample authenticity. After wafer images undergo adversarial learning between the generator and discriminator, a dataset of pseudo-defect images is generated. The adversarial learning training process between the generator and discriminator in the adversarial network enables interactive learning, creating a generative model (composed of two networks, the generator and the discriminator) that resembles the distribution of real images. The loss function for this generative model is as follows:

[0013] WGAN (Wasserstein Generative Adversarial Network) loss includes generator loss L G and the discriminator loss L D , the expression is as follows:

[0014]

[0015] Where D(x) is the score of the discriminator for the real image, and D(G(z)) is the score of the discriminator for the generated image; For the real data distribution p data The expected value of all samples x of (x); Denotes the noise distribution p from the prior z (z) is the expected value of all random variables z in the generative adversarial network. In the generative adversarial network, these random variables z are random noise vectors input to the generator, which are converted into fake samples by the generator; G(z) is the result of the generator G acting on the random noise vector z, that is, the generator generates a fake sample based on the input noise vector, and the goal is to make this fake sample as close to the real data sample as possible; p z (z) is the distribution of the random noise vector z input to the generator; z is the random noise vector input to the generator, and x is the distribution of the real data p data A sample drawn from (x);

[0016] The perceptual loss measures the difference between the generated image G(z) and the real image x in the feature space:

[0017]

[0018] Among them, L perceptual It is the overall expression of the perceptual loss function, which is used to quantify the difference in feature representation between the generated image G(z) and the real image x; It is a specific component of the perceptual loss, which represents the squared Euclidean distance to the feature map of the i-th layer, that is, the square of the L2 norm. Here, φ i refers to the i-th layer feature extractor of a pre-trained convolutional neural network, φ i (G(z)) is the feature extracted from the generated image, and φ i (x) is the feature extracted from the real image;

[0019] Perceptual loss can make the network's generated images visually closer to real images. The final generator loss is the weighted sum of WGAN loss and perceptual loss.

[0020]

[0021] Among them, λ WGAN and λ perceptual is the weight coefficient, which is used to adjust the importance of the two parts of loss.

[0022] Furthermore, the generator of the adversarial network is a four-layer recurrent neural network. By obtaining the distribution of real images and inputting the noise source into the generator, a pseudo image similar to the real image is generated. The discriminator is a three-layer neural network of binary classifiers, which is responsible for evaluating the authenticity of pseudo images and distinguishing real images from simulated images.

[0023] Furthermore, the generator and discriminator are trained simultaneously by minimizing an adversarial loss function. The specific process is as follows:

[0024] Training the discriminator: Randomly sample a batch of real images x from the real defect image dataset; sample a batch of random noise z from the random noise distribution, and generate a batch of fake defect images G(z) through the generator; fix the parameters of the generator, calculate the gradient based on the discriminator's loss function, and update the discriminator's parameters so that the discriminator can better distinguish between real images and generated images;

[0025] Training the generator: Sample a batch of random noise z from the random noise distribution, fix the parameters of the discriminator, calculate the gradient according to the generator's loss function, and update the generator's parameters so that the image generated by the generator is closer to the real image, thereby deceiving the discriminator;

[0026] Iterative training: Repeat the above alternating training steps until the pseudo-defect images generated by the generator are realistic enough and the discriminator has difficulty distinguishing between real images and generated images. During the training process, as the discriminator's ability improves, the generator will be further improved, thereby continuously improving the quality of the generated images.

[0027] Furthermore, in step S30, the first network uses a twin network structure to simultaneously process defect-free wafer image samples and defective wafer image samples to extract discriminative features, and uses the VGG16 (convolutional neural network) network as a feature extractor to extract features from different levels; the structure of the second twin network adopts an autoencoder, and inputs the output features from the first twin network for reconstruction and repair. By reconstructing and repairing the discriminant features, a predicted image is finally obtained.

[0028] Furthermore, in step S40, the specific steps include:

[0029] Step S41: inputting the normal wafer image and the corresponding defective wafer image into the twin network for feature extraction simultaneously;

[0030] Step S42: The feature extraction twin network extracts a complete set of discriminative positive sample features and their corresponding abnormal features, and uses the autoencoder (AE) as the backbone convolutional neural network (DCNN) of the defect perception twin network to reconstruct and repair the fused feature set;

[0031] Step S43: In the inference stage, the sample to be inspected is used as the dual input of the feature extraction twin network, and the difference between the input and its reconstructed and repaired feature set is calculated to generate the final anomaly map.

[0032] Furthermore, during the training process, the validation set is used to evaluate the performance of the model, and the training strategy is adjusted based on the performance feedback. The loss function used is as follows:

[0033]

[0034] Where h and w represent the height and width of the fused feature map, respectively. j and i are index variables used to traverse the entire feature map. j is used to traverse the height direction of the feature map, while i is used to traverse the width direction. λ is a weight parameter used to balance the ratio between reconstruction loss and restoration loss. It is a summation symbol, which means that the errors at all positions on the feature map are accumulated;

[0035] L reconstruction and L inpainting is the loss function in the reconstruction and repair process, L reconstruction It is used to evaluate the similarity between the reconstructed features and the original defect-free wafer image, so that the feature set FAE1 Close to normal sample F m1 , to obtain the reconstruction effect; L inpainting It is used to evaluate the similarity between the features after defect repair and the original defect-free wafer mold image, so that the repaired defect feature set F AE2 For the repair effect to be as close to F as possible m1 ; F m1 is the feature set of the original normal wafer image sample, F AE1 is the feature set after normal wafer image sample reconstruction, F AE2 is the feature set of the defective wafer image sample after repair, L reconstruction and L inpainting A pixel error metric with mean squared error loss is selected to improve segmentation accuracy and computational speed.

[0036] Compared with the prior art, the present invention has the following advantages:

[0037] (1) Lighter and more efficient network structure: Compared with the patent that uses heavy modules such as Swin-Transformer, which has a high dependence on computing resources, this patent is based on the VGG16 backbone + multi-scale feature fusion (MTUF) + AE reconstruction mechanism, which is lighter in calculation and more friendly to deployment, especially suitable for edge deployment or rapid detection needs in industrial scenarios.

[0038] (2) Stronger structural hierarchical understanding capability: Compared with the previous patent which mainly uses the patch-level Transformer, which has strong semantics but may sacrifice spatial consistency, this patent uses MTUF to fuse the multi-level features of each block of VGG16, guiding AE to better understand the position and structural relationship between defects and non-defective areas, avoiding "image-level modeling" errors and improving positioning accuracy.

[0039] (3) The GAN loss function is optimized to generate more realistic defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Flow chart of the detection method of the present invention;

[0041] Figure 2 This is a diagram of the architecture of the adversarial network of the present invention;

[0042] Figure 3 This is a flowchart of the twin network of the present invention;

[0043] Figure 4 This is a flowchart of the reasoning stage of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] like Figure 1 As shown, a wafer defect detection method based on a dual twin network includes the following specific steps:

[0046] Step 10: Image acquisition.

[0047] The camera captures images of the wafer to be inspected and preprocesses them. The wafer images include images of both defect-free and defective wafers. Preprocessing includes filtering, denoising, and flat-field correction to improve image quality. Because the data preprocessing process is prior art and not the focus of this application, further detailed description is omitted.

[0048] Step 20. Generate defective images using a GAN: Use an adversarial network to generate pseudo-defective images on defect-free wafer images, and transform the original data through data augmentation (such as rotation, scaling, translation, flipping, cropping, color adjustment, noise addition, geometric transformation, color adjustment, random erasing and mixing techniques, etc.) to generate new twin network training samples.

[0049] To overcome the difficulty of collecting large image datasets, the powerful generative capabilities of a generative adversarial network (GAN) are leveraged to enrich the diversity of defect types. The adversarial network consists of two key components: a generator (G) for creating new samples and a discriminator (D) for assessing sample authenticity. Initially, the generator produces low-quality pseudo-images, allowing the discriminator to easily detect the difference between wafer images and real images. As training progresses, by the second iteration, the pseudo-images produced by the generator improve in quality and begin to confuse the discriminator. As the discriminator's performance continues to improve, its ability to distinguish between real and fake images also gradually improves, which in turn drives the generator to continuously improve and produce more realistic images. Through this continuous adversarial learning process, the generator gradually learns to mimic the data distribution of real images, thereby creating more realistic pseudo-defect images. As the learning task becomes relatively simple, the model's objective function converges rapidly, ultimately enabling the generator to produce pseudo-defects that are virtually indistinguishable from real particle defects. The final step is to integrate these pseudo-particle defects produced by the generator into defect-free wafer dies to construct a new, expanded pseudo-image dataset.

[0050] like Figure 2As shown in the figure, the generator (G) of a Generative Adversarial Network (GAN) is a four-layer recurrent neural network. By acquiring the distribution of real images and inputting a noise source (z) into the generator, it produces pseudo images that are similar to real images. The discriminator (D) is a three-layer neural network composed of a binary classifier, responsible for evaluating the authenticity of the pseudo images. As the discriminator's capabilities improve, it can distinguish between real and simulated images, driving improvements in the generator. The generator then generates pseudo images that are closer to real images for the discriminator to judge, until the generator can generate pseudo images that are very similar to real images.

[0051] The generator and discriminator are trained simultaneously by minimizing an adversarial loss function. The specific process is as follows:

[0052] Training the discriminator: Randomly sample a batch of real images x from the real defect image dataset; sample a batch of random noise z from the random noise distribution, and generate a batch of fake defect images G(z) through the generator; fix the parameters of the generator, calculate the gradient based on the discriminator's loss function, and update the discriminator's parameters so that the discriminator can better distinguish between real images and generated images;

[0053] Training the generator: Sample a batch of random noise z from the random noise distribution, fix the parameters of the discriminator, calculate the gradient according to the generator's loss function, and update the generator's parameters so that the image generated by the generator is closer to the real image, thereby deceiving the discriminator;

[0054] Iterative training: Repeat the above alternating training steps until the pseudo-defect images generated by the generator are realistic enough and the discriminator has difficulty distinguishing between real images and generated images. During the training process, as the discriminator's ability improves, the generator will be further improved, thereby continuously improving the quality of the generated images.

[0055] The adversarial learning training process is continued between the generator and the discriminator to achieve interactive learning and create a generative model (composed of two networks, the generator and the discriminator) that is similar to the real image distribution. The loss function of the generative model is as follows:

[0056] WGAN loss includes generator loss L G and the discriminator loss L D , the expression is as follows:

[0057]

[0058] Where D(x) is the score of the discriminator for the real image, and D(G(z)) is the score of the discriminator for the generated image; For the real data distribution p data The expected value of all samples x of (x); Denotes the noise distribution p from the prior z (z) is the expected value of all random variables z in the generative adversarial network. In the generative adversarial network, these random variables z are random noise vectors input to the generator, which are converted into fake samples by the generator; G(z) is the result of the generator G acting on the random noise vector z, that is, the generator generates a fake sample based on the input noise vector, and the goal is to make this fake sample as close to the real data sample as possible; p z (z) is the distribution of the random noise vector z input to the generator; z is the random noise vector input to the generator, and x is the distribution of the real data p data A sample drawn from (x).

[0059] The perceptual loss measures the difference between the generated image G(z) and the real image x in the feature space:

[0060]

[0061] Among them, L perceptual It is the overall expression of the perceptual loss function, which is used to quantify the difference in feature representation between the generated image G(z) and the real image x; It is a specific component of the perceptual loss, which represents the squared Euclidean distance to the feature map of the i-th layer, that is, the square of the L2 norm. Here, φ i refers to the i-th layer feature extractor of a pre-trained convolutional neural network, φ i (G(z)) is the feature extracted from the generated image, and φ i (x) is the feature extracted from the real image.

[0062] WGAN (Wasserstein GAN) loss can solve the training instability problem in traditional GAN ​​and is particularly suitable for generating complex images. Perceptual loss can make the network's generated images visually closer to real images. The final generator loss is the weighted sum of WGAN loss and perceptual loss:

[0063]

[0064] Among them, λ WGAN and λ perceptual is the weight coefficient, which is used to adjust the importance of the two parts of loss.

[0065] Step 30: Build a twin network: It includes two twin networks, which reconstruct and repair the discriminant features to finally obtain the predicted image.

[0066] like Figure 3As shown, the present application includes two twin networks. The first twin network uses a twin network structure to simultaneously process defect-free wafer image samples and defective wafer image samples to extract discriminative features, and uses the VGG16 (convolutional neural network) network as a feature extractor to extract features from different levels; the structure of the second twin network adopts an autoencoder, and inputs the output features from the first twin network for reconstruction and repair. By reconstructing and repairing the discriminant features, the predicted image is finally obtained.

[0067] Different numbers of neural network layers correspond to features at different levels of functionality. Shallow filters act as simple edge detectors or texture extraction units. Later layers represent more semantic features. Excessively deep network structures can compromise detection efficiency. To reduce the potential for overfitting during small-sample training, the VGG16 network is used as a feature extractor, extracting features from different layers, including edge and semantic information.

[0068] To balance computational efficiency and semantic level coverage, the output of the last layer of each block is selected during subsequent MTUF multi-scale fusion to facilitate upsampling and fusion. The VGG16 network pre-trained on ImageNet is used as an example of an embedded feature extractor. The VGG16 network contains five convolutional blocks, each containing 2, 2, 3, 3, and 3 convolutional layers, respectively. Therefore, the output of the feature extraction twin network consists of 13 feature maps. The two output feature sets of the entire network are F1 and F2.

[0069] Since the features of different layers have different receptive field sizes and scales, in order to achieve effective integration of multi-scale semantic features, a feature fusion module based on the FPN (Feature Pyramid Network) structure is proposed, called Multi-scale Top-down Unified Fusion (MTUF), which is used to generate a unified feature representation of the autoencoder input. Specifically, the last layer output of each convolution block is first extracted from the VGG16 backbone network, corresponding to feature maps of different spatial scales and semantic levels; in order to unify the channel dimension, each feature map is first compressed to the same number of channels through a 1×1 convolution layer. Then, a top-down fusion method similar to FPN is adopted, starting from the highest layer, and step-by-step upsampling is performed, and the feature maps of the adjacent shallow layers are added and fused layer by layer, so as to realize the complement of the shallow structure by the deep semantics. Finally, the shallowest fused feature map will be used as the global fusion result, and will be further integrated and output as a unified fusion feature set through a 3×3 convolution as F m1 or F m2Enter the autoencoder network. This module effectively integrates the multi-layer feature information of VGG16, combining shallow details with deep semantic expression, significantly improving the model's ability to perceive small defect areas.

[0070] This fused feature map fully combines the structural and semantic information under multi-scale receptive fields, which helps to improve the network's perception of fine-grained defect areas and the overall reconstruction accuracy.

[0071] The main structure of the defect-aware twin network is AE (autoencoder), which uses the output of the MTUF module: the fusion feature set F of the normal sample m1 and the fusion feature set F of the defect sample m2 In addition, 1×1 convolution is used instead of the traditional 3×3 convolution of AE to ensure that the size of feature information is not lost. Each convolution layer is followed by batch normalization and ReLUs activation layer. In order to further improve the reconstruction and repair effect, skip connection is used in the AE network by adding layers. The reconstructed defect feature set is F AE1 , the defect feature set after repair is F AE2 .

[0072] Step 40: Network training:

[0073] The defect-free wafer image samples and defective wafer image samples are input into the network for training at the same time. The network is trained by minimizing the loss function so that the network can learn the distribution of defect-free areas and distinguish between defective and non-defective areas. Specifically, the normal image and the defective image are input into the feature extraction twin network at the same time. The feature extraction twin network extracts a complete set of discriminative positive sample features and their corresponding abnormal features, and uses the autoencoder (AE) as the backbone convolutional neural network (DCNN) of the defect perception twin network to reconstruct and repair the fused feature set. In the inference stage, the wafer image samples to be inspected are used as the dual input of the feature extraction twin network, and the difference between the input and its reconstructed and repaired feature set is calculated to generate the final abnormality map.

[0074] During the training process, the validation set is used to evaluate the performance of the model. The training strategy is adjusted based on the performance feedback, including adjusting the learning rate, loss function weight or other hyperparameters to convert the feature set of defect-free wafer samples into As the true values ​​of the two losses, the loss functions used are as follows:

[0075]

[0076] h and w represent the height and width of the fused feature map, respectively. j and i are index variables used to traverse the entire feature map. j is used to traverse the height direction of the feature map, while i is used to traverse the width direction. λ is a weight parameter used to balance the ratio between reconstruction loss and restoration loss. It is a summation symbol, which means that the errors at all positions on the feature map are accumulated.

[0077] L reconstruction and L inpainting is the loss function in the reconstruction and repair process, L reconstruction It is used to evaluate the similarity between the reconstructed features and the original defect-free wafer image, so that the feature set F AE1 Close to normal sample F m1 , to obtain the reconstruction effect. L inpainting It is used to evaluate the similarity between the features after defect repair and the original defect-free wafer mold image, so that the repaired defect feature set F AE2 For the repair effect to be as close to F as possible m1 , F m1 is the feature set of the original normal wafer image sample, F AE1 is the feature set after normal wafer image sample reconstruction, F AE2 is the feature set of the defective wafer image sample after repair. L reconstruction and L inpainting A pixel error metric with mean squared error loss is selected to improve segmentation accuracy and computational speed.

[0078] Step 5: Reasoning.

[0079] like Figure 4 As shown in the figure, in the reasoning, the input of the defect perception twin network is the wafer image to be tested, and the two branches in the second twin network share weights. AE1 and F AE2 The predicted image is first obtained by the input F of the second twin network. m1 and output F AE1 The difference between them is generated and then adjusted to the input image size. The final segmentation result is obtained by binarization. The proposed method directly outputs the predicted heat map.

[0080] As can be understood, the deep learning method of the present invention comprises two twin networks and two inputs: a standard, defect-free wafer image input and a defective wafer image input. The first feature extraction twin network extracts features from both defect-free and defective wafer images to improve feature extraction accuracy. The second defect perception twin network captures normal and abnormal information to generate anomaly maps with defect annotations, enabling wafer defect detection.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A wafer defect detection method based on a dual twin network, characterized in that: The specific steps include: S10. Image acquisition: Capturing images of the wafer to be inspected using a camera and preprocessing the captured images. The wafer images include images of both defect-free wafers and images of defective wafers. S20. Generate defective images using an adversarial network: Use an adversarial network to generate pseudo-defective images on defect-free wafer images. Transform the original data through data augmentation to generate new Siamese network training samples. S30. Constructing a twin network: This includes two twin networks that reconstruct and repair the discriminant features to ultimately obtain a predicted image. S40. Network Training: Use defect-free and defective images as datasets for the twin network training. The network is trained by minimizing the loss function, enabling it to learn the distribution of defect-free areas. The trained model is then used to detect wafer defects and distinguish between defective and non-defective areas. S50. Reasoning: Input the image to be tested into the twin network, and calculate the input feature set F of a sub-network of the second twin network. m1 And the output feature set F AE1 The difference between and generates the predicted image.

2. The wafer defect detection method based on a dual twin network according to claim 1, characterized in that: The adversarial network consists of two key components: a generator for creating new samples and a discriminator for evaluating the authenticity of samples. After the wafer image undergoes adversarial learning between the generator and the discriminator, a pseudo-defect image dataset is generated.

3. The wafer defect detection method based on a dual twin network according to claim 2, characterized in that: The generator of the adversarial network is a four-layer recurrent neural network. By obtaining the distribution of real images and inputting the noise source into the generator, it produces pseudo images similar to real images. The discriminator is a three-layer neural network of binary classifiers, which is responsible for evaluating the authenticity of pseudo images and distinguishing real images from simulated images.

4. The wafer defect detection method based on a dual twin network according to claim 3, characterized in that: The generator and discriminator are trained simultaneously by minimizing an adversarial loss function. The specific process is as follows: Training the discriminator: Randomly sample a batch of real images x from the real defect image dataset; sample a batch of random noise z from the random noise distribution, and generate a batch of fake defect images G(z) through the generator; fix the parameters of the generator, calculate the gradient based on the discriminator's loss function, and update the discriminator's parameters so that the discriminator can better distinguish between real images and generated images; Training the generator: Sample a batch of random noise z from the random noise distribution, fix the parameters of the discriminator, calculate the gradient according to the generator's loss function, and update the generator's parameters so that the image generated by the generator is closer to the real image, thereby deceiving the discriminator; Iterative training: Repeat the above alternating training steps until the pseudo-defect images generated by the generator are realistic enough and the discriminator has difficulty distinguishing between real images and generated images.

5. The wafer defect detection method based on a dual twin network according to claim 4, characterized in that: The adversarial learning training process between the generator and discriminator of the adversarial network achieves interactive learning and creates a generative model that is similar to the real image distribution. The loss function of the generative model is as follows: WGAN loss includes generator loss L G and the discriminator loss L D , the expression is as follows: Where D(x) is the score of the discriminator for the real image, and D(G(z)) is the score of the discriminator for the generated image; For the real data distribution p data The expected value of all samples x of (x); Denotes the noise distribution p from the prior z The expected value of all random variables z in (z). In the generative adversarial network, these random variables z are random noise vectors input to the generator, which are converted into fake samples by the generator; G(z) is the result of the generator G acting on the random noise vector z, that is, the generator generates a fake sample based on the input noise vector, with the goal of making the fake sample close to the real data sample; p z (z) is the distribution of the random noise vector z input to the generator; z is the random noise vector input to the generator, and x is the distribution of the real data p data A sample drawn from (x); The perceptual loss measures the difference between the generated image G(z) and the real image x in the feature space: Among them, L perceptual It is the overall expression of the perceptual loss function, which is used to quantify the difference in feature representation between the generated image G(z) and the real image x; It is a specific component of the perceptual loss, which represents the squared Euclidean distance to the feature map of the i-th layer, that is, the square of the L2 norm, φ i refers to the i-th layer feature extractor of a pre-trained convolutional neural network, φ i (G(z)) is the feature extracted from the generated image, and φ i (x) is the feature extracted from the real image; Perceptual loss can make the network's generated images visually closer to real images. The final generator loss is the weighted sum of WGAN loss and perceptual loss. Among them, λ WGAN and λ perceptual is the weight coefficient, which is used to adjust the importance of the two parts of loss.

6. The wafer defect detection method based on a dual twin network according to claim 1, characterized in that: In step S30, the first twin network uses the twin network structure to simultaneously process defect-free wafer image samples and defective wafer image samples to extract discriminative features, and uses the VGG16 network as a feature extractor to extract features from different levels; the structure of the second twin network adopts an autoencoder, and inputs the output features from the first twin network for reconstruction and repair. By reconstructing and repairing the discriminant features, the predicted image is finally obtained.

7. The wafer defect detection method based on a dual twin network according to claim 1, characterized in that: In step S40, the specific steps include: Step S41: input the normal image and the defect image into the twin network for feature extraction simultaneously; Step S42: The twin network extracts a complete set of discriminative positive sample features and their corresponding abnormal features, and uses the autoencoder as the backbone convolutional neural network of the defect-aware twin network to reconstruct and repair the fused feature set; Step S43: In the inference stage, the sample to be inspected is used as the dual input of the feature extraction twin network, and the difference between the input and its reconstructed and repaired feature set is calculated to generate the final anomaly map.

8. The wafer defect detection method based on a dual twin network according to claim 7, characterized in that: During the training process, the validation set is used to evaluate the performance of the model, and the training strategy is adjusted based on the performance feedback. The loss function used is as follows: Where h and w represent the height and width of the fused feature map, respectively. j and i are index variables used to traverse the entire feature map. j is used to traverse the height direction of the feature map, while i is used to traverse the width direction. λ is a weight parameter used to balance the ratio between reconstruction loss and restoration loss. It is a summation symbol, which means that the errors at all positions on the feature map are accumulated; L reconstruction and L inpainting is the loss function in the reconstruction and repair process, L reconstruction It is used to evaluate the similarity between the reconstructed features and the original defect-free wafer image, so that the feature set F AE1 Close to normal sample F m1 , to obtain the reconstruction effect; L inpainting It is used to evaluate the similarity between the features after defect repair and the original defect-free wafer mold image, so that the repaired defect feature set F AE2 For the repair effect to be as close to F as possible m1 ; F m1 is the feature set of the original normal wafer image sample, F AE1 is the feature set after normal wafer image sample reconstruction, F AE2 is the feature set of the defective wafer image sample after repair, L reconstruction and L inpainting A pixel error metric with mean squared error loss is selected to improve segmentation accuracy and computational speed.

Images