Method and system for detecting wafer defects through semi-supervised transformation

Abstract

The invention provides a method and a system for detecting wafer defects through semi-supervised transformation, based on multi-disturbance consistency regularization, and the method comprises the following steps: processing a wafer image to obtain labeled wafer data and unlabeled wafer data; constructing a change detection network, and inputting labeled wafer data into the change detection network for supervised training; performing strong and weak data enhancement on the unlabeled wafer data to obtain intermediate features of the unlabeled wafer data, and performing image strong and weak change consistency learning; performing various disturbances on the intermediate features of the unlabeled wafer data to generate a disturbed feature map, and calculating a feature disturbance consistency loss function; calculating a total loss function of the trained model to obtain a final model; and inputting unlabeled wafer data into the final model to obtain a wafer defect detection result. According to the method, semi-supervised learning is introduced, so that the manual annotation cost is greatly reduced; and through multi-disturbance consistency regularization, the accuracy and robustness of wafer change detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of wafer defect detection, and in particular to a method and system for semi-supervised transformation detection of wafer defects based on multi-perturbation consistency regularization. Background Art

[0002] Template-based wafer defect detection mainly includes image processing-based techniques and deep learning-based methods.

[0003] Image processing-based techniques mainly obtain the position of defects by processing interference signals such as illumination and noise of images and then comparing the template and defect pictures; Deep learning-based methods output the precise position of defects by extracting high-dimensional features of the template and defect pictures and comparing them.

[0004] Image processing-based methods are difficult to completely remove the influence of interference such as illumination and noise, resulting in a large number of false detections and low practicability; deep learning-based methods require a large number of defect annotations, resulting in high costs. Summary of the Invention

[0005] The object of the present invention is achieved by the following technical solutions.

[0006] In view of the above defects, the present invention proposes a method for semi-supervised transformation detection of wafer defects based on multi-perturbation consistency regularization, which greatly saves the manual annotation cost by annotating a small amount of data and semi-supervised training of unannotated data, and improves the algorithm performance.

[0007] Specifically, according to the first aspect of the present invention, there is provided a method for semi-supervised transformation detection of wafer defects based on multi-perturbation consistency regularization, including: Processing the wafer image to obtain labeled wafer data and unlabeled wafer data; Constructing a change detection network, inputting the labeled wafer data into the change detection network for supervised training to obtain a trained model; Performing strong and weak data augmentation on the unlabeled wafer data and performing image strong and weak change consistency learning; Performing multiple perturbations on the intermediate features of the unlabeled wafer data to generate perturbed feature maps, and calculating the feature perturbation consistency loss function; Calculating the total loss function of the trained model to obtain the final model; Inputting the unlabeled wafer data into the final model to obtain the wafer defect detection result.

[0008] According to the second aspect of the present invention, there is also provided a system for semi-supervised transformation detection of wafer defects based on multi-perturbation consistency regularization, including: A labeling module for processing a wafer image to obtain labeled wafer data and unlabeled wafer data; A training module for constructing a change detection network, using the labeled wafer data to input into the change detection network for supervised training to obtain a trained model; A data augmentation module for performing strong and weak data augmentation on the unlabeled wafer data and performing image strong and weak change consistency learning; A perturbation consistency module for performing various perturbations on the intermediate features of the unlabeled wafer data to generate a perturbed feature map and calculating a feature perturbation consistency loss function; A final model module for calculating the total loss function of the trained model to obtain a final model; A result acquisition module for inputting the unlabeled wafer data into the final model to obtain a wafer defect detection result.

[0009] The advantages of the present invention are as follows: The present invention introduces semi-supervised learning technology, significantly reducing the manual labeling cost; through multiple perturbation consistency regularization, the present invention can more effectively utilize unlabeled data, improving the accuracy and robustness of wafer change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings: Figure 1 Shows a schematic diagram of a supervised training process according to an embodiment of the present invention.

[0011] Figure 2 Shows a schematic diagram of a strong and weak change consistency loss calculation process according to an embodiment of the present invention.

[0012] Figure 3 Shows a schematic diagram of a feature perturbation consistency loss calculation process according to an embodiment of the present invention.

[0013] Figure 4 Shows a system composition diagram for semi-supervised transformation detection of wafer defects according to an embodiment of the present invention.

[0014] Figure 5 Shows a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0015] Figure 6 Shows a schematic diagram of a storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0017] The present invention performs strong and weak data augmentation on unlabeled wafer data and conducts image strong and weak change consistency learning; performs various perturbations on the intermediate features of unlabeled wafer data, generates perturbed feature maps, and calculates a feature perturbation consistency loss function. Specifically, the present invention includes the following steps: Process the wafer images to obtain labeled wafer data and unlabeled wafer data; Construct a change detection network, input the labeled wafer data into the change detection network for supervised training to obtain a trained model; Perform strong and weak data augmentation on unlabeled wafer data and conduct image strong and weak change consistency learning; Perform various perturbations on the intermediate features of unlabeled wafer data, generate perturbed feature maps, and calculate a feature perturbation consistency loss function; Calculate the total loss function of the trained model to obtain a final model; Input the unlabeled wafer data into the final model to obtain a wafer defect detection result. Embodiment

[0018] Specifically, for example, an embodiment of the present invention is as follows:

[0019] S1. Scan the wafer images into grayscale pictures, align multiple wafer images, and take the median image I ref as a template image. Then slice the defect images into a size of 256*256, and label the defect parts of some of the pictures using annotation software.

[0020] S2. Divide the data into labeled data pairs ( , ) and unlabeled data pairs ( , ), where A represents the defect picture and B represents the corresponding slice of the template picture.

[0021] S3. Construct a change detection network N, which consists of a feature extractor (encoder) and a decoder. The encoder can use the feature extraction layer of resnet50, and the decoder consists of upsampling operations and multiple convolutions.

[0022] S4. Input ( , ) into N to obtain the final feature map , and calculate the cross-entropy with the corresponding annotation as the loss function for supervised training. After completing the supervised training, obtain the pre-trained model M, as Figure 1 shown.

[0023] .

[0024] S5. Apply weak augmentation to ( , ) to obtain an image pair ( , ), where the weak augmentation includes random flipping, random scaling, and random cropping.

[0025] S6. Apply two different strong augmentations to ( , ) to obtain image pairs ( , ), ( , ), where the strong augmentation includes cutmix and Gaussian blur.

[0026] S7. Input ( , ) into the pre-trained model M with fixed weights, and use the output D w of the feature extractor (encoder) as the intermediate feature, and the finally output feature is P w . Take the threshold t for P w as the pseudo-label: , where a value of 1 indicates a defective part, and a value of 0 indicates a non-defective part.

[0027] S8. ( , ), ( , is input into M to obtain the output feature map P s1 , P s2 .

[0028] S9. Calculate the consistency loss function of the image strength change, as Figure 2 shown: , where represents the cross-entropy loss function.

[0029] This method aims to enhance the robustness of the model to various data transformations. By applying strong-to-weak consistency constraints at the image level, the model can produce consistent feature maps even when the input images have undergone different enhancement operations.

[0030] S10. Perform feature perturbation on D w , including: random noise, random dropout, feature block dropout, etc. to obtain D f . Input D f into the decoder of M to obtain the final output feature P f .

[0031] S11. Calculate the consistency loss function of the feature perturbation, as Figure 3 shown: , This strategy focuses on improving the generalization ability and anti-interference ability of the model. It introduces various different perturbations at the feature level and enforces the consistency of these perturbed features.

[0032] S12. The total loss function of the model is: , Through the consistency loss function of the image strength change and the consistency loss function of the feature perturbation These two consistency loss functions can improve the robustness and generalization of the model.

[0033] S13. After obtaining the final model F, input the unlabeled image pair ( , ) into F to obtain the output feature P, and then binarize it through the threshold T , to obtain the final result . The part with a value of 1 represents the defective part in the wafer image.

[0034] such as Figure 4As shown in the figure, a semi-supervised transformation detection system for wafer defects, based on multi-perturbation consistency regularization, includes: An annotation module 401, configured to process wafer images to obtain labeled wafer data and unlabeled wafer data; A training module 402, configured to construct a change detection network, use the labeled wafer data to input the change detection network for supervised training, and obtain a trained model; A data augmentation module 403, configured to perform strong and weak data augmentation on the unlabeled wafer data, and perform image strong and weak change consistency learning; A perturbation consistency module 404, configured to perform various perturbations on the intermediate features of the unlabeled wafer data, generate perturbed feature maps, and calculate a feature perturbation consistency loss function; A final model module 405, configured to calculate the total loss function of the trained model to obtain a final model; A result acquisition module 406, configured to input the unlabeled wafer data into the final model to obtain a wafer defect detection result.

[0035] The semi-supervised transformation detection system for wafer defects provided in the above embodiments of the present invention and the semi-supervised transformation detection method for wafer defects provided in the embodiments of the present invention are based on the same inventive concept and have the same beneficial effects as the methods adopted, run, or implemented by the stored application programs.

[0036] The embodiments of the present invention also provide an electronic device corresponding to the semi-supervised transformation detection method for wafer defects provided in the foregoing embodiments to execute the semi-supervised transformation detection method for wafer defects. The embodiments of the present invention are not limited.

[0037] Please refer to Figure 5 , which shows a schematic diagram of an electronic device provided in some embodiments of the present invention. As Figure 5 shown, the electronic device 20 includes: a processor 200, a memory 201, a bus 202, and a communication interface 203. The processor 200, the communication interface 203, and the memory 201 are connected through the bus 202. A computer program that can run on the processor 200 is stored in the memory 201. When the processor 200 runs the computer program, it executes the semi-supervised transformation detection method for wafer defects provided in any of the foregoing embodiments of the present invention.

[0038] Among them, the memory 201 may include high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk memory. The communication connection between this system network element and at least one other network element is realized through at least one communication interface 203 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0039] The bus 202 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Among them, the memory 201 is used to store programs. After receiving an execution instruction, the processor 200 executes the program. Any implementation manner of the semi-supervised transformation detection wafer defect method disclosed in any implementation manner of the foregoing embodiments of the present invention can be applied to the processor 200 or implemented by the processor 200.

[0040] The processor 200 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 200 or the instructions in the form of software. The above-mentioned processor 200 can be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and combines its hardware to complete the steps of the above method.

[0041] The electronic device provided by the embodiment of the present invention and the semi-supervised transformation detection wafer defect method provided by the embodiment of the present invention are based on the same inventive concept and have the same beneficial effects as the methods adopted, run, or implemented by them.

[0042] An embodiment of the present invention also provides a computer-readable storage medium corresponding to the method for semi-supervised transform detection of wafer defects provided in the foregoing embodiment. Please refer to Figure 6 , and the computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the method for semi-supervised transform detection of wafer defects provided in any of the foregoing embodiments.

[0043] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other optical and magnetic storage media, which will not be elaborated here one by one.

[0044] The computer-readable storage medium provided in the above embodiment of the present invention and the method for semi-supervised transform detection of wafer defects provided in the embodiment of the present invention are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0045] It should be noted that: The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings provided herein. The structure required to construct such a system will be apparent from the above description. In addition, the present invention is not directed to any particular programming language. It should be understood that the content of the present invention described herein can be implemented using various programming languages, and the description of a particular language above is for the purpose of disclosing the best mode of the present invention.

[0046] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that the embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and technologies have not been shown in detail so as not to obscure the understanding of this specification.

[0047] Similarly, it should be understood that, for the purpose of streamlining the present invention and facilitating the understanding of one or more of the various inventive aspects, in the foregoing description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, the inventive aspects lie in less than all of the features of the single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate embodiment of the present invention.

[0048] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and disposed in one or more devices different from those of the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) can be replaced by an alternative feature that provides the same, equivalent, or similar purpose.

[0049] In addition, those skilled in the art will be able to understand that, although some of the embodiments described herein include certain features included in other embodiments rather than other features, the combination of the features of different embodiments is meant to be within the scope of the present invention and forms different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination.

[0050] Each component embodiment of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the virtual machine creation system according to the embodiments of the present invention. The present invention can also be implemented as a device or system program (for example, a computer program and a computer program product) for executing some or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0051] It should be noted that the above embodiments illustrate the present invention rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In the unit claims enumerating several systems, several of these systems can be embodied by the same hardware item. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names.

[0052] As described above, the above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A semi-supervised transformation detection method for wafer defects, based on multi-perturbation consistency regularization, characterized in that, Including: Process the wafer image to obtain labeled wafer data and unlabeled wafer data; Construct a change detection network, input the labeled wafer data into the change detection network for supervised training, and obtain a trained model; Perform strong and weak data augmentation on the unlabeled wafer data to obtain intermediate features of the unlabeled wafer data, and perform image strong-weak change consistency learning; Perform various perturbations on the intermediate features of the unlabeled wafer data to generate perturbed feature maps, and calculate the feature perturbation consistency loss function; Calculate the total loss function of the trained model to obtain the final model; Input the unlabeled wafer data into the final model to obtain the wafer defect detection result.

2. The method for semi-supervised transformation detection of wafer defects according to claim 1, wherein The processing of the wafer image to obtain labeled wafer data and unlabeled wafer data includes: S1. Scan the wafer image into a grayscale picture, align multiple wafer images, and take the median image I ref as the template image; then slice the defect image into a size of 256*256, and label the defect parts of some of the pictures using annotation software; S2. Divide the data into labeled data pairs ( , ) and unlabeled data pairs ( , ), where A represents a defective picture and B represents the corresponding slice of the template picture.

3. The method for semi-supervised transformation detection of wafer defects according to claim 2, wherein The construction of the change detection network, inputting the labeled wafer data into the change detection network for supervised training to obtain a trained model, includes: S3. Construct a change detection network N, which consists of a feature extractor and a decoder. The feature extractor uses the feature extraction layer of resnet50, and the decoder consists of upsampling operations and multiple convolutions; S4. Input ( , ) into N to obtain the feature map , and calculate the cross entropy with the corresponding annotation , which is used as the loss function for supervised training. After completing the supervised training, the pre-trained model M is obtained.

4. The method for semi-supervised transformation detection of wafer defects according to claim 3, wherein The performing of strong and weak data augmentation on the unlabeled wafer data to obtain intermediate features of the unlabeled wafer data and performing image strong-weak change consistency learning includes: S5. Perform weak augmentation on ( , ) to obtain an image pair ( , ), where the weak augmentation includes random flipping, random scaling, and / or random cropping; S6. Perform two different strong augmentations on ( , ) to obtain image pairs ( , ), ( , ), where the strong augmentation includes cutmix and / or Gaussian blur; S7. Input ( , ) into the pre-trained model M with fixed weights, and take the output D w of the feature extractor as the intermediate feature. The finally output feature is P w . Take the threshold t for P w as the pseudo label: ， Among them, a value of 1 indicates a defective part, and a value of 0 indicates a non-defective part; S8, ( , ), ( , ) are input into M to obtain the output feature map P s1 and P s2 ; S9. Calculate the image strong-weak change consistency loss function: ， Among them represents the cross-entropy loss function.

5. The method for semi-supervised transformation detection of wafer defects according to claim 4, wherein The performing of various perturbations on the intermediate features of the unlabeled wafer data to generate perturbed feature maps and calculating the feature perturbation consistency loss function includes: S10. Perturb the features of D w including: adding random noise, randomly dropping, and / or dropping feature blocks to obtain D f , and input D f into the decoder of M to obtain the final output feature P f ; S11. Calculate the feature perturbation consistency loss function: 。 6. The method for semi-supervised transformation detection of wafer defects according to claim 5, wherein The calculating of the total loss function of the trained model to obtain the final model includes: S12. The total loss function of the model is: 。 7. The method for semi-supervised transformation detection of wafer defects according to claim 6, wherein The inputting of the unlabeled wafer data into the final model to obtain the wafer defect detection result includes: S13. After obtaining the final model F, input the unlabeled image pair ( , ) into F to obtain the output feature P, and then perform binarization through the threshold T: ， Obtain the final result , where the part with a value of 1 represents the defective part in the wafer image.

8. A semi-supervised transformation detection system for wafer defects, based on multi-perturbation consistency regularization, characterized in that Including: A labeling module for processing the wafer image to obtain labeled wafer data and unlabeled wafer data; A training module for constructing a change detection network, inputting the labeled wafer data into the change detection network for supervised training to obtain a trained model; A data augmentation module for performing strong and weak data augmentation on the unlabeled wafer data and performing image strong-weak change consistency learning; The perturbation consistency module is used to perform various perturbations on the intermediate features of the unlabeled wafer data, generate the perturbed feature maps, and calculate the feature perturbation consistency loss function; The final model module is used to calculate the total loss function of the trained model to obtain the final model; The result acquisition module is used to input the unlabeled wafer data into the final model to obtain the wafer defect detection result.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor runs the computer program to implement the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method according to any one of claims 1-7.

Images