Wafer defect image generation method, training method of generation model and storage medium
By using a diffusion model training method, noise is gradually added and removed using defect category information, generating high-quality wafer defect images. This solves the problem of insufficient samples in the model and improves the detection capability.
Patent Information
- Application Number
- CN202511536355.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-24
- Publication Date
- 2026-03-03
AI Technical Summary
In existing technologies, wafer defect detection models lack sufficient defect image samples, resulting in insufficient model generalization ability and making them unsuitable for effective application in actual production.
A diffusion model is adopted to acquire wafer defect sample images and their defect category information from multiple different defect categories. The defect category information is used as a guide to perform progressive noise addition and denoising processing to train the generative model to generate high-quality wafer defect images.
High-quality wafer defect images of various defect categories were generated, improving the model's generalization ability, meeting the confidentiality requirements of the semiconductor industry, and increasing detection accuracy.
Smart Images
Figure CN121600344A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, specifically to a method for generating wafer defect images and a training method and storage medium for the generation model. Background Technology
[0002] Semiconductor devices such as chips are manufactured using wafers as a substrate. During the manufacturing process of semiconductor devices, some products may have defects on their surface, such as cracks, bumps (a type of raised structure on the chip surface), and scratches. It is necessary to detect wafer surface defects to ensure product quality.
[0003] Currently, deep learning technology has been applied to wafer defect detection. The neural network model used relies on learning from a large number of defect image samples. By using rich defect image samples from various products and processes, the model can be taught to identify more defect categories, have stronger generalization ability, and higher detection accuracy.
[0004] However, the number of wafer defect images for some defect categories is limited. Furthermore, due to the confidentiality requirements of semiconductor industry manufacturers, typical samples of on-site wafer defect images cannot be directly reused at the data level, thus failing to effectively enhance the practical application performance of the model. Summary of the Invention
[0005] This invention primarily addresses the technical problem of insufficient wafer defect images required for training models.
[0006] According to a first aspect, one embodiment provides a training method for a wafer defect image generation model, comprising: Acquire wafer defect sample images and their defect category information for multiple different defect categories; Using a diffusion model and defect category information as guiding information, each wafer defect sample image is progressively denoised and then progressively denoised to recover the wafer defect sample image and obtain the sample reconstruction image. The diffusion model is trained using the reconstructed sample image and the wafer defect sample image until a preset training stopping condition is reached, thereby obtaining a wafer defect image generation model; wherein, the training objective includes making the reconstructed sample image consistent with the corresponding wafer defect sample image.
[0007] In some embodiments, the defect category information is defect category prompt text, which is encoded into a data format of the diffusion model's guidance information for input into the diffusion model.
[0008] In some embodiments, the diffusion model is a Diffusion Transformers model.
[0009] In some embodiments, the defect category information is used as an embedding condition for the Diffusion Transformers model; at each iteration, the wafer defect sample image, the noise of the current iteration, and the time step t are fused and used as input representations of the Diffusion Transformers model.
[0010] In some embodiments, the Diffusion Transformers model includes a block layer, multiple Transformer blocks, and a reshaping layer; The block layer is used to divide the noise tensor into multiple noise tensor blocks and add position encoding to each noise tensor block, so that the multiple noise tensor blocks are fused with the wafer defect sample image and the time step t; The multiple Transformer blocks are used to extract features from the input representation and fuse them with the defect category information to obtain feature information; The reshaping layer is used to recover an image of the same size as the wafer defect sample image based on the feature information, so as to obtain the sample reconstruction image.
[0011] In some embodiments, the defect category information is defect category hint text, which is encoded into a data format of the embedding conditions of the Diffusion Transformers model for input into the Diffusion Transformers model.
[0012] In some embodiments, the defect category prompt text is encoded using a CLIP model text encoder.
[0013] In some embodiments, the defect category prompt text includes a defect category name and / or a descriptive text for the defect category.
[0014] According to a second aspect, one embodiment provides a method for generating a wafer defect image, comprising: Obtain defect category information for the noise to be reconstructed and the target defect category; The wafer defect image generation model obtained by the training method described in the first aspect above uses the defect category information of the target defect category as guiding information to perform stepwise denoising processing on the noise to be reconstructed to generate a wafer defect image.
[0015] In some embodiments, the diffusion model is a Diffusion Transformers model, and the defect category information of the target defect category is used as the embedding condition of the Diffusion Transformers model. In each iteration, the noise and time step t of the current iteration are fused and used as the input representation of the Diffusion Transformers model.
[0016] In some embodiments, the defect category information of the target defect category is the defect category prompt text of the target defect category, which includes the name and / or description text of the target defect category.
[0017] According to a third aspect, one embodiment provides a computer-readable storage medium storing a computer program that can be executed by a processor to implement the training method for the wafer defect image generation model as described in the first aspect or the wafer defect image generation method as described in the second aspect.
[0018] According to the above embodiments of the wafer defect image generation model training method, wafer defect image generation method, and computer-readable storage medium, multiple wafer defect sample images of different defect categories and their defect category information are acquired as training samples. A diffusion model is used, employing defect category information as guiding information, to progressively add noise to each wafer defect sample image and then progressively denoise it to recover the wafer defect sample image, obtaining a sample reconstructed image. The diffusion model is trained using the sample reconstructed image and the wafer defect sample image until a preset training stopping condition is reached, thus obtaining a wafer defect image generation model. The training objective includes making the sample reconstructed image consistent with the corresponding wafer defect sample image, enabling the obtained wafer defect image generation model to generate wafer defect images of various different defect categories. Using a diffusion model as the wafer defect image generation model allows the introduction of defect category information as guiding information, learning to generate wafer defect images through a noise-adding and denoising approach, enabling the final model to generate high-quality wafer defect images that conform to the target defect category. Attached Figure Description
[0019] Figure 1 A flowchart illustrating a training method for a wafer defect image generation model according to one embodiment; Figure 2 This is a schematic diagram of defect category prompt text for one embodiment; Figure 3This is a schematic diagram of the Diffusion Transformers model. Figure 4 This is a schematic diagram showing the input of a text encoder using the CLIP model to the diffusion model after encoding the defect warning text. Figure 5 This is a schematic diagram of the CLIP model; Figure 6 A flowchart illustrating a method for generating wafer defect images according to one embodiment; Figure 7 This is a schematic diagram of a wafer defect image generated using the wafer defect image generation method of this application. Detailed Implementation
[0020] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.
[0021] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.
[0022] The serial numbers assigned to components or physical quantities in this document, such as "first," "second," etc., are used only to distinguish the described objects and have no sequential or technical meaning. They should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. "Multiple" means two or more. Unless otherwise specified, "connection" or "linkage" in this application includes both direct and indirect connections (linkages).
[0023] To expand the amount of wafer defect image data used for model training, existing data augmentation methods mainly rely on simple methods such as flipping, translating, and changing the color of the original images. However, these methods lack the ability to expand the diversity of the original images (such as the shape, size, location, and background changes of defects). This results in a low degree of matching with the diverse changes in wafer defects during the production process, leading to poor detection performance of the trained models on wafer defect images obtained in actual production.
[0024] To address the aforementioned issues, this application provides a training method for a wafer defect image generation model. It employs diffusion modeling techniques from deep learning, using collected wafer defect sample images for training. The input consists of wafer defect sample images and their defect category information, while the output is a reconstructed sample image. Through training, the diffusion model learns to generate wafer defect images that match the input defect category information. Upon convergence, a wafer defect image generation model capable of generating various types of wafer defects is obtained. The trained diffusion model can automatically generate high-quality wafer defect images of the input defect category, which are then used as training samples to expand the wafer defect image data used for training.
[0025] The diffusion model is a generative model derived from the diffusion process in non-equilibrium thermodynamics. It simulates the process of gradually adding noise to a dataset (the diffusion process), and then gradually denoising through a reverse process (the reverse diffusion process) to recover the original data from the noise. In the diffusion process, the model gradually introduces noise into the image until it becomes almost pure noise. The model starts from the original state (i.e., the original image) x0, adding noise to the original image at each time step, and reaching state x after T time steps of iteration. T In the reverse diffusion process, the diffusion model recovers the original data from the noise, and the model recovers the original data from the noise state x. T Initially, after T time steps of gradual denoising, the original image x0 is restored.
[0026] Diffusion models can be given guidance information to help them better understand the generation target, control the content they generate, and guide them to generate images of specific types or under specific conditions.
[0027] Please refer to Figure 1 The training method for the wafer defect image generation model in one embodiment of this application includes steps 110-130, which are described in detail below.
[0028] Step 110: Obtain wafer defect sample images and their defect category information for multiple different defect categories.
[0029] Wafer defect sample images are pre-collected images of wafer surface defects captured by an imaging sensor. Defect category information may include category labels indicating the defect category, etc.
[0030] Typically, a large number of wafer defect sample images are collected for model training. In one embodiment, wafer defect sample images of 25 defect categories are used for model training, with 500 wafer defect sample images for each category, for a total of 12,500 wafer defect sample images.
[0031] Step 120: Using a diffusion model and defect category information as guiding information, noise is gradually added to each wafer defect sample image and then gradually removed to recover the wafer defect sample image and obtain the sample reconstruction image.
[0032] The diffusion model used can be U-Net, etc. Defect category information and wafer defect sample images can be encoded before being input into the diffusion model. This encoding can be done according to the input format requirements of the diffusion model, such as encoding into a vector representation.
[0033] The total number of time steps T for the diffusion model iteration, i.e. the number of iterations each wafer defect sample image needs to participate in, is a preset amount that can be set according to actual needs, and T is an integer not less than 2.
[0034] During the noise addition process, randomly generated noise is added to the wafer defect sample image; Gaussian noise, for example, can be used. In each iteration at time step, the current iteration's time step t can also be incorporated, representing the current noise level. Time step t can also be encoded and fused with the wafer defect sample image.
[0035] Step 130: Train the diffusion model using the reconstructed image and the wafer defect sample image until the preset training stopping condition is reached, and obtain the wafer defect image generation model.
[0036] The training objective includes ensuring that the reconstructed image of the sample matches the corresponding wafer defect sample image, enabling the diffusion model to learn to reconstruct the wafer defect sample image. Loss functions such as mean squared error can be used to minimize the difference between the reconstructed image of the dissimilar sample and the corresponding wafer defect sample image, thus achieving the training objective.
[0037] The preset training stopping condition can be that the number of training iterations reaches a preset number (e.g., the model is considered to have converged after 700 iterations) or the loss value calculated according to the loss function is less than a preset loss value threshold (e.g., the loss of the error loss function is gradually reduced until it is below 0.09).
[0038] The wafer defect image generation model obtained through the training method of this application can generate wafer defect images of various defect categories. Using a diffusion model as the wafer defect image generation model, and incorporating defect category information as guiding information, the model learns to generate wafer defect images through progressive noise addition and denoising, which better preserves the detailed information in the images and generates high-quality wafer defect images.
[0039] In some embodiments, the defect category information is defect category hint text, which is encoded into a data format of guidance information for the diffusion model and input into the diffusion model. The encoding method can refer to existing technologies.
[0040] The defect category prompt text may include the defect category name and / or the defect category description text. The defect category description text describes the characteristics, severity, etc. of the defect category and may include one or more statements.
[0041] In one embodiment, wafer defect sample images of 25 defect categories are used for model training, and the defect category prompt text for each defect category is as follows: Figure 2 As shown, the defect category name is located at the beginning of the defect category prompt text, such as "Big Bump" or "Bridging". The description text of the defect category is separated from the defect category name by a comma and is located after the defect category name. For example, "one or more Bumps are particularly large" is the description text of the defect category "Big Bump".
[0042] By using defect category prompt text as defect category information and encoding the defect category prompt text into the data format of the diffusion model's guidance information, the diffusion model can be input in text form. Users can directly input the defect category prompt text to generate a matching wafer defect image, without being limited to a fixed input format, making it more convenient and flexible to use.
[0043] In some embodiments, the diffusion model is the Diffusion Transformers model.
[0044] Figure 3 This illustrates the basic structure of the Diffusion Transformers model. Please refer to [link / reference]. Figure 3Defect category information can be used as an embedding condition for the Diffusion Transformers model. In each iteration, the wafer defect sample image, the noise of the current iteration, and the time step t are fused and used as input tokens for the Diffusion Transformers model.
[0045] If the defect category information is defect category hint text, then the defect category hint text is encoded into the data format (usually a feature vector) of the embedding conditions of the DiffusionTransformers model to be input into the DiffusionTransformers model.
[0046] like Figure 3 As shown, the Diffusion Transformers model includes a patchify layer, multiple Transformer blocks (DiT Blocks), and a reshaping layer (Linear and Reshape).
[0047] The block layer is used to divide the noise tensor into multiple noise tensor blocks and add position encoding to each noise tensor block, so that the multiple noise tensor blocks are fused with the wafer defect sample image and time step t.
[0048] Specifically, noise can be encoded into a tensor of a predetermined size (e.g., 32×32×5), then divided into blocks through a block layer, and a positional encoding is added to each noise tensor block; the wafer defect sample image and time step t are also encoded (embedded), and then fused with multiple noise tensor blocks with added positional encoding to form the input representation of the Diffusion Transformers model.
[0049] Multiple Transformer blocks are used to extract features from the input representation and fuse them with defect category information to obtain feature information. This feature information can be a feature vector or a feature matrix.
[0050] Figure 3The diagram illustrates that the Diffusion Transformers model comprises N Transformer blocks, where N is an integer not less than 2. Each Transformer block is constructed based on the Transformer model and includes a multi-head self-attention module and a pointwise feedforward network. Each Transformer block can accept defect category information as embedding conditions. The embedding conditions are fused with the input representation using methods such as adaLN_Zero, in-context Conditioning, and adaLN. Figure 3 Taking adaLN_Zero as an example, the feature vector of defect category information is fused with the input representation after feature extraction by a multilayer perceptron (MLP).
[0051] The remodeling layer is used to recover an image of the same size as the wafer defect sample image based on feature information, so as to obtain the sample reconstruction image.
[0052] Specifically, the reshaping layer first performs a linear transformation on the obtained feature information to adjust its dimensions to adapt to the subsequent reshaping operation, and then readjusts the linearly transformed vector to an image of the same size as the wafer defect sample image.
[0053] Please refer to Figure 3 The Diffusion Transformers model can also include a normalization layer (Layer Norm), which normalizes the output of each sub-layer of the Transformer block (e.g., multi-head self-attention module and feedforward network) so that the output of each layer has the same mean and variance.
[0054] This application introduces the Diffusion Transformers model to generate wafer defect images, providing a novel approach for wafer defect image generation in semiconductor inspection. The attention mechanism of the Diffusion Transformers model enables it to understand the structure of data (such as text grammar and image spatial relationships), generating more logical and structurally consistent wafer defect images. For example, it can accurately locate defect positions in wafer defect image generation, achieving personalized defect generation. Furthermore, the Diffusion Transformers model possesses long sequence processing capabilities, effectively handling long text data and overcoming the limitations of traditional diffusion models (such as U-Net) in processing long sequences. This facilitates the use of defect category hint text to provide defect category information.
[0055] Please refer to Figure 4In some embodiments, the defect category prompt text is encoded using a CLIP model text encoder.
[0056] Please refer to Figure 5 The CLIP model includes a text encoder and an image encoder, enabling the modeling of the relationship between images and text. The text encoder encodes the input text to obtain text feature vectors, such as... Figure 5 For N texts, encoding yields N text feature vectors T1, T2, ..., T. N An image encoder is used to encode an input image to obtain an image feature vector, for example... Figure 5 For N images, encoding yields N image feature vectors I1, I2, ..., I... N .
[0057] In this embodiment, the defect category prompt text is encoded by introducing CLIP's text encoder to obtain its text feature vector input to the diffusion model. This can efficiently encode the defect category prompt text and is beneficial for aligning the semantics of the defect category prompt text and the image, effectively integrating the defect category prompt text with the diffusion model.
[0058] Based on the training method of the aforementioned wafer defect image generation model, this application also provides a wafer defect image generation method, please refer to [reference needed]. Figure 6 In some embodiments, the method includes the following steps: Step 210: Obtain the defect category information of the noise to be reconstructed and the target defect category; wherein, the noise to be reconstructed can be randomly generated noise, such as Gaussian noise, etc. Step 220: The wafer defect image generation model obtained by the training method of any embodiment of this application uses the defect category information of the target defect category as guiding information to perform stepwise denoising processing on the noise to be reconstructed to generate a wafer defect image.
[0059] In the above training method, defect category information is used as guiding information, and the model learns by adding and removing noise. This enables the model to learn to generate wafer defect images that meet the defect category requirements from noise. When using this model to generate wafer defect images, only noise needs to be input to generate high-quality wafer defect images of the target defect category. It does not depend on actual wafer defect sample images and can meet the confidentiality requirements of semiconductor manufacturers.
[0060] In some embodiments, the diffusion model is a Diffusion Transformers model. The defect category information of the target defect category can be used as the embedding condition input to the Diffusion Transformers model; at each iteration, the noise of the current iteration and the time step t are fused and used as the input representation of the Diffusion Transformers model.
[0061] For details, please refer to the relevant introduction above. The difference is that here, only noise and time step t are needed to form the input representation, and actual wafer defect sample images are not required.
[0062] In some embodiments, the defect category information of the target defect category is the defect category hint text of the target defect category, which includes the name and / or description text of the target defect category.
[0063] If only the defect category name was used during training, then the name of the target defect category must be entered here; if only the description text of the defect category was used during training, then the description text of the target defect category must be entered here; if both the defect category name and the description text of the defect category were used during training, then either the name of the target defect category or the description text can be entered here.
[0064] Figure 7 The image shows wafer defect images of three categories generated using the wafer defect image generation method of this application. Bump Scratch represents a bump opening scratch, appearing as a black dot-like structure with a bright white center in the image; Copper Halo represents a bump opening anomaly, appearing as a black opening with a non-circular outer contour (normally circular); Crack represents surface cracking, appearing as black scratches of varying lengths and sizes in the image. It can be seen that the wafer defect image generation method of this application can generate wafer defect images that are indistinguishable from actual visual appearance, preserving the main morphological features of various defects.
[0065] It should be noted that the diffusion model used in this application is employed as the wafer defect image generation model. This allows for the introduction of defect category information as guiding information, and the model learns to generate wafer defect images through a noise-adding and denoising process. This enables the final model to generate high-quality wafer defect images that conform to the target defect category. In other words, this application proposes a novel method for generating semiconductor wafer defect images based on the diffusion Transformers model. It utilizes the diffusion Transformers model technique from deep learning to train a diffusion model for semiconductor wafer defects. The automatically trained diffusion Transformers model can automatically generate high-quality defect images of the corresponding defect category based on the defect category input, which are then used to fill in high-quality training samples.
[0066] It should be noted that this application has several advantages in generating defect text descriptions using Diffusion Transformers: (1) In the case of a lack of defect categories, it can generate high-quality and diverse defect sample images in a timely manner for AI algorithm training. At the same time, due to the confidentiality principle of the semiconductor industry, it can generate high-quality imitation images without relying on actual sample data; (2) Using defect text or description as input for algorithm model generation helps business personnel to use defect category text or partial description encoding to directly generate defect images that match the description, without being limited to a fixed input format, making it more convenient for business personnel to use; (3) Based on the original Diffusion Transformers, by introducing the text encoding features of the CLIP large model, it effectively integrates text description information with the defect generation algorithm network Diffusion Transformers, providing a new algorithm processing approach.
[0067] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.
[0068] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.
Claims
1. A training method for a wafer defect image generation model, characterized in that, include: Acquire wafer defect sample images and their defect category information for multiple different defect categories; Using a diffusion model and defect category information as guiding information, each wafer defect sample image is progressively denoised and then progressively denoised to recover the wafer defect sample image and obtain the sample reconstruction image. The diffusion model is trained using the reconstructed sample image and the wafer defect sample image until a preset training stopping condition is reached, thereby obtaining a wafer defect image generation model; wherein, the training objective includes making the reconstructed sample image consistent with the corresponding wafer defect sample image.
2. The training method as described in claim 1, characterized in that, The defect category information is defect category prompt text, which is encoded into the data format of the guidance information of the diffusion model for input into the diffusion model.
3. The training method as described in claim 1, characterized in that, The diffusion model is the DiffusionTransformers model.
4. The training method as described in claim 3, characterized in that, The defect category information is used as an embedding condition for the Diffusion Transformers model. During each iteration, the wafer defect sample image, the noise of the current iteration, and the time step t are fused and used as input representations for the Diffusion Transformers model.
5. The training method as described in claim 4, characterized in that, The Diffusion Transformers model includes a block layer, multiple Transformer blocks, and a reshaping layer; The block layer is used to divide the noise tensor into multiple noise tensor blocks and add position encoding to each noise tensor block, so that the multiple noise tensor blocks are fused with the wafer defect sample image and the time step t; The multiple Transformer blocks are used to extract features from the input representation and fuse them with the defect category information to obtain feature information; The reshaping layer is used to recover an image of the same size as the wafer defect sample image based on the feature information, so as to obtain the sample reconstruction image.
6. The training method as described in claim 4, characterized in that, The defect category information is defect category prompt text, which is encoded into the data format of the embedding conditions of the Diffusion Transformers model for input into the Diffusion Transformers model.
7. The training method as described in claim 2 or 6, characterized in that, The defect category prompt text is encoded using a CLIP model text encoder.
8. The training method as described in claim 2 or 6, characterized in that, The defect category prompt text includes the defect category name and / or a description of the defect category.
9. A method for generating wafer defect images, characterized in that, include: Obtain defect category information for the noise to be reconstructed and the target defect category; The wafer defect image generation model obtained by the training method according to any one of claims 1 to 8 uses the defect category information of the target defect category as guiding information to perform stepwise denoising processing on the noise to be reconstructed to generate a wafer defect image.
10. The wafer defect image generation method as described in claim 9, characterized in that, The diffusion model is a Diffusion Transformers model. The defect category information of the target defect category is used as the embedding condition of the Diffusion Transformers model and input into the Diffusion Transformers model. In each iteration, the noise and time step t of the current iteration are fused and used as the input representation of the Diffusion Transformers model.
11. The wafer defect image generation method as described in claim 9, characterized in that, The defect category information of the target defect category is the defect category prompt text of the target defect category, which includes the name and / or description text of the target defect category.
12. A computer-readable storage medium, characterized in that, The medium stores a computer program that can be executed by a processor to implement the training method for the wafer defect image generation model as described in any one of claims 1 to 8 or the wafer defect image generation method as described in any one of claims 9 to 11.