Chip defect detection method, device, equipment and medium based on diffusion model

Abstract

The present application discloses a chip defect detection method, device, equipment and medium based on a diffusion model, which relates to the technical field of chip defect detection, and includes: using a noise encoder to extract features from the noise-added historical segmentation image to obtain the current noise features; using an image encoder to extract features from the historical RGB original image to obtain the current image features, and combining the above features to obtain the current feature combination information; using an attention module to process the current feature combination information to obtain the current chip defect prediction image and the current noisy image; using the noise encoder to extract features from the current noisy image to obtain new current noise features, and using an extraction module to extract features from the current chip defect prediction image to obtain new current image features; repeating the above steps until a preset training end condition is met, and outputting the corresponding trained chip defect detection model. In this way, the detection of chip defects is accurately and flexibly realized.

Description

Technical Field

[0001] The present invention relates to the technical field of chip defect detection, and particularly to a chip defect detection method, device, equipment and medium based on a diffusion model. Background Art

[0002] Chip defect detection technology plays an important role in the semiconductor industry, especially in the chip design and manufacturing processes. With the rapid development of electronic products, especially the popularization of smartphones, Internet of Things devices and artificial intelligence, the requirements for the performance and quality of chips are getting higher and higher. This makes the detection of chip defects in the semiconductor manufacturing process become increasingly important. Traditional defect detection methods mainly rely on manual visual inspection and optical microscopes. Although these methods can identify some obvious defects, such as scratches, contamination and surface non-uniformity, etc., they have obvious limitations in the identification of tiny or hidden defects.

[0003] In recent years, with the continuous progress of chip manufacturing technology, the size of chips has become smaller and smaller, while the functions have become more and more complex. This change challenges the effectiveness of traditional detection methods. Especially when facing complex circuit structures, manual detection is not only time-consuming and laborious, but also prone to missed detection or misdetection, thus affecting the overall quality and reliability of chips. In addition, traditional technologies are difficult to meet the requirements of modern production lines for real-time and automation, resulting in bottlenecks in the detection process.

[0004] With the rapid development of technologies such as machine learning, deep learning and computer vision, chip defect detection has entered a new era. By using deep learning algorithms, complex features can be extracted from a large amount of image data, and various defects can be automatically identified, greatly improving the accuracy and speed of detection. However, existing methods still face some challenges, such as unbalanced data sets and insufficient generalization ability of training models. Therefore, how to comprehensively and flexibly detect chip defects is an urgent problem to be solved at present. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a chip defect detection method, device, equipment and medium based on a diffusion model, which can accurately and flexibly detect chip defects. The specific solutions are as follows:

[0006] In a first aspect, the present application provides a chip defect detection method based on a diffusion model, including:

[0007] Obtaining a historical chip defect data set; the historical chip defect data set includes historical RGB original images obtained by imaging the chips and corresponding historical segmentation images reflecting the chip defect regions;

[0008] Add noise to the historical segmented image to obtain a noisy image, and use the noise encoder in the chip defect detection model to be trained to extract features from the noisy image to obtain the current noise features; the chip defect detection model to be trained is a model constructed based on a diffusion model;

[0009] Use the image encoder in the chip defect detection model to be trained to extract features from the historical RGB original image to obtain the current image features, and combine the current noise features and the current image features to obtain the current feature combination information;

[0010] Use the attention module for step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained to process the current feature combination information to obtain the current chip defect prediction image and the current noisy image;

[0011] Extract features from the current noisy image through the noise encoder to obtain new current noise features, and use the extraction module for cyclic feature extraction to extract features from the current chip defect prediction image to obtain new current image features;

[0012] Jump to the step of combining the current noise features and the current image features to obtain the current feature combination information until the preset training end condition is met, and output the corresponding trained chip defect detection model;

[0013] When the current RGB original image corresponding to the chip to be detected is obtained, input the current RGB original image into the trained chip defect detection model to obtain the defect detection result of the chip to be detected output by the trained chip defect detection model.

[0014] Optionally, the step of adding noise to the historical segmented image to obtain a noisy image, and using the noise encoder in the chip defect detection model to be trained to extract features from the noisy image to obtain the current noise features includes:

[0015] Add noise to the historical segmented image based on a discrete uniform distribution and through a preset algorithm to obtain a noisy image;

[0016] Use the noise encoder in the chip defect detection model to be trained and based on the Unet downsampling structure to extract features from the noisy image to obtain the current noise features.

[0017] Optionally, the step of using the noise encoder in the chip defect detection model to be trained and based on the Unet downsampling structure to extract features from the noisy image to obtain the current noise features includes:

[0018] Add a preset attention module to the Unet downsampling structure to obtain an adjusted downsampling structure, and use the adjusted downsampling structure and the noise encoder in the chip defect detection model to be trained to extract features from the noise-added image to obtain the current noise features.

[0019] Optionally, the extracting current image features by using the image encoder in the chip defect detection model to be trained from the historical RGB original image includes:

[0020] Use the Pyramid Vision Transformer as the backbone network for extracting image features, and use the image encoder in the chip defect detection model to be trained to extract features from the historical RGB original image to obtain corresponding image output features, and perform channel reduction on the image output features to obtain the current image features.

[0021] Optionally, the processing of the current feature combination information by using the attention module for step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained to obtain the current chip defect prediction image and the current noisy image includes:

[0022] Use the attention module for step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained, and screen the chip defect position features from the current feature combination information by element-wise multiplication to obtain corresponding channel attention enhanced features;

[0023] Determine the current chip defect prediction image and the current noisy image based on the channel attention enhanced features.

[0024] Optionally, the extracting new current image features by using the extraction module for cyclic feature extraction from the current chip defect prediction image includes:

[0025] Use the extraction module for cyclic feature extraction to extract features from the current chip defect prediction image to obtain prediction image feature information;

[0026] Screen the prediction image feature information by element-wise multiplication to obtain a screening result, and determine new current image features based on the screening result.

[0027] Optionally, the chip defect detection method based on the diffusion model further includes:

[0028] Use the cross-entropy loss function to calculate the image pixels in each chip defect prediction image to obtain the loss value of the image pixels;

[0029] Sort the image pixels based on the magnitude of the loss value, and remove the image pixels with loss values less than a preset threshold from the sorting result.

[0030] In a second aspect, the present application provides a chip defect detection device based on a diffusion model, including:

[0031] A dataset acquisition module for acquiring a historical chip defect dataset; the historical chip defect dataset includes historical RGB original images obtained by imaging the chip and corresponding historical segmentation images reflecting the chip defect regions;

[0032] A feature acquisition module for performing noise addition processing on the historical segmentation image to obtain a noise-added image, and using a noise encoder in a chip defect detection model to be trained to extract features from the noise-added image to obtain a current noise feature; the chip defect detection model to be trained is a model constructed based on a diffusion model;

[0033] A feature combination module for using an image encoder in the chip defect detection model to be trained to extract features from the historical RGB original image to obtain a current image feature, and combining the current noise feature and the current image feature to obtain current feature combination information;

[0034] A picture acquisition module for using an attention module for step-by-step feature refinement in a preset decoder in the chip defect detection model to be trained to process the current feature combination information to obtain a current chip defect prediction image and a current noisy image;

[0035] A feature extraction module for using the noise encoder to extract features from the current noisy image to obtain a new current noise feature, and using an extraction module for cyclic feature extraction to extract features from the current chip defect prediction image to obtain a new current image feature;

[0036] A model output module for jumping to the step of combining the current noise feature and the current image feature to obtain the current feature combination information until a preset training end condition is met, and outputting a corresponding trained chip defect detection model;

[0037] An image detection module for, when obtaining a current RGB original image corresponding to a chip to be detected, inputting the current RGB original image into the trained chip defect detection model to obtain a defect detection result of the chip to be detected output by the trained chip defect detection model.

[0038] In a third aspect, the present application provides an electronic device, including:

[0039] A memory for storing a computer program;

[0040] A processor for executing the computer program to implement the foregoing chip defect detection method based on a diffusion model.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the foregoing chip defect detection method based on a diffusion model is implemented.

[0042] In the present application, first, a historical chip defect data set is obtained; the historical chip defect data set includes historical RGB original images obtained by imaging chips and corresponding historical segmentation images reflecting chip defect regions; the historical segmentation images are subjected to noise addition processing to obtain noise-added images, and a noise encoder in a chip defect detection model to be trained is used to extract features from the noise-added images to obtain current noise features; the chip defect detection model to be trained is a model constructed based on a diffusion model; an image encoder in the chip defect detection model to be trained is used to extract features from the historical RGB original images to obtain current image features, and the current noise features and the current image features are combined to obtain current feature combination information; an attention module for gradually refining features in a preset decoder in the chip defect detection model to be trained is used to process the current feature combination information to obtain a current chip defect prediction image and a current noisy image; the noise encoder is used to extract features from the current noisy image to obtain new current noise features, and an extraction module for cyclic feature extraction is used to extract features from the current chip defect prediction image to obtain new current image features; jump to the step of combining the current noise features and the current image features to obtain current feature combination information until a preset training end condition is met, and the trained chip defect detection model is output; when a current RGB original image corresponding to a chip to be detected is obtained, the current RGB original image is input into the trained chip defect detection model to obtain a defect detection result of the chip to be detected output by the trained chip defect detection model. As can be seen from the above, the present application can perform more refined distribution modeling on chip defects with a large number of categories and similar optical forms of categories, such as color and shape, so that the model has better generalization ability when tested in an environment with variable downstream tasks; the extraction module for cyclic feature extraction and the attention module for gradually refining features enhance the processing ability of the diffusion model for input features, thereby further improving the generalization ability of the diffusion model on chip defect data and improving the segmentation ability; finally, the performance of the trained chip defect detection model is tested to ensure the accuracy of model training. In this way, the detection of chip defects is accurately and flexibly realized. Description of the Drawings

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0044] Figure 1 Flowchart of a chip defect detection method based on a diffusion model disclosed in the present application;

[0045] Figure 2 Flowchart of the attention module disclosed in the present application;

[0046] Figure 3 Flowchart of the recurrent feature extraction module disclosed in the present application;

[0047] Figure 4 Algorithm flowchart of a chip defect detection method based on a diffusion model disclosed in the present application;

[0048] Figure 5 Structure diagram of a chip defect detection device based on a diffusion model disclosed in the present application;

[0049] Figure 6 Structure diagram of an electronic device disclosed in the present application. Detailed implementation manners

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0051] With the rapid development of technologies such as machine learning, deep learning, and computer vision, chip defect detection has entered a new era. By using deep learning algorithms, complex features can be extracted from a large amount of image data, and various defects can be automatically identified, greatly improving the accuracy and speed of detection. However, existing methods still face some challenges, such as unbalanced datasets, insufficient generalization ability of the trained models, etc. Therefore, this application will specifically introduce a chip defect detection method based on a diffusion model that can address the above issues.

[0052] See Figure 1 As shown, the embodiments of the present invention disclose a chip defect detection method based on a diffusion model, which may include:

[0053] Step S11: Obtain a historical chip defect dataset; the historical chip defect dataset includes historical RGB raw images obtained by imaging the chip and corresponding historical segmentation images reflecting the defective areas of the chip.

[0054] In this embodiment, a historical chip defect dataset including historical RGB raw images and historical segmentation images is obtained. Among them, the RGB raw images represent three color channels: Red, Green, and Blue. It should be noted that the historical RGB raw images are unprocessed color images obtained through the chip imaging process. The historical segmentation images are images that can reflect the defective areas of the chip based on an image segmentation algorithm and the historical RGB images.

[0055] Step S12: Perform noise addition processing on the historical segmentation image to obtain a noise-added image, and use the noise encoder in the chip defect detection model to be trained to extract features from the noise-added image to obtain the current noise features; the chip defect detection model to be trained is a model constructed based on a diffusion model.

[0056] In this embodiment, the historical segmentation image is processed by adding noise based on a discrete uniform distribution and a preset algorithm to obtain a noise-added image; the noise encoder in the chip defect detection model to be trained and based on the Unet (a deep learning framework for image segmentation) downsampling structure are used to extract features from the noise-added image to obtain the current noise features. It can be understood that the object of operation of the diffusion model in this embodiment is the historical segmentation image in a discrete state, and the discrete state can be represented as a single-channel integer-type matrix, and each bit of the matrix is represented by an integer indicating which category the picture pixel corresponding to this position in the matrix belongs to. In the forward and backward process modeling of DDPM (Denoising Diffusion Probabilistic Models), both rely on continuous random variables, Gaussian variables, to operate on the forward and backward processes. For a three-dimensional image, the pixel values in the image are continuous. Therefore, the original image data distribution can be damaged by adding continuous Gaussian noise collected from Gaussian variables, so that the distribution of the image gradually shifts to a complete Gaussian distribution. For the discrete matrix of the historical segmentation image, it can be regarded as a grayscale image. Adding Gaussian noise can shift the distribution of the grayscale image represented by the discrete matrix to the Gaussian distribution. However, the above approach is to transform discrete variables into continuous variables for processing, which is likely to introduce irrelevant noise data and thus affect the learning of the distribution. Another more reasonable approach should be to simulate the forward and backward processes of the diffusion model as a process in which each value in the discrete matrix changes to another value with a certain probability.

[0057] It should be noted that for a certain bit of the discrete matrix of the segmented image , the forward transition probability matrix can be expressed as , and the forward process can be expressed as . Similar to DDPM, the forward process can also be completed in one step under the condition of a Markov chain . Among them, Q is the probability transition matrix of the forward diffusion process, indicating the probability that the value of each pixel of the segmented image is converted to other values refers to the probability transition matrix of pixel ij at a certain time step; Cat represents Categorical Distribution, that is, the categorical distribution. In this discrete diffusion process, various different discrete distributions can be used as the noise similar to Gaussian noise in DDPM and added to the original input data. In this embodiment, the discrete uniform distribution is used as the noise. In this case ;

[0058] Among them, K represents the number of categories of defect types, 1 represents a column vector of all 1s, and the forward process can be expressed as , it should be noted that is the parameter in p .

[0059] For the reverse process of the diffusion model, in this embodiment, the final target, that is, the final segmented image, is directly predicted at each sampling step, and then the outputs of each reverse sampling step are collected and averaged as the final output . Among them represents the final prediction result of the model represents at the time step t the prediction of the segmentation result

[0060] In this embodiment, a preset attention module is added to the Unet downsampling structure to obtain an adjusted downsampling structure, and the adjusted downsampling structure and the noise encoder in the chip defect detection model to be trained are used to extract features from the noise-added image to obtain the current noise features. That is to say, the downsampling structure of Unet is used to extract features from the input noise image, that is, the segmented result image that has been damaged by random uniform distribution noise. In this embodiment, an attention module (that is, the attention mechanism module) in Transformer (that is, a deep learning model based on the self-attention mechanism) is added to each model block of the downsampling to enhance the ability of the downsampling module to extract features from the noisy image, and Transformer is used to extract features from the noise-added image to obtain the current noise features

[0061] Step S13: Use the image encoder in the to-be-trained chip defect detection model to extract features from the historical RGB original image to obtain the current image features, and combine the current noise features and the current image features to obtain the current feature combination information.

[0062] In this embodiment, the Pyramid Vision Transformer is used as the backbone network for extracting image features. The image encoder in the to-be-trained chip defect detection model is used to extract features from the historical RGB original image to obtain the corresponding image output features, and the channel of the image output features is reduced to obtain the current image features. Specifically, the pre-trained model PVT (Pyramid Vision Transformer) is used as the image extraction backbone network. It receives the optical image of the chip with defects and outputs four-level features of the image, and the feature sizes are 512, 320, 128, and 64 respectively. To reduce the computational complexity, the above four features are first reduced in channel to 64 respectively. During the downsampling process of Unet, the four-level features output by PVT are respectively combined with the output features of the four levels in the downsampling process, and then input into the next downsampling module. The downsampling module finally outputs a 64-dimensional vector , and the output features of each level , and then is input into a 6-layer Transformer block for feature enhancement.

[0063] Step S14: Use the attention module for step-by-step feature refinement in the preset decoder in the to-be-trained chip defect detection model to process the current feature combination information to obtain the current chip defect prediction image and the current noisy image.

[0064] In this embodiment, during the upsampling stage, for the output features of each upsampling level , they are respectively combined with the output features of each level in the downsampling process and then input into the linear layer of the prediction result to obtain the stage prediction result of each level. The prediction result is a 01 matrix, which shows the positions of all defects in the input image but does not show the types of defects. For each upsampling module at each level, there is an attention module for step-by-step feature refinement. The input of this module is the input feature F of this layer, and the stage prediction result P of the input feature of this layer is the current chip defect prediction image.

[0065] In this embodiment, the attention module for gradually refining features in the preset decoder of the chip defect detection model to be trained is utilized, and the chip defect position features are screened from the current feature combination information through element-wise multiplication to obtain corresponding channel attention enhanced features; based on the channel attention enhanced features, the current chip defect prediction image and the current noisy image are determined. The flowchart of the attention module for gradually refining features is as Figure 2 shown. Combining the above input features and prediction results, the input features are screened through element-wise multiplication. Only the features at the positions where defects existed in the previous prediction of the input features will participate in the upsampling process of this layer, and the features at the positions predicted as the background will be set to 0. After multiplication, the enhanced features are obtained , and is input into the attention enhancement module that combines spatial and channel attention. Specifically, for the input feature F, F consists of three dimensions C, H, and W. Among them, C is the channel, and H and W are the height and width respectively. First, the shape of F is reshaped to obtain three feature matrices Q, K, and V, and the matrix shapes are all C*(H*W). Then, the matrices Q and K are multiplied, and the channel attention matrix X of C*C is obtained through the softmax function:

[0066] ;

[0067] where, represents the influence magnitude of the j-th channel on the i-th channel, represents Q of the i-th channel, represents K of the j-th channel. Then, X is multiplied by V to obtain the channel attention enhanced feature , , so as to screen out the semantic information useful for identifying the chip defect target. Through the step-by-step refinement of the prediction results, the edge learning of the pixel features where the defects are located is enhanced.

[0068] Step S15: The current noisy image is subjected to feature extraction by the noise encoder to obtain new current noise features, and the current chip defect prediction image is subjected to feature extraction by the extraction module for cyclic feature extraction to obtain new current image features.

[0069] In this embodiment, the utilization of the extraction module for cyclic feature extraction to extract features from the current chip defect prediction image to obtain new current image features includes: using the extraction module for cyclic feature extraction to extract prediction image feature information from the current chip defect prediction image; screening the prediction image feature information through element-wise multiplication to obtain a screening result, and determining new current image features based on the screening result. The flowchart of the extraction module for cyclic feature extraction is as Figure 3As shown, in the reverse sampling stage of the diffusion model, a segmentation result is output for each sampling time period.

[0070] The segmentation result is element-wise multiplied with the output of the image feature extraction backbone to obtain , and then by element-wise multiplying 1 - P with the output of the image feature extraction backbone to obtain , and then combining the and of each level by channel to obtain , and taking as the feature of the image and inputting it into the Unet of the diffusion model.

[0071] Step S16: Jump to the step of combining the current noise feature and the current image feature to obtain the current feature combination information, until a preset training end condition is met, and output the corresponding trained chip defect detection model.

[0072] In this embodiment, after the preset training end condition is met and the corresponding trained chip defect detection model is output, it may further include: calculating the loss value of the image pixels in each chip defect prediction image by using the cross-entropy loss function; sorting the image pixels based on the magnitude of the loss value, and removing the image pixels with loss values less than a preset threshold from the sorting result. Specifically, use , as the loss function of the trained chip defect detection model, where is a cross-entropy loss function combined with online hard example mining, is a convex expansion based on submodular loss, P is the model prediction, and GT represents the ground truth. After calculating the loss of each pixel using the cross-entropy loss function, the image pixels are sorted from large to small based on the loss magnitude, and the pixels with small losses are removed, only retaining the pixels with large losses. In summary, the algorithm flow chart of this application is as Figure 4 shown.

[0073] Step S17: When the current RGB original image corresponding to the chip to be detected is obtained, input the current RGB original image into the trained chip defect detection model to obtain the defect detection result of the chip to be detected output by the trained chip defect detection model.

[0074] In this embodiment, the trained chip defect detection model is tested. It can be understood that the trained chip defect detection model has highly accurate and efficient image recognition and analysis capabilities, and this model can analyze the RGB original image corresponding to the chip to be detected input. Specifically, key features in the RGB original image are extracted. The key features include, but are not limited to, color changes, texture differences, shape irregularities, etc., and the key features are intelligently matched and compared to obtain the defect detection result of the chip to be detected.

[0075] As can be seen from the above, in this embodiment, a historical chip defect data set is first obtained; the historical chip defect data set includes historical RGB original images obtained after imaging the chip and corresponding historical segmentation images reflecting the chip defect areas; the historical segmentation images are subjected to noise addition processing to obtain the images after noise addition, and the noise encoder in the chip defect detection model to be trained is used to extract features from the images after noise addition to obtain the current noise features; the chip defect detection model to be trained is a model constructed based on a diffusion model; the image encoder in the chip defect detection model to be trained is used to extract features from the historical RGB original image to obtain the current image features, and the current noise features and the current image features are combined to obtain the current feature combination information; the attention module for gradually refining features in the preset decoder in the chip defect detection model to be trained is used to process the current feature combination information to obtain the current chip defect prediction image and the current noisy image; the noise encoder is used to extract features from the current noisy image to obtain the new current noise features, and the extraction module for cyclic feature extraction is used to extract features from the current chip defect prediction image to obtain the new current image features; jump to the step of combining the current noise features and the current image features to obtain the current feature combination information until the preset training end condition is met, and the corresponding trained chip defect detection model is output; when the current RGB original image corresponding to the chip to be detected is obtained, the current RGB original image is input into the trained chip defect detection model to obtain the defect detection result of the chip to be detected output by the trained chip defect detection model. As can be seen from the above, this embodiment can perform more refined distribution modeling on chip defects with a large number of categories and similar optical forms of categories, such as color and shape, so that the model has better generalization ability when tested in an environment with variable downstream tasks; the extraction module for cyclic feature extraction and the attention module for gradually refining features enhance the processing ability of the diffusion model for input features, thereby further improving the generalization ability of the diffusion model on chip defect data and improving the segmentation ability; finally, the performance of the trained chip defect detection model is tested to ensure the accuracy of model training. In this way, the detection of chip defects is accurately and flexibly realized.

[0076] Correspondingly, referring to Figure 5 as shown, an embodiment of the present application further provides a chip defect detection device based on a diffusion model, which may include:

[0077] A dataset acquisition module 11, configured to acquire a historical chip defect dataset; the historical chip defect dataset includes historical RGB original images obtained by imaging the chip and corresponding historical segmentation images reflecting the chip defect areas;

[0078] A feature acquisition module 12, configured to perform noise addition processing on the historical segmentation image to obtain a noise-added image, and use a noise encoder in the chip defect detection model to be trained to extract features from the noise-added image to obtain a current noise feature; the chip defect detection model to be trained is a model constructed based on a diffusion model;

[0079] A feature combination module 13, configured to use an image encoder in the chip defect detection model to be trained to extract features from the historical RGB original image to obtain a current image feature, and combine the current noise feature and the current image feature to obtain current feature combination information;

[0080] A picture acquisition module 14, configured to use an attention module for step-by-step feature refinement in a preset decoder in the chip defect detection model to be trained to process the current feature combination information to obtain a current chip defect prediction image and a current noisy image;

[0081] A feature extraction module 15, configured to use the noise encoder to extract features from the current noisy image to obtain a new current noise feature, and use an extraction module for cyclic feature extraction to extract features from the current chip defect prediction image to obtain a new current image feature;

[0082] A model output module 16, configured to jump to the step of combining the current noise feature and the current image feature to obtain current feature combination information until a preset training end condition is met, and output a corresponding trained chip defect detection model;

[0083] An image detection module 17, configured to, when a current RGB original image corresponding to a chip to be detected is obtained, input the current RGB original image into the trained chip defect detection model to obtain a defect detection result of the chip to be detected output by the trained chip defect detection model.

[0084] As can be seen from the above, the present application can perform more refined distribution modeling on chip defects with a large number of categories and similar optical forms of categories, such as color and shape, so that the model has better generalization ability when tested in an environment with variable downstream tasks; the extraction module for cyclic feature extraction and the attention module for gradual feature refinement enhance the processing ability of the diffusion model for input features, thereby further improving the generalization ability of the diffusion model on chip defect data and enhancing the segmentation ability; finally, the performance of the trained chip defect detection model is tested to ensure the accuracy of model training. In this way, the detection of chip defects is accurately and flexibly realized.

[0085] In some specific embodiments, the feature acquisition module 12 may include:

[0086] An image noise addition unit for adding noise to the historical segmentation image based on a discrete uniform distribution and through a preset algorithm to obtain a noise-added image;

[0087] A feature extraction sub-module for using the noise encoder in the chip defect detection model to be trained and based on the Unet downsampling structure to extract features from the noise-added image to obtain the current noise features.

[0088] In some specific embodiments, the feature extraction sub-module may include:

[0089] A feature extraction unit for adding a preset attention module to the Unet downsampling structure to obtain an adjusted downsampling structure, and using the adjusted downsampling structure and the noise encoder in the chip defect detection model to be trained to extract features from the noise-added image to obtain the current noise features.

[0090] In some specific embodiments, the feature combination module 13 may include:

[0091] A channel reduction unit for using the pyramid vision Transformer as the backbone network for extracting image features, and using the image encoder in the chip defect detection model to be trained to extract features from the historical RGB original image to obtain corresponding image output features, and reducing the channels of the image output features to obtain the current image features.

[0092] In some specific embodiments, the image acquisition module 14 may include:

[0093] An information screening unit for using the attention module for gradual feature refinement in the preset decoder in the chip defect detection model to be trained, and screening the chip defect position features from the current feature combination information through element-wise multiplication to obtain corresponding channel attention enhanced features;

[0094] An image determination unit for determining a current chip defect prediction image and a current noisy image based on the channel attention enhanced features.

[0095] In some specific embodiments, the feature extraction module 15 may include:

[0096] A feature extraction unit for extracting feature information of a prediction image by using an extraction module for cyclic feature extraction on the current chip defect prediction image;

[0097] A feature screening unit for screening the prediction image feature information by element-wise multiplication to obtain a screening result, and determining new current image features based on the screening result.

[0098] In some specific embodiments, the chip defect detection device based on a diffusion model may further include:

[0099] A loss value calculation unit for calculating a loss value of the image pixels in each chip defect prediction image by using a cross-entropy loss function;

[0100] A loss value sorting unit for sorting the image pixels based on the magnitude of the loss value, and removing the image pixels with loss values less than a preset threshold from the sorting result.

[0101] Furthermore, an embodiment of the present application also discloses an electronic device. Figure 6 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment. The content in the figure should not be considered as any limitation on the scope of use of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the chip defect detection method based on a diffusion model disclosed in any of the foregoing embodiments. Additionally, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0102] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and no specific limitation is imposed thereon herein; the input / output interface 25 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application requirements, and no specific limitation is made herein.

[0103] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a magnetic disk, an optical disc, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc. The storage method can be temporary storage or permanent storage.

[0104] Among them, the operating system 221 is used to manage and control each hardware device and the computer program 222 on the electronic device 20, and it can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the chip defect detection method based on the diffusion model executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks.

[0105] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the chip defect detection method based on the diffusion model disclosed above. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0106] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method section.

[0107] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0108] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0109] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising said element.

[0110] The technical solutions provided in this application have been introduced in detail above. Specific examples are used in this text to elaborate on the principles and implementation manners of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A chip defect detection method based on a diffusion model, characterized in that: include: Acquire a historical chip defect data set; the historical chip defect data set includes a historical RGB original image obtained after imaging the chip and a corresponding historical segmented image reflecting the chip defect area; Noising the historical segmented image to obtain a noisy image, and extracting features from the noisy image using a noise encoder in a chip defect detection model to be trained to obtain current noise features; the chip defect detection model to be trained is a model constructed based on a diffusion model; Using the image encoder in the chip defect detection model to be trained to perform feature extraction on the historical RGB original image to obtain current image features, and combining the current noise features with the current image features to obtain current feature combination information; Using the attention module for performing step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained to process the current feature combination information, so as to obtain the current chip defect prediction image and the current noisy image; Performing feature extraction on the current noisy image by the noise encoder to obtain a new current noise feature, and performing feature extraction on the current chip defect prediction image by using an extraction module for cyclic feature extraction to obtain a new current image feature; Jump to the step of combining the current noise feature with the current image feature to obtain current feature combination information, until the preset training end condition is met, and output the corresponding trained chip defect detection model; When the current RGB original image corresponding to the chip to be inspected is obtained, the current RGB original image is input into the trained chip defect detection model to obtain the defect detection result of the chip to be inspected output by the trained chip defect detection model.

2. The chip defect detection method based on the diffusion model according to claim 1, characterized in that: The step of performing noise processing on the historical segmented image to obtain a noisy image, and performing feature extraction on the noisy image using a noise encoder in the chip defect detection model to be trained to obtain a current noise feature, includes: Based on discrete uniform distribution and using a preset algorithm, the historical segmented image is subjected to noise processing to obtain a noisy image; The noise encoder in the chip defect detection model to be trained is used to extract features of the noisy image based on the Unet downsampling structure to obtain the current noise features.

3. The chip defect detection method based on diffusion model according to claim 2, characterized in that: The method of using the noise encoder in the chip defect detection model to be trained and extracting features from the noisy image based on the Unet downsampling structure to obtain the current noise features includes: A preset attention module is added to the Unet downsampling structure to obtain an adjusted downsampling structure, and the adjusted downsampling structure and the noise encoder in the chip defect detection model to be trained are used to perform feature extraction on the noisy image to obtain the current noise feature.

4. The chip defect detection method based on diffusion model according to claim 1, characterized in that: The step of extracting features from the historical RGB original image using the image encoder in the chip defect detection model to be trained to obtain current image features includes: The pyramid vision Transformer is used as the backbone network for extracting image features, and the image encoder in the chip defect detection model to be trained is used to extract features of the historical RGB original image to obtain corresponding image output features, and the image output features are channel-reduced to obtain current image features.

5. The chip defect detection method based on diffusion model according to claim 1, characterized in that: The method of using the attention module for performing step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained to process the current feature combination information to obtain the current chip defect prediction image and the current noisy image includes: Utilize the attention module for performing step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained, and filter the chip defect position features from the current feature combination information by element-by-element multiplication method to obtain the corresponding channel attention enhancement features; A current chip defect prediction image and a current noisy image are determined based on the channel attention enhancement feature.

6. The chip defect detection method based on diffusion model according to claim 1, characterized in that: The method of extracting features of the current chip defect prediction image using the extraction module for cyclic feature extraction to obtain new current image features includes: Using an extraction module for cyclic feature extraction to extract features of the current chip defect prediction image to obtain prediction image feature information; The predicted image feature information is filtered by an element-by-element multiplication method to obtain a filtering result, and a new current image feature is determined based on the filtering result.

7. The chip defect detection method based on the diffusion model according to any one of claims 1 to 6, characterized in that: Also includes: Using a cross entropy loss function to calculate image pixels in each chip defect prediction image to obtain a loss value of the image pixel; The image pixels are sorted based on the size of the loss value, and image pixels whose loss value is less than a preset threshold are eliminated from the sorting result.

8. A chip defect detection device based on a diffusion model, characterized in that: include: A data set acquisition module is used to acquire a historical chip defect data set; the historical chip defect data set includes a historical RGB original image obtained after imaging the chip and a corresponding historical segmented image reflecting the chip defect area; A feature acquisition module, used for performing noise processing on the historical segmented image to obtain a noisy image, and performing feature extraction on the noisy image using a noise encoder in a chip defect detection model to be trained to obtain a current noise feature; the chip defect detection model to be trained is a model constructed based on a diffusion model; A feature combination module is used to extract features of the historical RGB original image using the image encoder in the chip defect detection model to be trained to obtain current image features, and combine the current noise features with the current image features to obtain current feature combination information; An image acquisition module is used to process the current feature combination information using an attention module for performing step-by-step feature refinement in a preset decoder in the chip defect detection model to be trained, so as to obtain a current chip defect prediction image and a current noisy image; A feature extraction module, used to extract features of the current noisy image by the noise encoder to obtain new current noise features, and to extract features of the current chip defect prediction image by using the extraction module for cyclic feature extraction to obtain new current image features; A model output module, used to jump to the step of combining the current noise feature with the current image feature to obtain current feature combination information, until a preset training end condition is met, and output a corresponding trained chip defect detection model; The image detection module is used to input the current RGB original image corresponding to the chip to be detected into the trained chip defect detection model when the current RGB original image corresponding to the chip to be detected is obtained, so as to obtain the defect detection result of the chip to be detected output by the trained chip defect detection model.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the chip defect detection method based on the diffusion model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein, when the computer program is executed by a processor, the chip defect detection method based on the diffusion model as described in any one of claims 1 to 7 is implemented.

Images