Chip defect detection method and device based on diffusion model, equipment and medium

Abstract

The invention discloses a chip defect detection method and device based on a diffusion model, equipment and a medium, and relates to the technical field of chip defect detection, and the method comprises the steps: carrying out the feature extraction of a noise-added historical segmentation image through a noise encoder, and obtaining a current noise feature; performing feature extraction on the historical RGB original image by using an image encoder to obtain current image features, and combining the features to obtain current feature combination information; processing the current feature combination information by using an attention module to obtain a current chip defect prediction image and a current image with noise; performing feature extraction on the current image with noise through a noise encoder to obtain a new current noise feature, and performing feature extraction on the current chip defect prediction image through an extraction module to obtain a new current image feature; and repeating the steps until a preset training ending condition is met, and outputting a corresponding trained chip defect detection model. Therefore, the chip defect detection is accurately and flexibly realized.

Description

Technical Field

[0001] The present invention relates to the technical field of chip defect detection, and in particular to a chip defect detection method, device, equipment and medium based on a diffusion model. Background Art

[0002] Chip defect detection technology plays an important role in the semiconductor industry, especially in the chip design and manufacturing process. With the rapid development of electronic products, especially the popularity of smartphones, IoT devices and artificial intelligence, the performance and quality requirements of chips are getting higher and higher. This makes the detection of chip defects more and more important in the semiconductor manufacturing process. Traditional defect detection methods mainly rely on manual visual inspection and optical microscopes. Although these methods can identify some obvious defects such as scratches, contamination and surface unevenness, they have obvious limitations in identifying small or hidden defects.

[0003] In recent years, with the continuous advancement of chip manufacturing technology, the size of chips has become smaller and smaller, while the functions have become more and more complex. This change has challenged the effectiveness of traditional detection methods, especially when faced with complex circuit structures. Manual detection is not only time-consuming and labor-intensive, but also prone to missed detection or false detection, thus affecting the overall quality and reliability of the chip. In addition, traditional technology is difficult to meet the real-time and automation requirements of modern production lines, leading to bottlenecks in the detection process.

[0004] With the rapid development of technologies such as machine learning, deep learning, and computer vision, chip defect detection has entered a new era. Using deep learning algorithms, complex features can be extracted from large amounts of image data and various defects can be automatically identified, greatly improving the accuracy and speed of detection. However, existing methods still face some challenges, such as unbalanced data sets and insufficient generalization capabilities of training models. Therefore, how to comprehensively and flexibly detect chip defects is an urgent problem to be solved. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a chip defect detection method, device, equipment and medium based on a diffusion model, which can accurately and flexibly detect chip defects. The specific scheme is as follows: In a first aspect, the present application provides a chip defect detection method based on a diffusion model, comprising: Acquire a historical chip defect data set; the historical chip defect data set includes a historical RGB original image obtained after imaging the chip and a corresponding historical segmented image reflecting the chip defect area; Noising the historical segmented image to obtain a noisy image, and extracting features from the noisy image using a noise encoder in a chip defect detection model to be trained to obtain current noise features; the chip defect detection model to be trained is a model constructed based on a diffusion model; Using the image encoder in the chip defect detection model to be trained to perform feature extraction on the historical RGB original image to obtain current image features, and combining the current noise features with the current image features to obtain current feature combination information; Using the attention module for performing step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained to process the current feature combination information, so as to obtain the current chip defect prediction image and the current noisy image; Performing feature extraction on the current noisy image by the noise encoder to obtain a new current noise feature, and performing feature extraction on the current chip defect prediction image by using an extraction module for cyclic feature extraction to obtain a new current image feature; Jump to the step of combining the current noise feature with the current image feature to obtain current feature combination information, until the preset training end condition is met, and output the corresponding trained chip defect detection model; When the current RGB original image corresponding to the chip to be inspected is obtained, the current RGB original image is input into the trained chip defect detection model to obtain the defect detection result of the chip to be inspected output by the trained chip defect detection model.

[0006] Optionally, the performing noise processing on the historical segmented image to obtain a noisy image, and performing feature extraction on the noisy image using a noise encoder in the chip defect detection model to be trained to obtain a current noise feature, includes: Based on discrete uniform distribution and using a preset algorithm, the historical segmented image is subjected to noise processing to obtain a noisy image; The noise encoder in the chip defect detection model to be trained is used to extract features of the noisy image based on the Unet downsampling structure to obtain the current noise features.

[0007] Optionally, the step of using a noise encoder in the chip defect detection model to be trained and extracting features from the noisy image based on a Unet downsampling structure to obtain current noise features includes: A preset attention module is added to the Unet downsampling structure to obtain an adjusted downsampling structure, and the adjusted downsampling structure and the noise encoder in the chip defect detection model to be trained are used to perform feature extraction on the noisy image to obtain the current noise feature.

[0008] Optionally, the extracting features of the historical RGB original image using the image encoder in the chip defect detection model to be trained to obtain current image features includes: The pyramid vision Transformer is used as the backbone network for extracting image features, and the image encoder in the chip defect detection model to be trained is used to extract features of the historical RGB original image to obtain corresponding image output features, and the image output features are channel-reduced to obtain current image features.

[0009] Optionally, the using of the attention module for performing step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained to process the current feature combination information to obtain the current chip defect prediction image and the current noisy image includes: Utilize the attention module for performing step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained, and filter the chip defect position features from the current feature combination information by element-by-element multiplication method to obtain the corresponding channel attention enhancement features; A current chip defect prediction image and a current noisy image are determined based on the channel attention enhancement feature.

[0010] Optionally, the extracting module for cyclic feature extraction is used to extract features from the current chip defect prediction image to obtain new current image features, including: Using an extraction module for cyclic feature extraction to extract features of the current chip defect prediction image to obtain prediction image feature information; The predicted image feature information is filtered by an element-by-element multiplication method to obtain a filtering result, and a new current image feature is determined based on the filtering result.

[0011] Optionally, the chip defect detection method based on the diffusion model further includes: Using a cross entropy loss function to calculate image pixels in each chip defect prediction image to obtain a loss value of the image pixel; The image pixels are sorted based on the size of the loss value, and image pixels whose loss value is less than a preset threshold are eliminated from the sorting result.

[0012] In a second aspect, the present application provides a chip defect detection device based on a diffusion model, comprising: A data set acquisition module is used to acquire a historical chip defect data set; the historical chip defect data set includes a historical RGB original image obtained after imaging the chip and a corresponding historical segmented image reflecting the chip defect area; A feature acquisition module, used for performing noise processing on the historical segmented image to obtain a noisy image, and performing feature extraction on the noisy image using a noise encoder in a chip defect detection model to be trained to obtain a current noise feature; the chip defect detection model to be trained is a model constructed based on a diffusion model; A feature combination module is used to extract features of the historical RGB original image using the image encoder in the chip defect detection model to be trained to obtain current image features, and combine the current noise features with the current image features to obtain current feature combination information; An image acquisition module is used to process the current feature combination information using an attention module for performing step-by-step feature refinement in a preset decoder in the chip defect detection model to be trained, so as to obtain a current chip defect prediction image and a current noisy image; A feature extraction module, used to extract features of the current noisy image by the noise encoder to obtain new current noise features, and to extract features of the current chip defect prediction image by using the extraction module for cyclic feature extraction to obtain new current image features; A model output module, used to jump to the step of combining the current noise feature with the current image feature to obtain current feature combination information, until a preset training end condition is met, and output a corresponding trained chip defect detection model; The image detection module is used to input the current RGB original image corresponding to the chip to be detected into the trained chip defect detection model when the current RGB original image corresponding to the chip to be detected is obtained, so as to obtain the defect detection result of the chip to be detected output by the trained chip defect detection model.

[0013] In a third aspect, the present application provides an electronic device, including: Memory, used to store computer programs; The processor is used to execute the computer program to implement the aforementioned chip defect detection method based on the diffusion model.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned chip defect detection method based on the diffusion model is implemented.

[0015] In the present application, a historical chip defect data set is first obtained; the historical chip defect data set includes a historical RGB original image obtained after imaging the chip and a corresponding historical segmented image reflecting the chip defect area; the historical segmented image is denoised to obtain a noisy image, and the noise encoder in the chip defect detection model to be trained is used to extract features from the noisy image to obtain current noise features; the chip defect detection model to be trained is a model built based on a diffusion model; the image encoder in the chip defect detection model to be trained is used to extract features from the historical RGB original image to obtain current image features, and the current noise features and the current image features are combined to obtain current feature combination information; the preset decoder in the chip defect detection model to be trained is used to perform step-by-step feature extraction The refined attention module processes the current feature combination information to obtain the current chip defect prediction image and the current noisy image; performs feature extraction on the current noisy image through the noise encoder to obtain a new current noise feature, and uses the extraction module for cyclic feature extraction to extract features on the current chip defect prediction image to obtain a new current image feature; jumps to the step of combining the current noise feature with the current image feature to obtain the current feature combination information until the preset training end condition is met, and outputs the corresponding trained chip defect detection model; when the current RGB original image corresponding to the chip to be detected is obtained, the current RGB original image is input into the trained chip defect detection model to obtain the defect detection result of the chip to be detected output by the trained chip defect detection model. As can be seen from the above, the present application can perform more refined distribution modeling for chip defects with a large number of categories and similar optical forms, such as color and shape, so that the model has better generalization ability when tested in an environment with variable downstream tasks; the extraction module for cyclic feature extraction and the attention module for gradual feature refinement enhance the diffusion model's processing ability for input features, thereby further improving the diffusion model's generalization ability on chip defect data and improving the segmentation ability; finally, the performance test of the trained chip defect detection model is performed to ensure the accuracy of the model training. In this way, the detection of chip defects is achieved accurately and flexibly. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0017] Figure 1A flow chart of a chip defect detection method based on a diffusion model disclosed in this application; Figure 2 The flow chart of the attention module disclosed in this application; Figure 3 This is a flow chart of the cyclic feature extraction module disclosed in this application; Figure 4 This is an algorithm flow chart of a chip defect detection method based on a diffusion model disclosed in this application; Figure 5 A schematic diagram of the structure of a chip defect detection device based on a diffusion model disclosed in this application; Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] With the rapid development of technologies such as machine learning, deep learning, and computer vision, chip defect detection has entered a new era. Using deep learning algorithms, complex features can be extracted from large amounts of image data and various defects can be automatically identified, greatly improving the accuracy and speed of detection. However, existing methods still face some challenges, such as unbalanced data sets and insufficient generalization capabilities of training models. Therefore, this application will specifically introduce a chip defect detection method based on a diffusion model that can explain the above problems.

[0020] See also Figure 1 As shown, the embodiment of the present invention discloses a chip defect detection method based on a diffusion model, which may include: Step S11, obtaining a historical chip defect data set; the historical chip defect data set includes a historical RGB original image obtained after imaging the chip and a corresponding historical segmented image reflecting the chip defect area.

[0021] In this embodiment, a historical chip defect data set including a historical RGB original image and a historical segmented image is obtained, wherein the RGB original image represents three color channels: Red, Green, and Blue. It should be noted that the historical RGB original image is an unprocessed color image obtained through the chip imaging process. The historical segmented image is an image that can reflect the chip defect area based on the image segmentation algorithm and the historical RGB image.

[0022] Step S12, performing noise processing on the historical segmented image to obtain a noisy image, and using the noise encoder in the chip defect detection model to be trained to perform feature extraction on the noisy image to obtain current noise features; the chip defect detection model to be trained is a model constructed based on a diffusion model.

[0023] In this embodiment, the historical segmented image is subjected to noise processing based on discrete uniform distribution and a preset algorithm to obtain a noisy image; the noise encoder in the chip defect detection model to be trained is used and based on the Unet (a deep learning framework for image segmentation) downsampling structure to extract features from the noisy image to obtain the current noise feature. It can be understood that the object of the diffusion model operation in this embodiment is the historical segmented image in a discrete state, and the discrete state can be represented as a single-channel integer type matrix, and each bit of the matrix is ​​represented by an integer to indicate which category the image pixel corresponding to the position of the matrix belongs to. In the forward process and reverse process modeling of DDPM (Denoising Diffusion Probabilistic Models, i.e., denoising diffusion probability model), both rely on continuous random variables Gaussian variables to operate the forward process and reverse process. For three-dimensional images, the pixel values ​​in the image are continuous, so the original image data distribution can be damaged by adding continuous Gaussian noise collected from Gaussian variables, so that the distribution of the image gradually shifts to a complete Gaussian distribution. The discrete matrix of the historical segmentation image can be regarded as a grayscale image. Adding Gaussian noise can shift the distribution of the grayscale image represented by the discrete matrix to the Gaussian distribution. However, the above approach converts discrete variables into continuous variables for processing, which easily introduces irrelevant noise data and affects the learning of the distribution. Another more reasonable approach should be to simulate the forward and reverse processes of the diffusion model into a process in which the value of each bit in the discrete matrix is ​​converted to another value with a certain probability.

[0024] It should be pointed out that for a certain bit of the discrete matrix of the segmented image , the forward transition probability matrix can be expressed as , the forward process can be expressed as ,Similar to DDPM, the forward process can also be completed in one step under the Markov chain condition, Among them, Q is the probability transfer matrix of the forward diffusion process, which indicates the probability of converting the value of each pixel of the segmented image to other values; Refers to the probability transfer matrix of pixel ij at a certain time step; Cat represents Categorical Distribution, i.e., category distribution. In this discrete diffusion process, a variety of different discrete distributions can be used as noise similar to Gaussian noise in DDPM to add to the original input data. In this embodiment, discrete uniform distribution is used as noise. In this case: ; Among them, K represents the number of defect types, 1 represents a column vector of all 1s, and the forward process can be expressed as It should be noted that for Medium Parameters p .

[0025] For the reverse process of the diffusion model, this embodiment directly predicts the final target, that is, the final segmented image, at each sampling step, and then collects the output of each reverse sampling step and averages it as the final output. .in, Represents the final prediction result of the model. Indicates that at time step t Prediction of segmentation results.

[0026] In this embodiment, a preset attention module is added to the Unet downsampling structure to obtain an adjusted downsampling structure, and the adjusted downsampling structure and the noise encoder in the chip defect detection model to be trained are used to extract features from the noisy image to obtain the current noise features. In other words, the Unet downsampling structure is used to extract features from the input noise image, that is, the segmentation result image that has been corrupted by random uniformly distributed noise. In this embodiment, the attention module (that is, the attention mechanism module) in the Transformer (that is, the deep learning model based on the self-attention mechanism) is added to each model block of the downsampling to enhance the ability of the downsampling module to extract features from the noisy image, and the Transformer is used to extract features from the noisy image to obtain the current noise features.

[0027] Step S13: Use the image encoder in the chip defect detection model to be trained to perform feature extraction on the historical RGB original image to obtain current image features, and combine the current noise features with the current image features to obtain current feature combination information.

[0028] In this embodiment, the pyramid vision transformer is used as the backbone network for extracting image features, and the image encoder in the chip defect detection model to be trained is used to extract features from the historical RGB original image to obtain corresponding image output features, and the image output features are channel-reduced to obtain current image features. Specifically, a pre-trained model PVT (Pyramid Vision Transformer) is used as the image extraction backbone network, which receives an optical image of a defective chip and outputs four-level features of the image. The feature sizes are 512, 320, 128, and 64, respectively. In order to reduce the amount of calculation, the above four features are first channel-reduced to 64, respectively. In the Unet downsampling process, the four-level features output by the PVT are combined with the output features of the four levels in the downsampling process by channels, and then input into the next downsampling module. The downsampling module finally outputs a 64-dimensional vector , and the output features of each level , then The input is fed into a 6-layer Transformer block for feature enhancement.

[0029] Step S14: Use the attention module for performing step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained to process the current feature combination information to obtain the current chip defect prediction image and the current noisy image.

[0030] In this embodiment, during the upsampling stage, for the output features of each upsampling level, , are related to downsampling the output features of each level The channels are combined and then input into the linear layer of the prediction result to obtain the phased prediction result of each layer. The prediction result is a 01 matrix, which shows the location of all defects in the input image, but does not show the type of defects. For each level of upsampling module, there is an attention module for gradual feature refinement. The input of this module is the input feature F of this layer. The phased prediction result P of the input feature of this layer is the current chip defect prediction image.

[0031] In this embodiment, the attention module for performing step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained is used, and the chip defect position feature is filtered from the current feature combination information by element-by-element multiplication to obtain the corresponding channel attention enhancement feature; the current chip defect prediction image and the current noisy image are determined based on the channel attention enhancement feature. The flowchart of the attention module for performing step-by-step feature refinement is shown in FIG. Figure 2As shown, combining the above input features and prediction results, the input features are screened by element-by-element multiplication. Only the features of the positions where defects were predicted last time will participate in the upsampling process of this layer. The features of the positions predicted as background will be set to 0. After the multiplication, the enhanced features are obtained. ,Will Input into the attention enhancement module that combines spatial and channel attention. Specifically, for the input feature F, F consists of three dimensions C, H, and W. Among them, C is the channel, H and W are the height and width respectively. First, the shape of F is reshaped to obtain three feature matrices Q, K, and V. The matrix shape is C*(H*W), and then the matrix Q and K are multiplied, and then the C*C channel attention matrix X is obtained through the softmax function: ; in, Represents the influence of the j-th channel on the i-th channel, represents the Q of the i-th channel, Represents K of the jth channel. Then multiply X and V to get the channel attention enhancement feature , , in order to filter out semantic information that is useful for identifying chip defect targets. By refining the prediction results layer by layer, the edge learning of the pixel features where the defect is located is enhanced.

[0032] Step S15: extracting features of the current noisy image by using the noise encoder to obtain new current noise features, and extracting features of the current chip defect prediction image by using the extraction module for cyclic feature extraction to obtain new current image features.

[0033] In this embodiment, the extraction module for cyclic feature extraction is used to extract features from the current chip defect prediction image to obtain new current image features, including: extracting features from the current chip defect prediction image using the extraction module for cyclic feature extraction to obtain predicted image feature information; filtering the predicted image feature information by element-by-element multiplication to obtain a filtering result, and determining new current image features based on the filtering result. The flowchart of the extraction module for cyclic feature extraction is shown in FIG. Figure 3 As shown, in the reverse sampling stage of the diffusion model, a segmentation result is output for each sampling time period.

[0034] The segmentation results will be combined with the output of the image feature extraction backbone Perform element-wise multiplication to get , and then extract the output of the backbone by combining 1-P with the image feature Perform element-wise multiplication to get , and then each level and Channel combination ,Will As the feature of the image, it is input into the Unet of the diffusion model.

[0035] Step S16, jump to the step of combining the current noise feature and the current image feature to obtain the current feature combination information, until the preset training end condition is met, and output the corresponding trained chip defect detection model.

[0036] In this embodiment, after satisfying the preset training end condition and outputting the corresponding trained chip defect detection model, the following may also be included: using the cross entropy loss function to calculate the image pixels in each chip defect prediction image to obtain the loss value of the image pixel; sorting the image pixels based on the size of the loss value, and removing the image pixels whose loss value is less than the preset threshold from the sorting result. Specifically, using , as the loss function of the chip defect detection model after training, where It is a cross entropy loss function combined with online hard example mining. It is a convex Expanded, P is the model prediction, GT represents the true value. After the cross entropy loss function calculates the pixel loss for each pixel, the image pixels are sorted from large to small based on the loss size, and the pixels with small losses are removed and only the pixels with large losses are retained. In summary, the algorithm flow chart of this application is as follows Figure 4 shown.

[0037] Step S17: When the current RGB original image corresponding to the chip to be detected is obtained, the current RGB original image is input into the trained chip defect detection model to obtain the defect detection result of the chip to be detected output by the trained chip defect detection model.

[0038] In this embodiment, the trained chip defect detection model is tested. It can be understood that the trained chip defect detection model has highly accurate and efficient image recognition and analysis capabilities, and the model can analyze the input RGB original image corresponding to the chip to be detected. Specifically, the key features in the RGB original image are extracted, and the key features include but are not limited to color changes, texture differences, irregular shapes, etc., and the key features are intelligently matched and compared to obtain the defect detection results of the chip to be detected.

[0039] As can be seen from the above, this embodiment first obtains a historical chip defect data set; the historical chip defect data set includes a historical RGB original image obtained after imaging the chip and a corresponding historical segmented image reflecting the chip defect area; the historical segmented image is denoised to obtain a noisy image, and the noise encoder in the chip defect detection model to be trained is used to extract features from the noisy image to obtain the current noise feature; the chip defect detection model to be trained is a model built based on the diffusion model; the image encoder in the chip defect detection model to be trained is used to extract features from the historical RGB original image to obtain the current image feature, and the current noise feature and the current image feature are combined to obtain the current feature combination information; the preset decoder in the chip defect detection model to be trained is used to perform step-by-step The feature-refined attention module processes the current feature combination information to obtain the current chip defect prediction image and the current noisy image; performs feature extraction on the current noisy image through the noise encoder to obtain a new current noise feature, and uses the extraction module for cyclic feature extraction to extract features on the current chip defect prediction image to obtain a new current image feature; jumps to the step of combining the current noise feature with the current image feature to obtain the current feature combination information until the preset training end condition is met, and outputs the corresponding trained chip defect detection model; when the current RGB original image corresponding to the chip to be detected is obtained, the current RGB original image is input into the trained chip defect detection model to obtain the defect detection result of the chip to be detected output by the trained chip defect detection model. As can be seen from the above, this embodiment can perform more refined distribution modeling for chip defects with a large number of categories and similar optical forms, such as color and shape, so that the model has better generalization ability when tested in an environment with variable downstream tasks; the extraction module for cyclic feature extraction and the attention module for gradual feature refinement enhance the diffusion model's processing ability for input features, thereby further improving the diffusion model's generalization ability on chip defect data and improving the segmentation ability; finally, the performance test of the trained chip defect detection model is performed to ensure the accuracy of the model training. In this way, the detection of chip defects is achieved accurately and flexibly.

[0040] Accordingly, see Figure 5 As shown, the embodiment of the present application also provides a chip defect detection device based on a diffusion model, which may include: The data set acquisition module 11 is used to acquire a historical chip defect data set; the historical chip defect data set includes a historical RGB original image obtained after imaging the chip and a corresponding historical segmented image reflecting the chip defect area; A feature acquisition module 12 is used to perform noise processing on the historical segmented image to obtain a noisy image, and use a noise encoder in the chip defect detection model to be trained to perform feature extraction on the noisy image to obtain a current noise feature; the chip defect detection model to be trained is a model built based on a diffusion model; A feature combining module 13 is used to extract features of the historical RGB original image using the image encoder in the chip defect detection model to be trained to obtain current image features, and combine the current noise features with the current image features to obtain current feature combination information; An image acquisition module 14 is used to process the current feature combination information using an attention module for performing step-by-step feature refinement in a preset decoder in the chip defect detection model to be trained, so as to obtain a current chip defect prediction image and a current noisy image; A feature extraction module 15, used to extract features of the current noisy image by the noise encoder to obtain new current noise features, and to extract features of the current chip defect prediction image by using the extraction module for cyclic feature extraction to obtain new current image features; The model output module 16 is used to jump to the step of combining the current noise feature with the current image feature to obtain the current feature combination information until the preset training end condition is met, and output the corresponding trained chip defect detection model; The image detection module 17 is used to input the current RGB original image corresponding to the chip to be detected into the trained chip defect detection model when the current RGB original image corresponding to the chip to be detected is obtained, so as to obtain the defect detection result of the chip to be detected output by the trained chip defect detection model.

[0041] As can be seen from the above, the present application can perform more refined distribution modeling for chip defects with a large number of categories and similar optical forms, such as color and shape, so that the model has better generalization ability when tested in an environment with variable downstream tasks; the extraction module for cyclic feature extraction and the attention module for gradual feature refinement enhance the diffusion model's processing ability for input features, thereby further improving the diffusion model's generalization ability on chip defect data and improving the segmentation ability; finally, the performance test of the trained chip defect detection model is performed to ensure the accuracy of the model training. In this way, the detection of chip defects is achieved accurately and flexibly.

[0042] In some specific implementations, the feature acquisition module 12 may include: An image noise adding unit, used for performing noise adding processing on the historical segmented image based on discrete uniform distribution and by using a preset algorithm to obtain a noisy image; The feature extraction submodule is used to utilize the noise encoder in the chip defect detection model to be trained and to perform feature extraction on the noisy image based on the Unet downsampling structure to obtain the current noise feature.

[0043] In some specific implementations, the feature extraction submodule may include: A feature extraction unit is used to add a preset attention module to the Unet downsampling structure to obtain an adjusted downsampling structure, and use the adjusted downsampling structure and the noise encoder in the chip defect detection model to be trained to perform feature extraction on the noisy image to obtain a current noise feature.

[0044] In some specific implementations, the feature combination module 13 may include: A channel reduction unit is used to use the pyramid vision Transformer as a backbone network for extracting image features, and use the image encoder in the chip defect detection model to be trained to extract features from the historical RGB original image to obtain corresponding image output features, and perform channel reduction on the image output features to obtain current image features.

[0045] In some specific implementations, the image acquisition module 14 may include: An information screening unit, used to utilize an attention module for performing step-by-step feature refinement in a preset decoder in the chip defect detection model to be trained, and to screen chip defect position features from current feature combination information by an element-by-element multiplication method to obtain corresponding channel attention enhancement features; An image determination unit is used to determine a current chip defect prediction image and a current noisy image based on the channel attention enhancement feature.

[0046] In some specific implementations, the feature extraction module 15 may include: A feature extraction unit, used for performing feature extraction on the current chip defect prediction image using an extraction module for cyclic feature extraction to obtain prediction image feature information; The feature screening unit is used to screen the predicted image feature information by element-by-element multiplication to obtain a screening result, and determine a new current image feature based on the screening result.

[0047] In some specific implementations, the chip defect detection device based on the diffusion model may further include: A loss value calculation unit, used to calculate the image pixels in each chip defect prediction image using a cross entropy loss function to obtain a loss value of the image pixel; The loss value sorting unit is used to sort the image pixels based on the size of the loss value, and eliminate the image pixels whose loss value is less than a preset threshold from the sorting result.

[0048] Furthermore, the present application also discloses an electronic device. Figure 6 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input and output interface 25 and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the chip defect detection method based on the diffusion model disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment can specifically be an electronic computer.

[0049] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0050] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0051] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the chip defect detection method based on the diffusion model performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0052] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed chip defect detection method based on a diffusion model is implemented. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, and no further description will be given here.

[0053] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0054] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0055] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0056] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0057] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A chip defect detection method based on a diffusion model, characterized in that: include: Acquire a historical chip defect data set; the historical chip defect data set includes a historical RGB original image obtained after imaging the chip and a corresponding historical segmented image reflecting the chip defect area; Noising the historical segmented image to obtain a noisy image, and extracting features from the noisy image using a noise encoder in a chip defect detection model to be trained to obtain current noise features; the chip defect detection model to be trained is a model constructed based on a diffusion model; Using the image encoder in the chip defect detection model to be trained to perform feature extraction on the historical RGB original image to obtain current image features, and combining the current noise features with the current image features to obtain current feature combination information; Using the attention module for performing step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained to process the current feature combination information, so as to obtain the current chip defect prediction image and the current noisy image; Performing feature extraction on the current noisy image by the noise encoder to obtain a new current noise feature, and performing feature extraction on the current chip defect prediction image by using an extraction module for cyclic feature extraction to obtain a new current image feature; Jump to the step of combining the current noise feature with the current image feature to obtain current feature combination information, until the preset training end condition is met, and output the corresponding trained chip defect detection model; When the current RGB original image corresponding to the chip to be inspected is obtained, the current RGB original image is input into the trained chip defect detection model to obtain the defect detection result of the chip to be inspected output by the trained chip defect detection model.

2. The chip defect detection method based on the diffusion model according to claim 1, characterized in that: The step of performing noise processing on the historical segmented image to obtain a noisy image, and performing feature extraction on the noisy image using a noise encoder in the chip defect detection model to be trained to obtain a current noise feature, includes: Based on discrete uniform distribution and using a preset algorithm, the historical segmented image is subjected to noise processing to obtain a noisy image; The noise encoder in the chip defect detection model to be trained is used to extract features of the noisy image based on the Unet downsampling structure to obtain the current noise features.

3. The chip defect detection method based on diffusion model according to claim 2, characterized in that: The method of using the noise encoder in the chip defect detection model to be trained and extracting features from the noisy image based on the Unet downsampling structure to obtain the current noise features includes: A preset attention module is added to the Unet downsampling structure to obtain an adjusted downsampling structure, and the adjusted downsampling structure and the noise encoder in the chip defect detection model to be trained are used to perform feature extraction on the noisy image to obtain the current noise feature.

4. The chip defect detection method based on diffusion model according to claim 1, characterized in that: The step of extracting features from the historical RGB original image using the image encoder in the chip defect detection model to be trained to obtain current image features includes: The pyramid vision Transformer is used as the backbone network for extracting image features, and the image encoder in the chip defect detection model to be trained is used to extract features of the historical RGB original image to obtain corresponding image output features, and the image output features are channel-reduced to obtain current image features.

5. The chip defect detection method based on diffusion model according to claim 1, characterized in that: The method of using the attention module for performing step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained to process the current feature combination information to obtain the current chip defect prediction image and the current noisy image includes: Utilize the attention module for performing step-by-step feature refinement in the preset decoder in the chip defect detection model to be trained, and filter the chip defect position features from the current feature combination information by element-by-element multiplication method to obtain the corresponding channel attention enhancement features; A current chip defect prediction image and a current noisy image are determined based on the channel attention enhancement feature.

6. The chip defect detection method based on diffusion model according to claim 1, characterized in that: The extracting module for cyclic feature extraction is used to extract features from the current chip defect prediction image to obtain new current image features, including: Using an extraction module for cyclic feature extraction to extract features of the current chip defect prediction image to obtain prediction image feature information; The predicted image feature information is filtered by an element-by-element multiplication method to obtain a filtering result, and a new current image feature is determined based on the filtering result.

7. The chip defect detection method based on the diffusion model according to any one of claims 1 to 6, characterized in that: Also includes: Using a cross entropy loss function to calculate image pixels in each chip defect prediction image to obtain a loss value of the image pixel; The image pixels are sorted based on the size of the loss value, and image pixels whose loss value is less than a preset threshold are eliminated from the sorting result.

8. A chip defect detection device based on a diffusion model, characterized in that: include: A data set acquisition module is used to acquire a historical chip defect data set; the historical chip defect data set includes a historical RGB original image obtained after imaging the chip and a corresponding historical segmented image reflecting the chip defect area; A feature acquisition module, used for performing noise processing on the historical segmented image to obtain a noisy image, and performing feature extraction on the noisy image using a noise encoder in a chip defect detection model to be trained to obtain a current noise feature; the chip defect detection model to be trained is a model constructed based on a diffusion model; A feature combination module is used to extract features of the historical RGB original image using the image encoder in the chip defect detection model to be trained to obtain current image features, and combine the current noise features with the current image features to obtain current feature combination information; An image acquisition module is used to process the current feature combination information using an attention module for performing step-by-step feature refinement in a preset decoder in the chip defect detection model to be trained, so as to obtain a current chip defect prediction image and a current noisy image; A feature extraction module, used to extract features of the current noisy image by the noise encoder to obtain new current noise features, and to extract features of the current chip defect prediction image by using the extraction module for cyclic feature extraction to obtain new current image features; A model output module, used to jump to the step of combining the current noise feature with the current image feature to obtain current feature combination information, until a preset training end condition is met, and output a corresponding trained chip defect detection model; The image detection module is used to input the current RGB original image corresponding to the chip to be detected into the trained chip defect detection model when the current RGB original image corresponding to the chip to be detected is obtained, so as to obtain the defect detection result of the chip to be detected output by the trained chip defect detection model.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the chip defect detection method based on the diffusion model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein, when the computer program is executed by a processor, the chip defect detection method based on the diffusion model as described in any one of claims 1 to 7 is implemented.

Images