Chip defect detection method and device and medium

Abstract

The invention discloses a chip defect detection method, chip defect detection equipment and a medium. The method comprises the following steps: acquiring a to-be-detected chip image; inputting the obtained chip image into a semantic segmentation model based on a diffusion model, and outputting a defect detection mask of the chip; wherein the semantic segmentation model comprises an image encoder, an attention fusion module and an image decoder; the image encoder is used for extracting features of the chip image, and the attention fusion module is used for fusing the extracted features of the chip image with preset noise features to obtain fused features; the image decoder is used for generating a defect detection mask of the chip according to the fusion features; in addition, before the semantic segmentation model is trained, the image encoder is subjected to independent denoising pre-training, so that the performance of the image encoder is improved. According to the method, a semantic segmentation framework of a diffusion model is adopted, a special network structure is designed, and the model can better learn defect boundary information from an original image. The method can be widely applied to the field of chip defect detection.

Description

Technical Field

[0001] The present invention relates to the field of chip defect detection, and in particular to a chip defect detection method, equipment and medium. Background Art

[0002] In the field of chip defect detection, chip images typically exhibit high noise and low contrast, making defect boundaries visually unclear, which makes the defect detection task particularly complex. These characteristics of high noise, blurred boundaries, and high similarity between foreground and background make the technical challenges of chip defect detection similar to those of camouflaged object detection. Existing technical solutions draw inspiration from camouflage detection technology and improve it to adapt to the special needs of chip defect detection. Currently, research on camouflaged object detection has proposed a variety of technical strategies to address the challenges posed by camouflage characteristics, mainly including the following categories:

[0003] 1) Multi-stream framework: This type of approach explicitly learns multi-source representations from multiple input streams, combining different information channels to capture object details and context. For example, some methods combine edge detection streams and texture feature streams to form multi-path parallel feature extraction, enabling more accurate segmentation and identification of camouflaged objects.

[0004] 2) Bottom-up and top-down frameworks: These methods use deep features to gradually enhance shallow features in a single forward propagation process, forming a multi-level feature fusion. Bottom-up methods start from low-level features and gradually build semantic information upwards, while top-down methods gradually pass detailed information from high-level features, thereby enhancing the detection of boundaries and subtle features.

[0005] 3) Branching Framework: This approach employs a single-input, multiple-output architecture, where the main branch performs the core segmentation task, while auxiliary branches handle related tasks such as boundary detection, classification, or saliency detection. This design helps the model enhance its ability to discern camouflaged objects by leveraging auxiliary information during segmentation. For example, the additional boundary detection branch can help the model more accurately locate and segment the edges of camouflaged objects.

[0006] While these methods have made significant progress in camouflage detection tasks, existing methods still have limitations in detecting subtle boundaries due to the high similarity between the contours of camouflaged objects and the background, resulting in reduced segmentation accuracy. This problem is particularly prominent for chip defect detection, as unclear defect boundaries can affect the area estimation and positioning of defective regions, leading to missed detections and misjudgments of products. Furthermore, traditional image encoder parameters are typically not fine-tuned for downstream tasks, which results in low learning efficiency in the early stages of training and poor model performance in feature refinement and boundary recognition. Traditional segmentation paradigms based on pixel-level probability can also produce overconfident erroneous predictions, increasing the false detection rate and limiting the reliability of detection systems in production environments. Summary of the Invention

[0007] In order to at least partially solve one of the technical problems existing in the prior art, the present invention aims to provide a chip defect detection method, device and medium based on diffusion model semantic segmentation.

[0008] The first technical solution adopted by the present invention is:

[0009] A chip defect detection method comprises the following steps:

[0010] Acquire an image of the chip to be tested;

[0011] The chip image is input into the semantic segmentation model based on the diffusion model, and the chip defect detection mask is output;

[0012] The semantic segmentation model includes an image encoder, an attention fusion module, and an image decoder; the image encoder is used to extract chip image features, the attention fusion module is used to fuse the extracted chip image features with preset noise features to obtain fused features; the image decoder is used to generate a chip defect detection mask based on the fused features;

[0013] In addition, before training the semantic segmentation model, the image encoder is individually pre-trained for denoising to improve the image encoder performance.

[0014] Furthermore, the image encoder first performs unsupervised denoising pre-training through a diffusion model to learn the deep structure and features of the image. This pre-training process denoises the noisy image through the diffusion model, so that the encoder can effectively extract the potential features of the original image;

[0015] The pre-trained image encoder is used in the semantic segmentation task of the diffusion model as part of the semantic segmentation model based on the diffusion model. In the semantic segmentation stage, the original image features extracted by the image encoder are used for supervised semantic segmentation to finally complete the image segmentation task.

[0016] Furthermore, the denoising autoencoder pre-training EDeP method is used to pre-train the image encoder for denoising.

[0017] Furthermore, the training process of the semantic segmentation model includes:

[0018] Obtain the chip image and the annotated mask image to form an image pair as the input of the model;

[0019] Input the chip image into the image encoder to extract conditional image features;

[0020] Add noise to the mask image to obtain the noise image features of the mask after adding noise;

[0021] The conditional image features and the mask noise image features are input into the attention fusion module for fusion to obtain the fusion features;

[0022] Input the fused features into the image decoder to generate the chip’s defect detection mask;

[0023] Calculate the loss based on the generated defect detection mask and the preset chip defect mask, and optimize the model parameters;

[0024] During the inference phase, the semantic segmentation model generates predictions inversely through the learned diffusion process, adjusting the noisy Gaussian distribution to the learned distribution guided by the test image.

[0025] Furthermore, the attention fusion module works as follows:

[0026] Generate query Q by linearly projecting the mask noise image feature D D , key K D Sum V D ; Generate intermediate feature P by linear projection of chip image feature F F Sum V F , the expression is as follows:

[0027]

[0028] Where, is the query Q of the mask noise image D , key K D Sum V D The respective weight parameter matrices, It is the intermediate feature P of the chip image feature F Sum V F The weight parameter matrix of

[0029] The feature map fusion method is as follows:

[0030]

[0031] O I =M att1 ·M att2 ·(V D +V F )

[0032] Where M att1 and M att2 Represents Q D -P F and K D -P F Attention map of O I Represents the final generated cross-attention fusion feature.

[0033] Furthermore, the image decoder is implemented using a simple stack of six layers of deformable attention transformers.

[0034] Furthermore, structural damage is introduced during the forward diffusion process:

[0035] Randomly destroy the contour of the true value and add Gaussian noise to enable the diffusion model to learn in a biased environment and recover the correct mask from the noise mask;

[0036] In the inference stage, the model only inputs the chip image and obtains the noise features by randomly sampling Gaussian noise.

[0037] Furthermore, an incremental denoising inference sampling strategy is adopted to perform T-step denoising on the noise samples drawn from the standard normal distribution. The specific inference process includes the following steps:

[0038] 1) Initial sample generation

[0039] Draw an initial sample x from a standard normal distribution T , as the input sample of the diffusion model; the initial sample contains pure Gaussian noise and is the starting point of the denoising process;

[0040] 2) Stepwise denoising

[0041] For each time step t (decreasing from T to 1), the sample x of the next time step is generated by the following formula t-1 :

[0042]

[0043] Where x t is the sample of the current time step, is the denoising result predicted at each time step, α t and is the time-related parameter preset in the model, β t represents the weight of the noise, σ t z represents the Gaussian noise introduced during the sampling process (when t = 1, z = 0);

[0044] At each step, the model is based on the current sample x t And the predicted denoising results Generate the next step sample x t-1 , through the diffusion model step by step denoising, eliminate the noise and generate a clear denoising result, and the denoising result of each time step Record it for later processing;

[0045] 3) Consensus Time Integration (CTE)

[0046] By integrating the prediction results of different time steps, the accuracy and robustness of the mask can be improved:

[0047] 3.1) Generate a binary mask for each time step:

[0048] For each sampling time step t, the denoised predicted image result Indicated as P t , and perform adaptive thresholding to generate a binary mask

[0049] 3.2) Voting and integration:

[0050] Binary mask for all time steps Voting and combining all prediction results P t The final mask P is calculated by taking the mean of emb , the mathematical expression is:

[0051]

[0052] Where T represents the sampling time step, mean(P t ) represents the prediction results P of all time steps t The mean of .

[0053] The second technical solution adopted by the present invention is:

[0054] An electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, the code set or the instruction set is loaded and executed by the processor to implement a chip defect detection method as described above.

[0055] The third technical solution adopted by the present invention is:

[0056] A computer-readable storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by a processor to implement a chip defect detection method as described above.

[0057] The fourth technical solution adopted by the present invention is:

[0058] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, so that the computer device performs the above method.

[0059] The present invention utilizes a semantic segmentation framework based on a diffusion model and designs a specialized network structure, enabling the model to better learn defect boundary information from raw images. Furthermore, the introduction of a structural destruction mechanism significantly improves the clarity and accuracy of defect boundaries, successfully reducing missed detections and significantly improving chip defect detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0061] Figure 1 is a structural diagram of a denoising autoencoder according to an embodiment of the present invention;

[0062] Figure 2 is an architectural diagram of a semantic segmentation model based on a diffusion model in an embodiment of the present invention;

[0063] Figure 3 2 is an architectural diagram of the attention fusion module IAM in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided for ease of explanation only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0065] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on the present invention.

[0066] In the description of the present invention, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0067] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0068] Explanation of terms:

[0069] EDeP: Encoder denoising pre-training aims to pre-train the encoder through the denoising task to help the encoder better extract original image features.

[0070] Diffusion: Diffusion refers to the process of introducing noise into the model and gradually removing it in a step-by-step iterative manner to generate high-quality samples or images.

[0071] Backbone: backbone network.

[0072] Upernet: A multi-scale feature map fusion module.

[0073] Injection Attention Module: Attention fusion module.

[0074] In general, the existing technical solutions have the following technical problems: 1) Unclear defect boundary detection: Chip images are characterized by high noise and low contrast, which makes it difficult to distinguish defect boundaries. This characteristic makes the area estimation and positioning during the defect detection process inaccurate, increasing the risk of missed detection and misjudgment. 2) Model false detection problem: Since the segmentation model relies on pixel-level probability for prediction, it is prone to overconfident erroneous predictions, resulting in a high false detection rate, which limits the reliability of the detection system in practical applications. 3) Low image encoder initialization efficiency: Image encoder parameters are usually not fine-tuned by downstream tasks, resulting in low learning efficiency in the early stages of training, affecting the model's performance in feature refinement and boundary detection.

[0075] To address these issues, this paper introduces a new paradigm for spoofing detection: it considers chip defect detection as a conditional mask generation task based on a diffusion model. The diffusion model generates results through a stepwise denoising approach, demonstrating superior generative capabilities and robust conditional awareness, enabling more effective chip defect detection. Following this new paradigm, this paper proposes a framework that leverages the denoising process of the diffusion model, using the image as an auxiliary condition to gradually eliminate the deviation between the initial noise and the true value.

[0076] Example 1

[0077] This embodiment provides a chip defect detection method based on diffusion model semantic segmentation. The chip defect detection is regarded as a conditional mask generation task based on the diffusion model, including the following steps:

[0078] S1. Obtain an image of the chip to be tested;

[0079] S2. Input the obtained chip image into the semantic segmentation model based on the diffusion model, and output the chip defect detection mask;

[0080] The semantic segmentation model includes an image encoder, an attention fusion module, and an image decoder; the image encoder is used to extract chip image features, the attention fusion module is used to fuse the extracted chip image features with preset noise features to obtain fused features; the image decoder is used to generate a chip defect detection mask based on the fused features;

[0081] In addition, before training the semantic segmentation model, the image encoder is individually pre-trained for denoising to improve the image encoder performance.

[0082] The operating principle of this embodiment is as follows: During denoising pre-training of the image encoder, the chip image is noisy and fed into the image encoder for denoising. This chip denoising pre-training task improves the image encoder's performance. Specifically, diffusion is first used on the image encoder to perform denoising to better initialize the encoder's parameters. The pre-trained image encoder is then used as part of the diffusion model to extract chip-conditioned image features.

[0083] During semantic segmentation, the chip image is simply fed into the diffusion model as a conditional image and does not require noise. In other words, the original chip image only needs to be fed into the image encoder to extract conditional image features. However, it is the mask image encoding that requires noise. During semantic segmentation model training, the mask image needs to be noised to obtain the corresponding noise features. These noise features are then fused with the image features extracted by the image encoder. The fused features are then fed into the image decoder to predict and generate the chip's defect detection mask.

[0084] Compared with the complex diffusion perception models of the past, the framework of this embodiment implements a decoupled design of the image encoder and decoder. That is, in the multi-step denoising of inference sampling, the image encoder only needs to run once, and the diffusion process is only executed in the lightweight decoding head. In some embodiments, by introducing the unsupervised pre-training EDep method of the denoising autoencoder, the initialization parameters of the image encoder are optimized, and the unlabeled data is fully utilized to improve the efficiency of image feature extraction, thereby significantly improving the performance of the image encoder in feature refinement and boundary recognition. With this efficient design, the method of this embodiment can be easily applied to chip defect detection tasks, significantly improving detection effect and efficiency.

[0085] As an implementation, during the model training phase, Gaussian noise is added to the encoded true values through a controlled noise schedule to generate noisy images. These noise maps are then fused with the conditional features extracted by the image encoder (e.g., PVTv2) in an attention fusion module (IAM). Finally, the fused features are fed into a lightweight decoder (i.e., image decoder) to generate noise-free predictions. During the inference phase, this diffusion framework generates predictions by inversely working from the learned diffusion process, adjusting the noisy Gaussian distribution to a learned distribution guided by the test image.

[0086] The following is a detailed description of each module in the model with reference to the accompanying drawings and specific implementation methods.

[0087] (1) Pre-training EDep method for denoising autoencoders

[0088] Since the amount of unlabeled chip image data for chip defect detection is large, in order to improve the feature extraction efficiency of the image encoder, an unsupervised pre-training method can be used to fully utilize the unlabeled data. This embodiment introduces the unsupervised pre-training EDep method, which adopts the principle of denoising autoencoder. The structure of the denoising autoencoder is as follows: Figure 1 As shown, it includes a backbone network Backbone and a multi-scale feature map fusion unit.

[0089] Specifically, the backbone network is responsible for gradually extracting features of each level of the image from low to high levels, while the multi-scale feature map fusion unit further enhances the expressive power of the features by integrating information from different scales. In this embodiment, the image encoder first performs unsupervised denoising pre-training through a diffusion model to learn the deep structure and features of the image. The pre-training process denoises the noisy image through a diffusion model, so that the encoder can effectively extract the potential features of the original image. After the training is completed, the pre-trained image encoder is applied to the semantic segmentation task of the diffusion model. In the semantic segmentation stage, the image encoder, as part of the diffusion model, uses the extracted original image features to perform supervised semantic segmentation, and finally completes the image segmentation task.

[0090] For example, a noisy image is passed through an image encoder backbone network (Backbone) to extract a hidden layer representation. This hidden layer representation is then restored to a noise-free image using a multi-scale feature map fusion unit (Upernet). This eliminates the need for labeled data, requiring only the chip image. Through this process, robust feature representations of the chip image can be learned by restoring the noisy data to its original state. The image encoder is pre-trained using the unsupervised pre-training (EDep) method described above.

[0091] By enabling the image encoder to recover data from noise, the image encoder can learn deeper features from unlabeled chip images, more effectively initialize parameters, and improve the encoder's ability to extract chip image features.

[0092] (2) Design of semantic segmentation model based on diffusion model

[0093] In this embodiment, the model framework includes an image encoder, an attention fusion module (IAM), and an image decoder, which respectively encode the conditional image and decode the mask for denoising. Furthermore, this embodiment designs a specialized learning strategy during training and inference to ensure that technical issues are addressed, including noisy image encoding design, structural destruction, and consistent temporal aggregation.

[0094] The specific model structure is as follows Figure 2As shown in Figure 1, the model structure mainly consists of an image encoder (Backbone, Uppernet), an attention fusion module (IAM), and an image decoder (Map Decoder). The mask image is encoded by the embedding layer and then forward-noised by diffusion to obtain the noisy mask noise feature (Encoding & Corruption); the chip image is input into the pre-trained image decoder to extract robust features, and is input into the IAM module together with the encoded noise mask feature to fuse the chip image and mask noise image features. Finally, it is input into the deformable decoder (MapDecoder) to predict the noise, and further deduce the denoised image x. t-1 .

[0095] For example, see Figure 2 The semantic segmentation model of this embodiment uses the chip image as a conditional image to generate the final segmentation mask image. The specific training process is as follows:

[0096] a) Feature extraction process

[0097] a.1) Input the chip image into the image decoder to obtain the chip image features.

[0098] a.2) Add noise to the encoded mask image (manually annotated chip defect mask image gt) to obtain noise mask image features.

[0099] b) Feature Fusion

[0100] The chip image features and noise mask image features are input into the attention module for fusion.

[0101] c) Feature decoding

[0102] The fused features are decoded to obtain the predicted chip defect mask image, and the manually annotated chip defect mask is used as ground truth for supervision.

[0103] 2.1) Image Encoder

[0104] The image encoder consists of a backbone network and a multi-scale feature map fusion module (Upernet). For example, this embodiment uses the PVTv2 backbone network as the backbone of the image encoder, and also uses a conventional multi-scale fusion module of the Upernet to perform multi-scale feature fusion.

[0105] It's important to note that the image encoder modules (Backbone, Upernet) here are the same denoising autoencoders pre-trained in the pre-training scheme. The image encoder uses PVTv2 as its backbone network, combined with the feature fusion module Upernet. This is effectively initialized with unlabeled chip defect data to improve performance and make chip image features more effective in guiding the denoising of the noise mask.

[0106] In this embodiment, in semantic segmentation, the chip image is used as a conditional image, and no noise is added to the chip image. The chip image (original image) is used as a conditional image and input into the image encoder to directly extract features.

[0107] 2.2) Injection Attention Module

[0108] See also Figure 3 The attention fusion module performs attention calculation fusion on the chip image features and the mask noise image features, and introduces the texture and position information of the original features into the noise prediction process. Specifically, the chip image feature F is the output feature map of the image encoder module, and the mask noise image feature D comes from the class embedding layer encoding vector after adding noise. The feature map D is linearly projected to generate the query Q D , key K D Sum V D The feature F generates the intermediate feature P through linear projection F Sum V F Among them, the chip image feature F only generates an intermediate feature P for similarity comparison with D F , no query or key is generated for similarity comparison. The specific expression is as follows:

[0109]

[0110] in, d is the dimension.

[0111] Therefore, the IAM action is defined as follows:

[0112]

[0113] O I =M att1 ·M att2 ·(V D +V F )

[0114] Among them, M att1 and M att2 Represents Q D -P F and KD -P F Attention map. Represents the final generated cross-attention fusion feature.

[0115] 2.3) Image Decoder

[0116] This embodiment of the present invention uses a simple stacked six-layer deformable attention transformer as a decoder. Unlike the previous parameter-intensive U-net denoising network, the deformable decoder of this embodiment is lightweight and compact, and can effectively reuse shared parameters in the multi-step back diffusion process.

[0117] (3) Model training strategy

[0118] 3.1) Noisy Image Coding

[0119] Using a class embedding encoding strategy, discrete labels (i.e., class labels in the mask image, such as 0, 1, 2, 3, where 0 is background class and 1 is collapse class, etc.) are projected into a high-dimensional continuous space through a learnable embedding layer. The sigmoid function is then used for normalization, normalizing the range of the encoded labels and scaling them to [-scale, +scale]. It is worth noting that the scale factor controls the signal-to-noise ratio (SNR), which is an important hyperparameter of the diffusion model. Its formula is as follows:

[0120] map_enc=(σ(encoding(maps))×2-1)×scale

[0121] 3.2) Structural damage

[0122] Existing diffusion models generate noisy masks directly from the ground truth (GT) through pixel-level corruption. This causes the model to incorrectly assume that the contours recovered from the noisy mask are always accurate, making it impossible to correct for bias. However, this assumption does not necessarily hold during inference sampling. To address this issue, this embodiment introduces structural corruption during forward diffusion. Specifically, we randomly corrupt the GT contours and add Gaussian noise, enabling the model to learn how to recover the correct mask from the noisy mask despite the bias.

[0123] As an optional implementation, the method of randomly destroying the GT contour includes: first deleting some vertices on the contour to reduce its continuity; then randomly selecting and rearranging the remaining vertices to change the shape of the contour; then making the contour more irregular by randomly offsetting the vertices toward the center of mass of the contour; finally, (using the perturb_seg function) perturbing the generated mask, such as translating, deforming, cropping, and locally cropping and reducing it, to ensure that its intersection-over-union (IOU) with the target mask is close to a predetermined value, thereby completing the random destruction of the GT contour.

[0124] 3.3) Consistent Time Integration (CTE)

[0125] The present invention provides a chip defect detection method that adopts an incremental denoising inference sampling strategy to perform T-step denoising on noise samples drawn from a standard normal distribution and gradually generate denoising results. At each time step, through the inference process of the diffusion model, the denoising result P is generated based on the current time step sample and the predicted denoising result P. t Generate the next time step sample, gradually eliminate the noise, and finally get a clear denoised image. The prediction results of different time steps are integrated through the consensus time integration (CTE) method. t , improve the accuracy and robustness of the mask. At each time step, the denoised predicted image P t The mask is generated by binarization The final mask is generated by combining the prediction results of all time steps through a voting mechanism. This method uses voting average and the mean of consecutive predictions to ensure that the mask is not only accurate at the boundaries but also more complete in describing details, thereby improving the accuracy and stability of chip defect detection and ensuring that defect areas are accurately captured.

[0126] The specific reasoning process includes the following steps:

[0127] a) Initial sample generation

[0128] Draw an initial sample x from a standard normal distribution T , as the input sample of the diffusion model. The initial sample contains pure Gaussian noise and is the starting point of the denoising process.

[0129] b) Stepwise denoising

[0130] For each time step t (decreasing from T to 1), the sample x of the next time step is generated by the following formula t-1 :

[0131]

[0132] x t is the sample of the current time step, is the denoising result predicted at each time step, α t and is the time-related parameter preset in the model, β t represents the weight of the noise, σ t z represents the Gaussian noise introduced during the sampling process (when t=1, z=0).

[0133] At each step, the model is based on the current sample x t And the predicted denoising results Generate the next step sample x t-1, through the diffusion model step by step denoising, eliminate the noise and generate a clear denoising result, and the denoising result of each time step Record it for later processing.

[0134] c) Consensus time integration

[0135] By integrating the prediction results of different time steps, the accuracy and robustness of the mask are improved:

[0136] c.1) Generate a binary mask for each time step:

[0137] For each sampling time step t, the denoised predicted image result Indicated as P t , and perform adaptive thresholding to generate a binary mask

[0138] c.2) Voting and integration:

[0139] Binary mask for all time steps Voting and combining all prediction results P t The final mask P is calculated by taking the mean of emb The mathematical expression is:

[0140]

[0141] Where T represents the sampling time step, mean(P t ) represents the prediction results P of all time steps t The mean of .

[0142] The formula is detailed as follows:

[0143] Represents the binary mask for each time step The average value reflects the proportion of each pixel predicted as the target area at different time steps. The voting mechanism reduces the randomness in the single-step prediction and ensures more stable and consistent results.

[0144] Bias term A fixed bias is added to the voting results to prevent pixels with low voting scores from being ignored. At the same time, the probability values are moderately smoothed to improve the transition effect of the mask edge.

[0145] Time step continuous prediction mean(P t ), directly use the continuous prediction results P in the denoising process t The mean of the voting results Improve the mask's ability to describe details. Binary masks provide clear boundaries, while continuous predictions capture fuzzy areas and uncertainty. The combination of the two ensures that the mask has both accurate boundaries and complete details.

[0146] Through the above process, the final generated chip defect detection mask not only has higher accuracy and stability, but also can make better descriptions in details, ensuring that the defect area is accurately captured.

[0147] (4) Advantages and beneficial effects

[0148] This paper proposes several optimization strategies to address key issues in chip defect detection. First, to address the issue of unclear defect boundary detection, the paper employs a semantic segmentation framework based on a diffusion model and designs a specialized network structure to enhance the model's ability to learn defect boundary information from raw images. By introducing a structural destruction mechanism, the model can more accurately detect defect boundaries, significantly improving detection accuracy and effectively reducing missed detections.

[0149] Secondly, to address the overconfidence issue, this paper introduces a consistent temporal integration strategy (CTE). Through multiple detection sampling and a point voting mechanism, this strategy gradually reduces the overconfidence generated in each prediction. This multiple sampling model can reduce over-inspection and improve overall detection reliability, thereby better capturing chip defects.

[0150] Finally, to address performance bottlenecks, this paper employs a pre-trained image encoder to accelerate training and decouples the image encoder and decoder. During the pre-training phase, the image encoder only needs to be run once to extract features, while the diffusion process is performed in a lightweight decoding head. This design not only optimizes model performance but also improves training efficiency and overall effectiveness, providing a precise and efficient solution for chip defect detection.

[0151] Example 2

[0152] An embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the following Figure 2 A chip defect detection method is shown.

[0153] It is understood that the memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the server, etc.

[0154] The processor may include one or more processing cores. The processor utilizes various interfaces and circuits to connect various components within the server. It executes various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, as well as accessing data stored in memory. Optionally, the processor may be implemented using at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor may integrate one or a combination of a central processing unit (CPU) and a modem. The CPU primarily processes the operating system and application programs, while the modem handles wireless communications. It is understood that the modem may not be integrated into the processor and may be implemented separately via a single chip.

[0155] Since the electronic device is an electronic device corresponding to a chip defect detection method in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0156] Example 3

[0157] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 2 A chip defect detection method is shown.

[0158] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0159] Since the storage medium is a storage medium corresponding to a chip defect detection method of an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0160] Example 4

[0161] In some possible implementations, various aspects of the methods of the embodiments of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is executed on a computer device, the program code is used to cause the computer device to perform the steps of a chip defect detection method according to various exemplary embodiments of the present application described above in this specification. The executable computer program code or "code" used to perform the various embodiments may be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0162] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0163] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0164] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A chip defect detection method, characterized in that: The following steps are involved: Acquire an image of the chip to be tested; The chip image is input into the semantic segmentation model based on the diffusion model, and the defect detection mask of the chip is output; The semantic segmentation model includes an image encoder, an attention fusion module, and an image decoder; the image encoder is used to extract chip image features, and the attention fusion module is used to fuse the extracted chip image features with preset noise features to obtain fused features; The image decoder is used to generate a chip defect detection mask based on the fusion features; In addition, before training the semantic segmentation model, the image encoder is individually pre-trained for denoising to improve the image encoder performance.

2. A chip defect detection method according to claim 1, characterized in that: The image encoder first performs unsupervised denoising pre-training using a diffusion model to learn the deep structure and features of the image. This pre-training process denoises the noisy image using the diffusion model, enabling the encoder to effectively extract the latent features of the original image. The pre-trained image encoder is used as part of the semantic segmentation model based on the diffusion model and is applied to the semantic segmentation task of the diffusion model; In the semantic segmentation stage, the original image features extracted by the image encoder are used to perform supervised semantic segmentation, and finally the image segmentation task is completed.

3. The chip defect detection method according to claim 1, characterized in that: The image encoder is pre-trained for denoising using the denoising autoencoder pre-training method.

4. A chip defect detection method according to claim 1, characterized in that: The training process of the semantic segmentation model includes: Obtain the chip image and the annotated mask image to form an image pair as the input of the model; Input the chip image into the image encoder to extract conditional image features; Encode the mask image and then add noise to obtain the noise image features of the mask after adding noise; The conditional image features and the mask noise image features are input into the attention fusion module for fusion to obtain the fusion features; Input the fused features into the image decoder to generate the chip’s defect detection mask; Calculate the loss based on the generated defect detection mask and the preset chip defect mask, and optimize the model parameters; During the inference phase, the semantic segmentation model generates predictions inversely through the learned diffusion process, adjusting the noisy Gaussian distribution to the learned distribution guided by the test image.

5. The chip defect detection method according to claim 1, characterized in that: The attention fusion module works as follows: Generate query Q by linearly projecting the mask noise image feature D D , key K D Sum V D ; Generate intermediate feature P by linear projection of chip image feature F F Sum V F , the expression is as follows: Where, is the query Q of the mask noise image D , key K D Sum V D The respective weight parameter matrices, It is the intermediate feature P of the chip image feature F Sum V F The weight parameter matrix of The feature map fusion method is as follows: About I =M att1 ·M att2 ·(In D +V F ) Where M att1 and M att2 Represents Q D -P F and K D -P F Attention map of O I Represents the final generated cross-attention fusion feature.

6. A chip defect detection method according to claim 1, characterized in that: The image decoder is implemented using a stacked six-layer deformable attention transformer.

7. The chip defect detection method according to claim 1, characterized in that: Structural damage is introduced during the forward diffusion process: Randomly destroy the contour of the true value and add Gaussian noise to enable the diffusion model to learn in a biased environment and recover the correct mask from the noise mask; In the inference stage, the model only inputs the chip image and obtains the noise features by randomly sampling Gaussian noise.

8. The chip defect detection method according to claim 1, characterized in that: The incremental denoising inference sampling strategy is adopted to perform T-step denoising on the noise samples drawn from the standard normal distribution. The specific inference process includes the following steps: 1) Initial sample generation Draw an initial sample x from a standard normal distribution T , as the input sample of the diffusion model; the initial sample contains pure Gaussian noise and is the starting point of the denoising process; 2) Stepwise denoising For each time step t, the sample x of the next time step is generated by the following formula t-1 : Where x t is the sample of the current time step, is the denoising result predicted at each time step, α t and is the time-related parameter preset in the model, β t represents the weight of the noise, σ t z represents the Gaussian noise introduced during the sampling process; At each step, the model is based on the current sample x t And the predicted denoising results Generate the next step sample x t-1 , through the diffusion model step by step denoising, eliminate the noise and generate a clear denoising result, and the denoising result of each time step Record it for later processing; 3) Consensus time integration By integrating the prediction results of different time steps, the accuracy and robustness of the mask can be improved: 3.1) Generate a binary mask for each time step: For each sampling time step t, the denoised predicted image result Indicated as P t , and perform adaptive thresholding to generate a binary mask 3.2) Voting and integration: Binary mask for all time steps Voting and combining all prediction results P t The final mask P is calculated by taking the mean of emb , the mathematical expression is: Where T represents the sampling time step, mean(P t ) represents the prediction results P of all time steps t The mean of .

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.

Images