A chip defect detection method, device and medium

Abstract

The application discloses a chip defect detection method, equipment and medium, wherein the method comprises the following steps: obtaining a chip image to be detected; inputting the obtained chip image into a semantic segmentation model based on a diffusion model to output a chip defect detection mask; wherein the semantic segmentation model comprises an image encoder, an attention fusion module and an image decoder; the image encoder is used for extracting features of the chip image, the attention fusion module is used for fusing the extracted chip image features with preset noise features to obtain fused features; and the image decoder is used for generating the chip defect detection mask according to the fused features; in addition, before training the semantic segmentation model, the image encoder is separately pre-trained for denoising to improve the performance of the image encoder. The application adopts a semantic segmentation framework of a diffusion model, designs a special network structure, and the model can better learn defect boundary information from an original image. The application can be widely applied to the field of chip defect detection.

Description

Technical Field

[0001] This invention relates to the field of chip defect detection, and more particularly to a chip defect detection method, equipment, and medium. Background Technology

[0002] In the field of chip defect detection, chip images typically exhibit high noise and low contrast, making defect boundaries visually unclear, which complicates the defect detection task significantly. These characteristics—high noise, blurred boundaries, and high similarity between foreground and background—make chip defect detection similar in technical challenge to camouflage object detection. Existing technologies draw inspiration from camouflage detection techniques and improve upon them to meet the specific needs of chip defect detection. Currently, research on camouflage object detection has proposed various technical strategies to address the challenges posed by camouflage characteristics, mainly including the following categories:

[0003] 1) Multi-stream framework: This type of method explicitly learns multi-source representations through multiple input streams, combining different information channels to capture the details and contextual information of objects. For example, some methods combine edge detection streams, texture feature streams, etc., to form multi-parallel feature extraction, so as to more accurately segment and identify disguised objects.

[0004] 2) Bottom-up and Top-down Frameworks: These methods enhance shallow features by utilizing deep features during a single forward propagation, forming multi-level feature fusion. Bottom-up methods build semantic information from low-level features upwards, while top-down methods pass detailed information from high-level features, thereby strengthening the detection of boundaries and subtle features.

[0005] 3) Branching Framework: A single-input, multiple-output architecture is adopted, where the main branch performs the core segmentation task, while auxiliary branches handle related tasks such as boundary detection, classification, or saliency detection. This design helps the model enhance its ability to identify camouflaged objects by leveraging auxiliary information when performing segmentation tasks. For example, an additional boundary detection branch can help the model more accurately locate and segment the edges of camouflaged objects.

[0006] Although these methods have made significant progress in camouflage detection tasks, due to the high similarity of the outline of the camouflage object and the background, the existing methods still have limitations in detecting subtle boundaries, resulting in a decrease in segmentation accuracy. For chip defect detection, this problem is particularly prominent, because unclear defect boundaries can affect the area estimation and positioning of the defect region, causing missed detection and misjudgment of the product. At the same time, the traditional image encoder parameters are usually not fine-tuned by the downstream task, which makes the learning efficiency of the image encoder low in the early stage of training, resulting in poor performance of the model in feature refinement and boundary recognition. The traditional segmentation paradigm based on pixel-level probability can also produce overconfident false predictions, increasing the false detection rate and limiting the reliability of the detection system in a production environment. SUMMARY

[0007] To at least partially solve one of the technical problems existing in the prior art, the purpose of the present application is to provide a chip defect detection method, device and medium based on diffusion model semantic segmentation.

[0008] The first technical solution adopted by the present application is:

[0009] A chip defect detection method, comprising the following steps:

[0010] obtaining a chip image to be detected;

[0011] inputting the obtained chip image into a semantic segmentation model based on a diffusion model, and outputting a defect detection mask of the chip;

[0012] The semantic segmentation model comprises an image encoder, an attention fusion module and an image decoder; the image encoder is used to extract features of the chip image, the attention fusion module is used to fuse the extracted chip image features with preset noise features to obtain fused features, and the image decoder is used to generate a defect detection mask of the chip according to the fused features.

[0013] In addition, before training the semantic segmentation model, the image encoder is separately pre-trained for denoising to improve the performance of the image encoder.

[0014] Further, the image encoder is first pre-trained for unsupervised denoising by a diffusion model to learn the deep structure and features of the image, and the pre-training process is to denoise the noisy image by the diffusion model, so that the encoder can effectively extract the potential features of the original image.

[0015] The pre-trained image encoder is applied to the semantic segmentation task based on the diffusion model as part of the semantic segmentation model based on the diffusion model; in the semantic segmentation stage, the original image features extracted by the image encoder are used for supervised semantic segmentation, and finally the image segmentation task is completed.

[0016] Furthermore, the image encoder is pre-trained using the EDeP method for denoising autoencoders.

[0017] Furthermore, the training process of the semantic segmentation model includes:

[0018] Obtain the chip image and the labeled mask image, form an image pair, and use it as input to the model;

[0019] The chip image is input into the image encoder to extract conditional image features;

[0020] Add noise to the mask image to obtain the features of the noise-added mask image;

[0021] Conditional image features and mask noise image features are input into the attention fusion module for fusion to obtain fused features;

[0022] The fused features are input into the image decoder to generate a defect detection mask for the chip;

[0023] The loss is calculated based on the generated defect detection mask and the preset chip defect mask, and the parameters of the model are optimized.

[0024] During the inference phase, the semantic segmentation model generates predictions in reverse through the learned diffusion process, adjusting the noisy Gaussian distribution to a learned distribution guided by the test image.

[0025] Furthermore, the attention fusion module operates as follows:

[0026] The mask noise image features D are used to generate query Q through linear projection. D Key K D Sum V D The chip image features F are used to generate intermediate features P through linear projection. F Sum V F The expression is as follows:

[0027]

[0028] In the formula, The query Q is for a masked noisy image. D Key K D Sum V D Their respective weight parameter matrices, It is the intermediate feature P of the chip image feature F Sum V F The weight parameter matrix;

[0029] The feature map fusion method is as follows:

[0030]

[0031] O I =M att1 ·M att2 ·(V D +V F )

[0032] In the formula, M att1 and M att2 They represent Q respectively D -P F and K D -P F Attention map; O I This represents the final generated cross-attention fusion feature.

[0033] Furthermore, the image decoder is implemented using a simple stacked six-layer deformable attention transformer.

[0034] Furthermore, structural damage is introduced during the forward diffusion process:

[0035] By randomly disrupting the contours of the true values ​​and adding Gaussian noise, the diffusion model can learn under biased conditions and recover the correct mask from the noise mask.

[0036] During the inference phase, the model takes only chip images as input and obtains noise features by randomly sampling Gaussian noise.

[0037] Furthermore, an incremental denoising inference sampling strategy was adopted to perform T-step denoising on the noise samples drawn from the standard normal distribution. The specific inference process includes the following steps:

[0038] 1) Initial Sample Generation

[0039] An initial sample x is drawn from a standard normal distribution. T , as input samples for the diffusion model; the initial samples contain pure Gaussian noise and are the starting point for the denoising process;

[0040] 2) Gradual noise reduction

[0041] For each time step t (decreasing sequentially from T to 1), the sample x for the next time step is generated using the following formula. t-1 :

[0042]

[0043] In the formula, x t For the current time step, It is the denoising result predicted at each time step, α t and is a preset time-dependent parameter in the model, β t denotes the weight of noise, σ t z represents Gaussian noise introduced during sampling (when t = 1, z = 0);

[0044] In each step, the model generates the next sample x t based on the current sample x and the predicted denoising result t-1 , and the denoising result is recorded for subsequent processing.

[0045] 3) Consensus Time Integration (CTE)

[0046] By integrating the prediction results of different time steps, the accuracy and robustness of the mask are improved:

[0047] 3.1) Generate a binary mask for each time step:

[0048] For each sampling time step t, the denoised predicted image result is denoted as P t , and adaptive thresholding is performed to generate a binary mask

[0049] 3.2) Voting and integration:

[0050] Vote on the binary masks of all time steps , and calculate the final mask P t by combining the mean of all prediction results P emb , the mathematical expression is:

[0051]

[0052] where T represents the sampling time step, and mean(P t ) represents the mean of all prediction results P t .

[0053] The second technical solution adopted by the present application is:

[0054] An electronic device, comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement a chip defect detection method as described above.

[0055] The third technical solution adopted by the present application is:

[0056] A computer readable storage medium, the storage medium stores at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to realize a chip defect detection method as described above.

[0057] The fourth technical solution adopted by the present application is:

[0058] A computer program product or computer program, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer readable storage medium. The processor of the computer equipment can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer equipment executes the above method.

[0059] The beneficial effects of the present application are: the present application adopts a semantic segmentation framework of diffusion model, designs a special network structure, and the model can better learn the defect boundary information from the original image. In addition, after introducing the structure destruction mechanism, the model has been greatly improved in the clarity and accuracy of the defect boundary, successfully reducing the missed detection phenomenon, thereby significantly improving the detection precision of chip defects. BRIEF DESCRIPTION OF DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the present application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of clearly describing part of the embodiments in the technical solutions of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0061] Figure 1 is the structure diagram of the denoising autoencoder in the embodiment of the present application;

[0062] Figure 2 is the architecture diagram of the semantic segmentation model based on diffusion model in the embodiment of the present application;

[0063] Figure 3 is the architecture diagram of the attention fusion module IAM in the embodiment of the present application. DETAILED DESCRIPTION

[0064] Embodiments of the present application are described below in detail with reference to the accompanying drawings, examples of which are shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application. For the step numbers in the following embodiments, they are only set for the convenience of explaining the description, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0065] In the description of the present application, it should be understood that the orientation description, such as the orientation or position relationship indicated by up, down, front, back, left, right, etc. is based on the orientation or position relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as a limitation of the present application. The device or element indicated is not indicated or implied to have a specific orientation, to be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application.

[0066] In the description of the present application, the meaning of several is one or more, and the meaning of multiple is more than two. Greater than, less than, more than, etc. are understood to not include the number, and above, below, etc. are understood to include the number. If it is described as first, second, it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or the order of indicated technical features.

[0067] In the description of the present application, unless otherwise explicitly limited, the words such as setting, installing, connecting, etc. should be broadly understood, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0068] Term explanation:

[0069] EDeP: Encoder Denoising Pre-training, which aims to pre-train the encoder through the denoising task to help the encoder better extract the original image features.

[0070] Diffusion: Diffusion process refers to introducing noise into the model and gradually removing noise through step-by-step iteration to generate high-quality samples or images.

[0071] Backbone: Backbone network.

[0072] Upernet: Multi-scale feature fusion module.

[0073] Injection Attention Module: Attention fusion module.

[0074] In general, the existing technical solutions have the following technical problems: 1) The defect boundary detection is not clear: the chip image has the characteristics of high noise and low contrast, which makes it difficult to distinguish the defect boundary. This characteristic makes the area estimation and positioning in the defect detection process not accurate enough, increasing the risk of missed detection and misjudgment. 2) Model misjudgment problem: Since the segmentation model relies on pixel-level probability for prediction, it is easy to produce overly confident false predictions, resulting in a high misjudgment rate, limiting the reliability of the detection system in actual application. 3) Low efficiency of image encoder initialization: The image encoder parameters are usually not fine-tuned for downstream tasks, resulting in low learning efficiency in the early stages of training, affecting the performance of the model in feature refinement and boundary detection.

[0075] To solve the above problems, the present application introduces a new camouflage detection paradigm: chip defect detection is regarded as a conditional mask generation task based on diffusion model. The diffusion model generates results through a step-by-step denoising method, has excellent generation ability and good perception ability to conditions, so as to more effectively detect chip defects. Following this new paradigm, the present application proposes a framework that uses the denoising process of the diffusion model to gradually eliminate the deviation between the initial noise and the true value by taking the image as an auxiliary condition.

[0076] Embodiment 1

[0077] The embodiment provides a chip defect detection method based on diffusion model semantic segmentation, which regards chip defect detection as a conditional mask generation task based on diffusion model, comprising the following steps:

[0078] S1, obtaining a chip image to be detected;

[0079] S2, inputting the obtained chip image into a semantic segmentation model based on diffusion model to output a defect detection mask of the chip;

[0080] The semantic segmentation model comprises an image encoder, an attention fusion module and an image decoder; the image encoder is used to extract features of the chip image, the attention fusion module is used to fuse the extracted chip image features with preset noise features to obtain fused features, and the image decoder is used to generate a defect detection mask of the chip according to the fused features;

[0081] In addition, before training the semantic segmentation model, the image encoder is separately pre-trained for denoising to improve the performance of the image encoder.

[0082] The working principle of the embodiment is that when the denoising pre-training image encoder is performed, the chip image is input into the image encoder with noise and denoised, and the image encoder performance is improved through the chip denoising pre-training task. That is, the diffusion is used for the denoising task on the image encoder to better initialize the parameters of the image encoder. The pre-trained image encoder is used as part of the diffusion model to extract the chip conditional image features.

[0083] In semantic segmentation, the chip image is only input into the diffusion model as a conditional image, and does not need to be added with noise, that is, only the original chip image needs to be input into the image encoder to extract the conditional image features. The mask image encoding needs to be added with noise, and the corresponding noise features are obtained during the training process of the semantic segmentation model, and the noise features are fused with the image features extracted by the image encoder; the fused features are input into the image decoder to predict and generate the chip defect detection mask.

[0084] Compared with the complex diffusion perception model in the prior art, the framework of the embodiment realizes the decoupling design of the image encoder and the decoder, that is, in the multi-step denoising of inference sampling, the image encoder only needs to run once, and the diffusion process is only performed in the lightweight decoding head. In some embodiments, the initialization parameters of the image encoder are optimized by introducing the unsupervised pre-training EDep method of the denoising autoencoder, the unlabeled data is fully utilized to improve the image feature extraction efficiency, and the performance of the image encoder in feature refinement and boundary recognition is better improved. With this efficient design, the method of the embodiment can be easily applied to the chip defect detection task, and the detection effect and efficiency are significantly improved.

[0085] As an implementation, in the model training stage, Gaussian noise is added to the encoded real value through controlled noise scheduling to generate a noisy image. Then, the noise image and the conditional features extracted by the image encoder (such as PVTv2) are fused in the attention fusion module IAM. Finally, the fused features are sent to the lightweight decoder (i.e., image decoder) to generate a noise-free prediction. In the inference stage, the diffusion framework generates a prediction in reverse through the learned diffusion process, and adjusts the noise Gaussian distribution to a learned distribution under the guidance of the test image.

[0086] The modules in the model are described in detail below in combination with the accompanying drawings and specific embodiments.

[0087] (1) Pre-training EDep method of denoising autoencoder

[0088] Due to the large amount of unlabeled chip image data for chip defect detection, in order to improve the feature extraction efficiency of the image encoder, an unsupervised pre-training method can be used to fully utilize the unlabeled data. The embodiment introduces an unsupervised pre-training EDep method, which uses the principle of denoising autoencoder. The structure of the denoising autoencoder used is as shown in Figure 1

[0089] Specifically, the backbone network is responsible for gradually extracting the features of the image at different levels from low to high, while the multi-scale feature map fusion unit further enhances the expression ability of the features by integrating information from different scales. In this embodiment, the image encoder is first pre-trained unsupervisedly by a diffusion model to learn the deep structure and features of the image. This pre-training process is to denoise the noisy image by the diffusion model, so that the encoder can effectively extract the latent features of the original image. After training, the pre-trained image encoder is applied to the semantic segmentation task of the diffusion model. In the semantic segmentation stage, the image encoder is used as part of the diffusion model to perform supervised semantic segmentation using the features extracted from the original image, and finally completes the image segmentation task.

[0090] Exemplarily, the noisy image is extracted through the backbone network Backbone of an image encoder to obtain the hidden layer representation, and then the hidden layer representation is restored to a noise-free image through the multi-scale feature map fusion unit Upernet. In this way, no labeled data is needed, only chip images are needed. Through the above process, the robust feature representation of the chip image can be learned by restoring the noise data to the original data. The image encoder is pre-trained using the above unsupervised pre-training EDep method.

[0091] Through the process of recovering data from noise by the image encoder, the image encoder can learn deeper features from the unlabeled chip images, more effectively initialize parameters, and improve the ability of the encoder to extract features of the chip images.

[0092] (2) Semantic segmentation model design based on diffusion model

[0093] In this embodiment, the model framework includes an image encoder, an attention fusion module IAM, and an image decoder, which respectively encode the conditional image and decode the mask. In addition, this embodiment designs a special learning strategy in the training and inference process to ensure that the technical problems are solved, including noise image encoding design, structure destruction, and consistent time set.

[0094] The specific model structure is as shown in Figure 2 ​As shown, the model structure is mainly composed of an image encoder (Backbone, Upernet), an attention fusion module (IAM) and an image decoder (Map Decoder). The mask image is encoded by a class embedding layer and then subjected to forward noise diffusion to obtain the noise feature of the mask after noise addition (Encoding & Corruption); the chip image is input into the pre-trained image decoder to extract robust features, and is input into the IAM module together with the encoded noise mask feature for feature fusion of the chip image and the mask noise image, and finally input into the deformable decoder (Map Decoder) to predict the noise and further push the denoised image x t-1 .

[0095] Illustratively, referring to Figure 2 The semantic segmentation model of the embodiment takes the chip image as the conditional image to generate the final segmentation mask image. The specific training process is as follows:

[0096] a) Feature extraction process

[0097] a.1) Input the chip image into the image decoder to obtain the chip image feature.

[0098] a.2) Add noise to the encoded mask image (chip defect mask image annotated by artificial labeling) to obtain the noise mask image feature.

[0099] b) Feature fusion

[0100] Input the chip image feature and the noise mask image feature into the attention module for fusion.

[0101] c) Feature decoding

[0102] Decode the fused feature to obtain the predicted chip defect mask image, and use the chip defect mask annotated by artificial labeling as the ground truth for supervision.

[0103] 2.1) Image encoder

[0104] The image encoder is composed of a backbone network Backbone and a multi-scale feature map fusion module (Upernet). Illustratively, the backbone network PVTv2 is used as the Backbone of the image encoder, and a conventional multi-scale fusion module of Upernet is additionally used for multi-scale fusion of features.

[0105] It should be noted that the image encoder module (Backbone, Upernet) here is the denoising autoencoder pre-trained in the pre-training scheme. The image encoder uses PVTv2 as the backbone network, and then adds the feature fusion module Upernet to form the image encoder. The unannotated chip defect data is used to effectively initialize it to improve the performance, so that the chip image features are more guiding to the denoising of the noise mask.

[0106] In this embodiment, in semantic segmentation, the chip image is taken as the condition image, and the chip image is not added with noise. The chip image (original image) is taken as the condition image and input to the image encoder to directly extract the features.

[0107] 2.2) Attention fusion module (Injection Attention Module)

[0108] Referring to Figure 3 , the attention fusion module performs attention calculation and fusion on the chip image features and the mask noise image features, and introduces the texture and position information of the original features in the noise prediction process. Specifically, the chip image features F are the output feature maps of the image encoder module, and the mask noise image features D come from the noisy class embedding layer encoding vectors. The feature map D generates query Q D , key K D and value V D through linear projection. The feature F generates intermediate feature P F and value V F through linear projection. Among them, the chip image features F only generate the intermediate feature P F for similarity comparison with D, and do not generate the query and key for similarity comparison. The specific expression is as follows:

[0109]

[0110] Wherein, d is the dimension.

[0111] Therefore, the IAM operation is defined as follows:

[0112]

[0113] O I =M att1 ·M att2 ·(V D +V F )

[0114] Wherein, M att1 and M att2 represent Q D -P F and KD -P F attention map. represents the final generated cross-attention fusion feature.

[0115] 2.3) Image decoder

[0116] The embodiment of the present application adopts a simple stacked six-layer deformable attention transformer as the decoder. Unlike the previous parameter-intensive U-net denoising network, the deformable decoder of the embodiment is light and compact, and can effectively reuse shared parameters in the multi-step backpropagation process.

[0117] (3) Training strategy of the model

[0118] 3.1) Noise image encoding

[0119] An embedding-like encoding strategy is adopted. The discrete label (i.e. the class label in the mask image, such as 0, 1, 2, 3, where 0 is the background class, 1 is the collapse class, etc.) is projected into a high-dimensional continuous space through a learnable embedding layer, and normalized using a Sigmoid function. The range of the encoded label is normalized and scaled to [-scale, +scale]. It is worth noting that the scale factor scale controls the signal-to-noise ratio (SNR), which is an important hyperparameter of the diffusion model, and its formula is as follows:

[0120] map_enc=(σ(encoding(maps))×2-1)×scale

[0121] 3.2) Structural damage

[0122] Existing diffusion models directly generate noisy masks from ground truth (GT) by pixel-level destruction, which leads the model to falsely assume that the contours recovered from noisy masks are always accurate, thus failing to correct the bias. However, this assumption does not necessarily hold during inference sampling. To address this issue, the embodiment introduces structural damage during the forward diffusion process. Specifically, we randomly destroy the contours of the GT and add Gaussian noise, allowing the model to learn how to recover the correct mask from the noisy mask in a biased manner.

[0123] As an optional implementation, the way to randomly destroy the contours of the GT includes: first, deleting part of the vertices on the contour to reduce its continuity; then randomly selecting and rearranging the remaining vertices to change the shape of the contour; next, making the contour more irregular by randomly offsetting the vertices in the direction of the contour centroid; finally, perturbing the generated mask (using the perturb_seg function) such as translation, deformation, cropping, local cropping and scaling, to ensure that its intersection over union (IOU) with the target mask is close to a predetermined value, thus completing the random destruction of the GT contour.

[0124] 3.3) Consistent Time Integration (CTE)

[0125] This invention provides a chip defect detection method that employs an incremental denoising inference sampling strategy. It generates a denoising result by performing T-step denoising on noise samples extracted from a standard normal distribution. At each time step, through the inference process of a diffusion model, based on the current time step sample and the predicted denoising result P... t The next time step sample is generated, and noise is gradually eliminated to obtain a clear, denoised image. The prediction results P from different time steps are integrated using the Consensus Temporal Integration (CTE) method. t This improves the accuracy and robustness of the mask. At each time step, the denoised predicted image P... t Binarization generates a mask The final mask is obtained by combining the prediction results of all time steps through a voting mechanism. This method utilizes the voting average and the mean of continuous predictions to ensure that the mask is not only accurate at the boundaries but also more complete in its detailed description, thereby improving the accuracy and stability of chip defect detection and ensuring accurate capture of defect areas.

[0126] The specific reasoning process includes the following steps:

[0127] a) Initial Sample Generation

[0128] An initial sample x is drawn from a standard normal distribution. T The initial samples, containing pure Gaussian noise, serve as the input samples for the diffusion model and are the starting point for the denoising process.

[0129] b) Gradual noise reduction

[0130] For each time step t (decreasing sequentially from T to 1), the sample x for the next time step is generated using the following formula. t-1 :

[0131]

[0132] x t For the current time step, It is the denoising result predicted at each time step, α t and β is a time-dependent parameter preset in the model. t The weight σ represents the noise. t z represents the Gaussian noise introduced during the sampling process (z = 0 when t = 1).

[0133] In each step, the model is based on the current sample x. t And the predicted denoising results Generate the next sample x t-1, by gradually denoising through a diffusion model, eliminating noise and generating clear denoised results, and the denoised results of each time step are recorded for subsequent processing.

[0134] c) consensus time integration

[0135] By integrating the prediction results of different time steps, the accuracy and robustness of the mask are improved:

[0136] c.1) Generate a binary mask for each time step:

[0137] For each sampling time step t, the denoised prediction image result is represented as P t , and adaptive thresholding is performed to generate a binary mask

[0138] c.2) Voting and integration:

[0139] The binary masks of all time steps are voted on, and the final mask P t is calculated by combining the mean of all prediction results P emb . The mathematical expression is:

[0140]

[0141] where T represents the sampling time step, and mean(P t ) represents the mean of all time step prediction results P t .

[0142] The formula is explained in detail as follows:

[0143] represents the average of the binary mask for each time step , reflecting the proportion of each pixel being predicted as a target area at different time steps. Through the voting mechanism, the randomness in single-step prediction is reduced, ensuring that the results are more stable and consistent;

[0144] bias term adds a fixed bias to the voting result, avoiding the direct neglect of low-voting score pixels, while moderately smoothing the probability value to improve the transition effect of the mask edge.

[0145] Time step continuous prediction mean(P t ), directly using the mean of the continuous prediction results P t in the denoising process, combined with the voting result to improve the mask's ability to describe details. The binary mask provides clear boundaries, and the continuous prediction captures fuzzy areas and uncertainties, ensuring that the mask has both boundary accuracy and detail integrity.

[0146] Through the above process, the finally generated chip defect detection mask not only has higher precision and stability, but also can make better description in details, ensuring accurate capture of the defect area.

[0147] (4) Advantages and beneficial effects

[0148] For the key problems in chip defect detection, the present application proposes a number of optimization strategies. First, in order to solve the problem of unclear defect boundary detection, the present application adopts a semantic segmentation framework based on diffusion model, and designs a special network structure, which enhances the ability of the model to learn defect boundary information from the original image. By introducing the structure destruction mechanism, the model can more accurately detect the defect boundary, significantly improve the detection accuracy, and effectively reduce the missed detection phenomenon.

[0149] Secondly, for the problem of overconfidence, the present application introduces a consistent time integration strategy (CTE), which gradually reduces the overconfidence generated in each prediction through multiple detection sampling and point voting mechanisms. Through multiple sampling, the model can reduce the over-detection phenomenon and improve the reliability of the overall detection, so as to better capture the chip defects.

[0150] Finally, in order to solve the performance bottleneck, the present application adopts a pre-trained image encoder to speed up the training process and decouples the image encoder and decoder. In the pre-training stage, only the image encoder needs to be run once to extract features, and the diffusion process is performed in the lightweight decoding head. This design not only optimizes the model performance, but also improves the training efficiency and overall effect, providing a precise and efficient solution for chip defect detection.

[0151] Embodiment 2

[0152] The embodiment of the present application also provides an electronic device, which comprises a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to realize a chip defect detection method as shown in Figure 2 .

[0153] It can be understood that the memory can include a random access memory (RAM) and can also include a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, codes, code sets or instruction sets. The memory can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function, instructions for implementing various method embodiments described above, etc.; and the data storage area can store data created according to the use of the server, etc.

[0154] The processor can include one or more processing cores. The processor connects various parts within the entire server through various interfaces and lines, executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor can be integrated with one or a combination of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes operating systems and application programs, etc.; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor, but can be implemented by a separate chip.

[0155] Since the electronic device is an electronic device corresponding to the chip defect detection method of the embodiments of the present application, and the principle of solving the problem of the electronic device is similar to that of the method, the implementation of the electronic device can be referred to the implementation process of the above-mentioned method embodiments, and the repeated parts will not be repeated.

[0156] Embodiment 3

[0157] The embodiments of the present application also provide a computer readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement a chip defect detection method as shown in Figure 2

[0158] ​Those skilled in the art can understand that all or part of the steps of various methods in the above embodiments can be instructed by programs to relevant hardware, and the programs can be stored in a computer readable storage medium, including Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other medium that can be used to carry or store data which can be used by a computer.

[0159] Since the storage medium is a storage medium corresponding to the chip defect detection method of the embodiments of the present application, and the principle of solving the problem of the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above method embodiments, and the repeated parts will not be described again.

[0160] Embodiment 4

[0161] In some possible implementation manners, various aspects of the method of the embodiments of the present application can also be implemented in the form of a program product, which includes program codes for causing a computer device to execute the steps of the chip defect detection method according to various exemplary embodiments of the present application described in the specification when the program product is run on the computer device. Wherein, the executable computer program codes or "codes" for executing various embodiments can be written in high-level programming languages such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (for example, Transact-SQL), Perl, or in various other programming languages.

[0162] It should be understood that various aspects of the application can be implemented in hardware, software, firmware or a combination of them. In the above embodiments, various steps or methods can be implemented in software or firmware which is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, can be used to implement the hardware: discrete logic circuitry having logic gates for implementing logic functions upon data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and so forth, or a combination of them.

[0163] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, a person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples, without contradiction.

[0164] The above embodiments are only for the purpose of illustrating the technical concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and to implement it, and cannot limit the protection scope of the present application. Any equivalent changes or modifications made in accordance with the essence of the present application should be covered within the protection scope of the present application.

Claims

1. A method of detecting defects in a chip, the method comprising: The method comprises the following steps: Obtain a chip image to be detected; Input the obtained chip image into a semantic segmentation model based on a diffusion model to output a defect detection mask of the chip; The semantic segmentation model comprises an image encoder, an attention fusion module, and an image decoder; the image encoder is used to extract features of the chip image, the attention fusion module is used to fuse the extracted features of the chip image with preset noise features to obtain fused features, and the image decoder is used to generate the defect detection mask of the chip according to the fused features; In addition, before training the semantic segmentation model, the image encoder is separately pre-trained for denoising to improve the performance of the image encoder; The training process of the semantic segmentation model comprises the following steps: Obtain a chip image and a labeled mask image to form an image pair as input of the model; Input the chip image into the image encoder to extract conditional image features; Encode and add noise to the mask image to obtain a noise-added mask noise image feature; Input the conditional image features and the mask noise image feature into the attention fusion module for fusion to obtain fused features; Input the fused features into the image decoder to generate the defect detection mask of the chip; Calculate a loss according to the generated defect detection mask and a preset chip defect mask to optimize parameters of the model; In the inference stage, the semantic segmentation model reversely generates a prediction through a learned diffusion process to adjust a noise Gaussian distribution to a learned distribution under the guidance of a test image.

2. The method of claim 1, wherein The image encoder is first pre-trained for unsupervised denoising through a diffusion model to learn deep structures and features of the image, and the pre-training process is to denoise a noisy image through the diffusion model, so that the encoder can effectively extract latent features of the original image; The pre-trained image encoder is used as part of the semantic segmentation model based on the diffusion model and applied to a semantic segmentation task of the diffusion model; In the semantic segmentation stage, the original image features extracted by the image encoder are used for supervised semantic segmentation to finally complete the image segmentation task.

3. The method of claim 1, wherein the step of detecting the defect of the chip is performed by using a defect detection device. The pre-training method of the denoising autoencoder is used to pre-train the image encoder.

4. The method of claim 1, wherein the step of detecting the defect of the chip is performed by using a defect detection device. The working mode of the attention fusion module is as follows: Mask noise image features Generate a query using linear projection. ,key Sum ; chip image features Intermediation features are generated through linear projection. Sum The expression is as follows: wherein is a query of the mask noise image , keys and values a respective weight parameter matrix, is an intermediate feature of the chip image feature and values a weight parameter matrix; The feature map fusion mode is as follows: In the formula, and respectively represent and attention maps; represent the cross-attention fusion features finally generated.

5. The method of claim 1, wherein the step of detecting the defect of the chip is performed by using a defect detection program. The image decoder is implemented by using a stacked six-layer deformable attention transformer.

6. The method of claim 1, wherein In the forward diffusion process, structural damage is introduced: The contour of the true value is randomly damaged and Gaussian noise is added, so that the diffusion model can learn in a biased manner to recover the correct mask from the noise mask; In the inference stage, only the chip image is input into the model, and noise features are obtained by randomly sampling Gaussian noise.

7. The method of claim 1, wherein the step of detecting the defect of the chip is performed by using a defect detection device. An incremental denoising inference sampling strategy is adopted, noise samples extracted from a standard normal distribution are denoised for T steps, and the specific inference process comprises the following steps: 1) Initial sample generation initial samples are drawn from a standard normal distribution as input samples for the diffusion model; the initial samples contain pure Gaussian noise and are the starting point for the denoising process; 2) Step-by-step denoising For each time step t, the sample for the next time step is generated by the following equation : In the formula, is a sample at a current time step, is a denoising result predicted at each time step, and is a preset time correlation parameter in the model, represents a weight of noise, represents Gaussian noise introduced in the sampling process; In each step, the model is based on the current sample. And the predicted denoising results Generate the next sample The noise is gradually denoised using a diffusion model, eliminating noise and generating a clear denoised result. The denoising results at each time step are then displayed. Record it for later processing; 3) Consensus time integration The prediction results of different time steps are integrated to improve the accuracy and robustness of the mask: 3.1) Generate a binary mask of each time step: For each sampling time step t, the denoised predicted image result is represented as and adaptive thresholding is performed to generate a binary mask ; 3.2) Voting and integration: a binary mask for all time steps voting and combining all prediction results the final mask is computed as the mean of mathematically expressed as: wherein denotes the sampling time step, denotes the mean of all time step prediction results .

8. An electronic device, comprising: The electronic device includes a processor and a memory having stored therein at least one instruction, at least one program, a code set or instruction set, which is loaded and executed by the processor to implement the method as claimed in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The storage medium has stored therein at least one instruction, at least one program, a code set or instruction set, which is loaded and executed by a processor to implement the method as claimed in any one of claims 1 to 7.

Images