Detection method, detection device, processing equipment and readable storage medium

Abstract

The invention provides a detection method, a detection device, processing equipment and a readable storage medium, and the detection method comprises the steps: carrying out the target restoration of a to-be-detected image of a to-be-detected sample through a machine learning model, so as to generate a target-free reference image; and obtaining a target area of the to-be-detected image according to the difference between the reference image and the to-be-detected image. According to the detection method, the detection precision can be improved, so that the wrong piece or omission ratio is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of detection, and particularly to a detection method, a detection device, a processing device and a readable storage medium. Background Art

[0002] In the process of wafer defect detection, it is necessary to identify defects in the detection image. Related technologies often obtain defect points by comparing the gray levels of each pixel of the detection image with those of the template image. However, due to the different background grays of different detection images, it is easy to have false detections or missed detections when using the same template image to identify defects in each detection image. Moreover, the template image is often obtained by averaging the pixels of the images of multiple objects to be measured, and the template image may also have defects, which further increases the probability of false detections or missed detections. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a detection method and a detection device, which can improve the detection accuracy and thus reduce the false part or missed detection rate.

[0004] The present application provides a detection method, including: using a machine learning model to perform target repair on a to-be-detected image of a to-be-detected sample to generate a reference image without a target; obtaining a target region of the to-be-detected image according to the difference between the reference image and the to-be-detected image.

[0005] Optionally, the machine learning model includes an information analysis module and a repair module; using the machine learning model to perform target repair on a to-be-detected image of a to-be-detected sample to generate a reference image without a target includes: obtaining semantic embedding information and defect embedding information of the to-be-detected image through the information analysis module; the repair module repairs the target of the to-be-detected image according to the semantic embedding information and the defect embedding information to obtain a reference image without a target.

[0006] Optionally, the information analysis module includes an image encoder and a prompt processor. Obtaining semantic embedding information and defect embedding information of the to-be-detected image includes: encoding the to-be-detected image using the image encoder; processing the encoded to-be-detected image using the prompt processor to obtain the semantic embedding information and the defect embedding information of the to-be-detected image.

[0007] Optionally, the repair module includes: a feature generator and an information-aware decoder; the repair module repairs the target of the to-be-detected image according to the semantic embedding information and the defect embedding information, including: respectively using the semantic embedding information and the defect embedding information as inputs of the feature generator of the repair module, so that the feature generator outputs a latent feature, and the latent feature includes high-level semantic information and guiding clues for detail repair; inputting the latent feature into the information-aware decoder to repair the target of the to-be-detected image.

[0008] Optionally, the feature generator includes a convolutional neural network and an image-guided control module; the semantic embedding information and the defect embedding information are respectively used as inputs to the feature generator of the repair module, so that the feature generator outputs latent features, including: inputting the image to be tested and the defect embedding information into the image-guided control module to obtain control information, where the control information is used to guide the convolutional neural network to regulate the target area; inputting the semantic embedding information and the control information into the convolutional neural network to output latent features. Optionally, the image-guided control module includes a control encoder and an image controller; among them, inputting the image to be tested and the defect embedding information into the image-guided control module to obtain control information includes: inputting the image to be tested and the defect embedding information into the control encoder to obtain an image encoding feature vector; inputting the image encoding feature vector into the image controller to obtain the control information; among them, inputting the semantic embedding information and the control information into the convolutional neural network to output latent features includes: inputting the semantic embedding information and random noise into the encoding layer of the convolutional neural network to obtain a semantic embedding feature vector; inputting the semantic embedding feature vector, the semantic embedding information and the control information into the decoding layer of the convolutional neural network to output the latent features.

[0009] Optionally, the steps of obtaining the machine learning model include: obtaining multiple test images and standard images of a test sample, constructing a training data set, where the standard image is a target-free image; training an initial model using the training data set to obtain the machine learning model.

[0010] Optionally, obtaining multiple test images and standard images of a test sample includes: photographing the test sample to obtain multiple test images of the test sample; obtaining a standard sample corresponding to the test sample, photographing the standard sample to obtain a standard image, where the standard sample has no such target; or, obtaining multiple test images and standard images of a test sample includes: obtaining multiple test images of multiple identical test samples; matching the multiple test images so that the pixels of the test images at the same position of the sample to be tested correspond to obtain a pixel group; obtaining the median of the pixel values of at least some pixels in the pixel group as the pixel value at the same position in the standard image; where the median includes one or more combinations of the median and the weighted value of the mean; the pixel values include one or more combinations of: gray value, light intensity, charge.

[0011] Optionally, the machine learning model includes: an information analysis model, a repair module, and a supervision model; the detection method further includes: optimizing the repair module using the supervision model; wherein, optimizing the repair module using the supervision model includes: obtaining semantic embedding information of the test image and defect embedding information of the test image through the information analysis module; the repair module repairs the target of the test image according to the semantic embedding information of the test image and the defect embedding information of the test image to obtain a reference image corresponding to the test image; inputting the test image and the defect embedding information of the test image into a control encoder to obtain an image coding feature vector of the test image; inputting the image coding feature vector of the test image into a control decoder to obtain a loss supervision image; performing loss supervision using the loss supervision image and the reference image corresponding to the test image; and updating the weights of the convolutional neural network based on the result of the loss supervision between the loss supervision image and the reference image corresponding to the test image.

[0012] Optionally, the step of obtaining the target area of the to-be-tested image according to the difference between the reference image and the to-be-tested image includes: obtaining the gray values of each pixel point in the reference image and the gray values of each pixel point in the to-be-tested image; determining the difference between the gray values of each pixel point in the reference image and the to-be-tested image; and when the difference is greater than a preset threshold, taking the area of the pixel point corresponding to the difference in the to-be-tested image as the target area of the to-be-tested image.

[0013] Optionally, the target is a defect, and the target area is a defect area.

[0014] The present application also provides a detection device, including: a generation module, configured to perform target repair on a to-be-tested image of a to-be-tested sample using a machine learning model to generate a reference image without a target; and a first acquisition module, configured to obtain the target area of the to-be-tested image according to the difference between the reference image and the to-be-tested image.

[0015] Optionally, the generation module includes: a construction sub-module, configured to obtain multiple test images and a standard image of a test sample and construct a training data set, where the standard image is an image without a target; and a training sub-module, configured to train an initial model using the training data set to obtain the machine learning model.

[0016] Optionally, the construction sub-module includes: a first acquisition module, configured to capture the test sample to obtain a plurality of test images of the test sample; a second acquisition module, configured to capture a standard sample corresponding to the test sample to obtain a standard image, where the standard sample has no target; or, the construction sub-module includes: a first acquisition module, configured to obtain a plurality of test images of a plurality of identical test samples; a matching module, configured to match the plurality of test images to make the pixels of the test images at the same position of the sample to be tested correspond, to obtain a pixel group; a second acquisition module, configured to obtain the median of the pixel values of at least some of the pixels in the pixel group as the pixel value at the same position in the standard image; where the median includes one or more combinations of a median and a weighted value of an average value; the pixel value includes: one or more combinations of a grayscale value, light intensity, and charge.

[0017] Optionally, the generation module further includes: an information analysis module, configured to obtain semantic embedding information and defect embedding information of the image to be tested; a repair module, configured to repair the target of the image to be tested according to the semantic embedding information and the defect embedding information, to obtain a reference image without a target.

[0018] Optionally, the information analysis module includes: an image encoder, configured to encode the image to be tested; a prompt processor, configured to process the encoded image to be tested to obtain the semantic embedding information and the defect embedding information of the image to be tested; and / or, the repair module includes: an output sub-module, configured to respectively use the semantic embedding information and the defect embedding information as inputs to a feature generator, so that the feature generator outputs a latent feature, the latent feature including high-level semantic information and guiding clues for detailed repair; and a repair sub-module, configured to input the latent feature into an information-aware decoder to repair the target of the image to be tested.

[0019] Optionally, the output sub-module includes: a first acquisition unit, configured to input the image to be tested and the defect embedding information into an image-guided control module to obtain control information, the control information being used to guide a convolutional neural network to regulate a target area; and a second acquisition unit, configured to input the semantic embedding information and the control information into the convolutional neural network and output a latent feature.

[0020] Optionally, the first obtaining unit includes: a first obtaining subunit, configured to input the image to be measured and the defect embedding information into a control encoder to obtain an image encoding feature vector; a second obtaining subunit, configured to input the image encoding feature vector into an image controller to obtain the control information; and / or, the second obtaining unit includes: a third obtaining subunit, configured to input the semantic embedding information and random noise into an encoding layer of a convolutional neural network to obtain a semantic embedding feature vector; a fourth obtaining subunit, configured to input the semantic embedding feature vector, the semantic embedding information, and the control information into a decoding layer of the convolutional neural network to output the latent feature.

[0021] The present application further provides a processing device, including: a processor, a memory, and a program or instruction stored on the memory and executable on the processor, where when the program or instruction is executed by the processor, the detection method of the present application is implemented.

[0022] The present application further provides a readable storage medium, where a program or instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the detection method of the present application is implemented.

[0023] The technical solution of the present invention at least includes the following beneficial effects: The detection method provided by the technical solution of the present invention performs target repair on the image to be measured of the sample to be measured by using a machine learning model to generate a reference image without a target; and obtains the target area of the image to be measured according to the difference between the reference image and the image to be measured, which can improve the detection accuracy and thus reduce the misclassification or missed detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flowchart of a detection method provided by an embodiment of the present invention; Figure 2 is a module architecture diagram of a machine learning model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] Hereinafter, exemplary embodiments of the present application will be described in more detail with reference to the drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be more thoroughly understood and the scope of the present invention can be fully conveyed to those skilled in the art.

[0026] As Figure 1 shown, an embodiment of the present invention provides a detection method, including: Step S1, performing target repair on the image to be measured 1 of the sample to be measured by using a machine learning model 15 to generate a reference image 14 without a target; Step S2: Obtain the target region of the to-be-tested image 1 based on the difference between the reference image 14 and the to-be-tested image 1.

[0027] Among them, the to-be-tested sample may include a wafer, and the to-be-tested image 1 may include an image obtained by photographing the wafer.

[0028] In this embodiment, the present invention repairs the target of the to-be-tested image 1 through the machine learning model 15 to obtain a reference image 14 without the target corresponding to the to-be-tested image 1, and calculates using the reference image 14 and the to-be-tested image 1 to obtain the position and contour information of the target in the to-be-tested image 1, which can improve the detection accuracy and thus reduce the mis-parts or missed detection rate.

[0029] In this embodiment, the target may be a defect; using a machine learning model to repair the target of the to-be-tested image of the to-be-tested sample to generate a reference image without the target includes: using a machine learning model to repair the defect of the to-be-tested image of the to-be-tested sample to generate a reference image without defects; among them, obtaining the target region of the to-be-tested image according to the difference between the reference image and the to-be-tested image includes: obtaining the defect region of the to-be-tested image according to the difference between the reference image and the to-be-tested image.

[0030] Among them, the machine learning model 15 may be a deep learning diffusion model.

[0031] In the embodiment of the present invention, in the step S1, the steps of obtaining the machine learning model 15 include: obtaining multiple test images and standard images of the test sample, constructing a training data set, and the standard image is an image without the target; training an initial model using the training data set to obtain the machine learning model 15. When the target is a defect, the standard image is an image without defects.

[0032] Specifically, obtain the feature element map of the test image; train the initial model using the training data set to output a test reference image; perform loss supervision using the test reference image and the feature element map; based on the result of the loss supervision, update the weights of the initial model to determine the machine learning model 15. Among them, the feature element map may be a pixel point map. When specifically implemented, 3000 feature element maps can be obtained to construct a training data set.

[0033] In this embodiment, by constructing a training data set of the machine learning model 15, establishing an initial model, training the initial model using the feature element map and the test image, and obtaining the optimal weights of the machine learning model 15, the machine learning model 15 is optimized. This can improve the processing performance of the machine learning model 15, so as to output a reference image 14 with excellent visual effects, accurate image structure, and high-definition image quality.

[0034] In a specific embodiment of the present application, obtaining a plurality of test images and a standard image of a test sample includes: photographing the test sample to obtain a plurality of test images of the test sample; obtaining a standard sample corresponding to the test sample, photographing the standard sample to obtain a standard image, and the standard sample has no such target.

[0035] In another specific embodiment of the present application, obtaining a plurality of test images and a standard image of a test sample includes: obtaining a plurality of test images of a plurality of identical test samples; matching the plurality of test images so that the pixels of the test images at the same position of the sample to be tested correspond to obtain a pixel group; obtaining the median of the pixel values of at least some pixels in the pixel group as the pixel value of the same position in the standard image; wherein, the median includes one or a combination of more of the median and the weighted value of the mean value; the pixel value includes one or a combination of more of the gray value, light intensity, and charge.

[0036] As Figure 2 shown, in an embodiment of the present invention, the machine learning model includes an information analysis module and a repair module; using the machine learning model to perform target repair on a test image of a sample to be tested to generate a reference image without a target, including: obtaining semantic embedding information 4 and defect embedding information 5 of the test image through the information analysis module; the repair module repairs the target of the test image 1 according to the semantic embedding information 4 and the defect embedding information 5 to obtain a reference image 14 without a target.

[0037] In an embodiment of the present invention, the information analysis module includes an image encoder and a prompt processor; obtaining semantic embedding information 4 and defect embedding information 5 of the test image includes: encoding the test image 1 using the image encoder 2; processing the encoded test image 1 using the prompt processor 3 to obtain the semantic embedding information 4 and the defect embedding information 5 of the test image 1.

[0038] In an embodiment of the present invention, the repair module includes: a feature generator and an information-aware decoder; the repair module repairs the target of the test image 1 according to the semantic embedding information 4 and the defect embedding information 5, including: respectively using the semantic embedding information 4 and the defect embedding information 5 as inputs to the feature generator of the repair module, so that the feature generator outputs a latent feature 10, and the latent feature 10 includes high-level semantic information and guiding clues for detail repair; inputting the latent feature 10 into the information-aware decoder 13 to repair the target of the test image 1.

[0039] In an embodiment of the present invention, the feature generator includes a convolutional neural network 7 and an image guidance control module. The semantic embedding information 4 and the defect embedding information 5 are respectively used as inputs to the feature generator of the repair module, so that the feature generator outputs latent features 10, including: inputting the image to be measured 1 and the defect embedding information 5 into the image guidance control module to obtain control information, where the control information is used to guide the convolutional neural network 7 to regulate the target area; inputting the semantic embedding information 4 and the control information into the convolutional neural network 7 to output latent features 10.

[0040] In an embodiment of the present invention, the image guidance control module includes a control encoder 8 and an image controller 9. Among them, inputting the image to be measured 1 and the defect embedding information 5 into the image guidance control module to obtain control information includes: inputting the image to be measured 1 and the defect embedding information 5 into the control encoder 8 to obtain an image encoding feature vector; inputting the image encoding feature vector into the image controller 9 to obtain the control information. Among them, inputting the semantic embedding information 4 and the control information into the convolutional neural network 7 to output latent features 10 includes: inputting the semantic embedding information 4 and random noise 6 into the encoding layer 71 of the convolutional neural network 7 to obtain a semantic embedding feature vector; inputting the semantic embedding feature vector, the semantic embedding information 4 and the control information into the decoding layer 72 of the denoising convolutional neural network 7, so that the decoding layer 72 outputs latent features 10.

[0041] Among them, the semantic embedding information 4 represents the overall semantic content of the image to be measured 1; the defect embedding information 5 represents the defect features of the image to be measured 1. The image encoding feature vector is a control vector obtained from the control encoder 8, and the image encoding feature vector is used to guide the generation or restoration process of the reference image 14. The control information is the result obtained after performing image control processing on the image encoding feature vector. The latent features 10 are the intermediate results output by the decoding layer 72 of the convolutional neural network 7, and the latent features 10 are used to obtain the reference image 14 after decoding or reconstruction. The semantic embedding feature vector is the result output by the encoding layer 71 of the convolutional neural network 7, and the semantic embedding feature vector is the result of numerically representing the image.

[0042] Specifically, when implemented, the specific implementation process of using the machine learning model 15 to perform defect repair on the image to be measured 1 of the sample to be measured to generate a defect-free reference image 14 may include: obtaining a set of the images to be measured 1 ; where n is the total number of the to-be-tested images 1, specifically 3000 images; the to-be-tested images 1 are encoded by an image encoder 2; a prompt processor 3 is used to process the encoded to-be-tested images 1 to obtain semantic embedding information 4 corresponding to the to-be-tested images 1 and defect embedding information 5 ; where the semantic embedding information 4 represents the overall semantic content of the to-be-tested image 1, and the defect embedding information 5 is used to capture defect features caused by process defects in the to-be-tested image. The to-be-tested image 1 and the defect embedding information 5 are input into a control encoder 8 to obtain an image encoding feature vector; the image encoding feature vector is input into an image controller 9 to obtain control information, and the control information is used to guide a convolutional neural network 7 to regulate a target area; the semantic embedding information 4 and random noise 6 are input into an encoding layer 71 of the convolutional neural network 7 to obtain a semantic embedding feature vector; the semantic embedding feature vector, the semantic embedding information 4, and the control information are input into a decoding layer 72 of the convolutional neural network 7 to obtain a latent feature 10; that is: ; where is the latent feature 10, is the convolutional neural network 7, is the semantic embedding information 4, is the control information; if the convolutional neural network 7 is processed m times, m latent features 10 can be obtained ; where m is the total number of the latent features 10. The latent feature 10 and the defect embedding information 5 are input into an information perception decoder 13 to obtain the reference image 14; that is ; where is the reference image 14, is the information perception decoder 13, is the latent feature 10, is the defect embedding information 5.

[0043] In this embodiment, when the machine learning model 15 is specifically applied, key visual prompt information, that is, the semantic embedding information 4 and the defect embedding information 5, is first extracted from the to-be-tested image 1 by the image encoder 2 and the prompt processor 3; where the semantic embedding information 4, as content prompt, can provide high-level semantic information for the machine learning model 15 to guide the content restoration in the generation process of the reference image 14, so as to ensure that the generated reference image 14 is consistent with the expectation at the semantic level.

[0044] The defect embedding information 5 is used to capture structural defects and the like caused by process defects in the image to be measured 1. The defect embedding information 5 is used to modulate the control encoder 8 and the image controller 9 to generate control information, and the reference image 14 is generated through the control information. The control information can accurately locate and regulate the target area of the image to be measured 1 to ensure that while maintaining the authenticity of the original image, the reference image 14 can accurately repair the defective area in the image to be measured 1, thereby avoiding distortion or structural misalignment of the reference image 14 and ensuring the visual effect, clarity, and accuracy of the reference image 14.

[0045] Through the convolutional neural network 7, taking the control information and the semantic embedding information 4 as inputs, the defects in the image to be measured 1 are gradually repaired after multiple denoising operations, and potential features that conform to the semantic and structural features of the image to be measured 1 are generated through conditional guidance, namely, potential feature 10.

[0046] The potential feature 10 contains high-level semantic information of the image to be measured 1 and guiding clues for detailed repair. Finally, the information perception decoder 13 further processes the potential feature 10 and converts the potential feature 10 into the high-quality reference image 14.

[0047] Through the information perception decoder 13, effective repair of the defective area of the image to be measured 1 can be achieved, thereby converting the potential encoding into the final high-quality reference image 14. The information perception decoder 13 can not only ensure the overall semantic consistency of the reference image 14, but also perform structural correction on the image to be measured 1 in combination with the defect information, and output the defect-free reference image 14. Through the perception correction of the potential encoding, the information perception decoder 13 can accurately convert complex defect features into the pixel domain, thereby ensuring that the generated reference image 14 meets the expected requirements visually and structurally, and at the same time reflecting the original details and quality of the image to be measured 1. The information perception decoder 13 not only corrects the defective area in the image to be measured 1, but also ensures that the restored reference image 14 is visually realistic, thus achieving the operator's visual expectation for the reference image 14.

[0048] In this way, not only can efficient repair of the image to be measured 1 be achieved, but also precise processing of complex defects can be realized by utilizing the embedding and control of the semantic embedding information 4 and the defect embedding information 5 of the image to be measured 1, enabling the machine learning model 15 to perform detailed defect repair while maintaining the original authenticity of the image to be measured 1, and significantly improving the quality and consistency of the reference image 14.

[0049] In specific implementation, the training process of the convolutional neural network 7 may include: obtaining a training data set, a validation data set, and a test data set, where the training data set, the validation data set, and the test data set may include: standard defect-free images and corresponding defective images; performing preprocessing processes such as standardization and data augmentation on the training data set; by preprocessing the training data set, the generalization ability of the convolutional neural network 7 can be improved; defining the architecture of the convolutional neural network 7; ensuring that the input and output shapes of the convolutional neural network 7 match the training data; the convolutional neural network can be optimized by mean squared error loss; perceptual loss and adversarial loss can also be added according to actual needs; the optimizer can be selected as Adam (Adaptive Moment Estimation), set the epoch (number of optimizations) to 100 times, the batch size to 16 batches, and set the learning rate to 0.00001.

[0050] For each epoch, the following steps are executed: input the test image into the convolutional neural network 7 to obtain a generated image; calculate the loss between the generated image and the test image; according to the loss between the generated image and the test image, perform backpropagation on the generated image to update the weights of the convolutional neural network 7, and optimize the parameters of the convolutional neural network 7 using the optimizer according to the weights; at the end of each epoch, evaluate the performance of the convolutional neural network 7 using the validation data set, record the loss value and PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index), MSE (Mean Squared Error), and select an early stopping strategy and adjust the learning rate according to the performance of the validation data set; after the training is completed, save the best weights of the convolutional neural network 7, which is convenient for the use and inference of the convolutional neural network 7; evaluate the final performance of the convolutional neural network 7 on the test data set, and calculate evaluation metrics to test the actual effect of the model.

[0051] As Figure 2As shown in the figure, in an embodiment of the present invention, the machine learning model includes: an information analysis model, a repair module, and a supervision model; the detection method further includes: optimizing the repair module using the supervision model; wherein, optimizing the repair module using the supervision model includes: obtaining semantic embedding information of a test image and defect embedding information of the test image through the information analysis module; the repair module repairs the target of the test image according to the semantic embedding information of the test image and the defect embedding information of the test image to obtain a reference image corresponding to the test image; inputting the test image and the defect embedding information of the test image into a control encoder to obtain an image coding feature vector of the test image; inputting the image coding feature vector of the test image into a control decoder 11 to obtain a loss supervision image 12; performing loss supervision using the loss supervision image 12 and the reference image 14 corresponding to the test image; based on the result of the loss supervision between the loss supervision image 12 and the reference image 14 corresponding to the test image, updating the weights of the convolutional neural network 7.

[0052] Wherein, the loss supervision image 12 is an image obtained after being processed by the control decoder 11, and the loss supervision image 12 is a result obtained through image restoration or image generation, and is used to supervise the process of image restoration.

[0053] Specifically, the specific implementation process of performing loss supervision using the loss supervision image 12 and the reference image 14 may include: According to , calculating the loss between the loss supervision image 12 and the reference image 14; wherein, is the loss between the loss supervision image 12 and the reference image 14, is the gray value of each pixel point in the loss supervision image 12, is the gray value of each pixel point in the reference image 14; based on the loss between the loss supervision image 12 and the reference image 14 , updating the weights of the denoising convolutional neural network 7; when updating the weights of the convolutional neural network 7, the loss between the loss supervision image 12 and the reference image 14 can be incorporated into the total loss function of the machine learning model 15.

[0054] In this embodiment, by comparing the loss supervision image 12 with the reference image 14 corresponding to the test image, additional feedback can be provided, thereby improving the quality of the reference image 14 corresponding to the image to be tested. During the decoding process by the information perception decoder 13, with the aid of the defect embedding information 5 and the latent feature 10, a reference image corresponding to the image to be tested with consistency and authenticity is generated.

[0055] The weights are used to adjust the convolutional neural network 7 and control the parameters of the decoder 11 to optimize the response to the input image. The weights can affect the feature extraction and reconstruction processes at each level, ensuring maximum preservation of semantic and structural consistency during the generation of the reference image 14, so that the finally output reference image 14 can accurately reflect the original details in the image to be measured 1, while repairing the defective areas to ensure the visual effect and structure of the reference image 14 are accurate.

[0056] In an embodiment of the present invention, step 12 may include: step 121, obtaining the gray values of each pixel point in the reference image 14 and the gray values of each pixel point in the image to be measured; step 122, determining the difference between the gray values of each pixel point in the reference image 14 and the image to be measured 1; step 123, when the difference is greater than a preset threshold, taking the area of the pixel point corresponding to the difference in the image to be measured 1 as the target area of the image to be measured 1.

[0057] Specifically, when implementing, the specific implementation process of obtaining the defective area of the image to be measured 1 according to the difference between the reference image 14 and the image to be measured 1 may include: obtaining the gray values of each pixel point in the reference image 14 and the gray values of each pixel point in the image to be measured 1; determining the difference between the gray values of each pixel point in the reference image 14 and the image to be measured 1: ; where is the gray value of the pixel point in the reference image 14 and the image to be measured 1, and the gray value of the pixel point in the reference image 14, is the gray value of the pixel point in the image to be measured 1, is the gray value of the pixel point in the reference image 14. When the difference is greater than the preset threshold, taking the area of the pixel point corresponding to the difference in the image to be measured 1 as the target area of the image to be measured 1; where the preset threshold may be the standard deviation of the difference or may be determined as a fixed value according to actual operation experience for determining whether the difference between the gray values of each pixel point in the image to be measured 1 and the gray values of the reference image 14 is significant.

[0058] It should be noted that after determining the target area of the image to be measured 1, morphological operations can be performed on the targets within the target area, and the number, size, and position of the target area can be counted. This can remove the noise and misjudgment in the image to be measured 1, making the determined target area more accurate.

[0059] In this embodiment, by comparing the gray values of each pixel point in the to-be-tested image 1 with the gray values of the reference image 14, the target area in the to-be-tested image 1 is determined. This determination method is simple to operate and easy to implement, and can quickly and accurately determine the target area in the to-be-tested image 1, improving the processing efficiency.

[0060] It should be noted that the solution of the present invention uses the machine learning model 15 to repair the to-be-tested image 1 to generate the reference image 14. This step provides a reference image 14 after defect repair through efficient denoising and defect filling, ensuring that the reference image 14 can truly reflect the semantic and structural features of a normal wafer, which is the key to subsequent target detection.

[0061] Taking the reference image 14 as the target output and the to-be-tested image 1 as the input, the machine learning model 15 is trained. In this way, the machine learning model 15 learns how to generate a defect-free reference image 14 from the to-be-tested image 1. This design effectively uses the standard reference image 14 as the repair target, improving the repair ability and accuracy of the machine learning model 15.

[0062] During the detection process, a corresponding reference image 14 is dynamically generated for each to-be-tested image 1, and the reference image 14 is used to compare with the to-be-tested image 1 to accurately locate the target area. This design ensures a high degree of matching between the reference image 14 for each comparison and the current to-be-tested image 1 by generating references for each image, thereby improving the accuracy of target detection.

[0063] Using the machine learning model, making full use of the conditional control and denoising capabilities during the learning process, the to-be-tested image 1 is repaired and generated. This choice makes full use of the powerful capabilities of the machine learning model 15 in image generation and repair, and is a core competitive and difficult-to-avoid design point in the technical solution of the present invention.

[0064] The core of the technical solution of the present invention lies in achieving high-quality target repair and accurate defect detection through the machine learning model 15. This combined strategy not only improves the reliability of target detection, but also makes the solution have a strong technical barrier, which is the key to technical protection.

[0065] The embodiment of the present invention further provides a detection device, including: A generation module, configured to use the machine learning model 15 to repair the defects of the to-be-tested image 1 of the to-be-tested sample to generate a reference image 14 without targets; A first acquisition module, configured to obtain the target area of the to-be-tested image according to the difference between the reference image 14 and the to-be-tested image 1.

[0066] In an embodiment of the present invention, the generation module includes: a construction sub-module, configured to obtain a plurality of test images and standard images of a test sample, and construct a training data set, where the standard image is a target-free image; a training sub-module, configured to train an initial model using the training data set to obtain the machine learning model 15.

[0067] In an embodiment of the present invention, the construction sub-module includes: a first acquisition module, configured to photograph the test sample to obtain a plurality of test images of the test sample; a second acquisition module, configured to photograph a standard sample corresponding to the test sample to obtain a standard image, where the standard sample has no such target.

[0068] In another embodiment of the present invention, the construction sub-module includes: a first acquisition module, configured to obtain a plurality of test images of a plurality of identical test samples; a matching module, configured to match the plurality of test images so that the pixels of the test images at the same position of the sample to be tested correspond to each other, to obtain a pixel group; a second acquisition module, configured to obtain the median of the pixel values of at least some pixels in the pixel group as the pixel value of the same position in the standard image; where the median includes one or a combination of more of the median and the weighted value of the mean value; the pixel value includes: one or a combination of more of the gray value, light intensity, and charge.

[0069] In an embodiment of the present invention, the generation module further includes: an information analysis module, configured to obtain semantic embedding information and defect embedding information of a test image; a repair module, configured to repair the target of the test image according to the semantic embedding information and the defect embedding information to obtain a target-free reference image.

[0070] In an embodiment of the present invention, the information analysis module includes: an image encoder, configured to encode the test image; a prompt processor, configured to process the encoded test image to obtain the semantic embedding information 4 and the defect embedding information 5 corresponding to the test image. And / or, the repair module includes: an output sub-module, configured to respectively use the semantic embedding information 4 and the defect embedding information 5 as inputs to a feature generator, so that the feature generator outputs a latent feature 10, where the latent feature 10 includes high-level semantic information and guiding clues for detail repair; a repair sub-module, configured to input the latent feature 10 into an information-aware decoder 13 to repair the target of the test image.

[0071] In an embodiment of the present invention, the output sub-module includes: a first acquisition unit, configured to input the test image and the defect embedding information 5 into an image-guided control module to obtain control information, where the control information is used to guide the convolutional neural network to regulate the target area; a second acquisition unit, configured to input the semantic embedding information 4 and the control information into a convolutional neural network and output a latent feature 10.

[0072] In an embodiment of the present invention, the first acquisition unit includes: a first acquisition subunit, configured to input the to-be-tested image and the defect embedding information into a control encoder 8 to obtain an image encoding feature vector; a second acquisition subunit, configured to input the image encoding feature vector into an image controller 9 to obtain control information. And / or, in an embodiment of the present invention, the second acquisition unit includes: a third acquisition subunit, configured to input the semantic embedding information 4 and random noise into an encoding layer 71 of a convolutional neural network 7 to obtain a semantic embedding feature vector; a fourth acquisition subunit, configured to input the semantic embedding feature vector, the semantic embedding information, and the control information into a decoding layer 72 of the convolutional neural network 7 to obtain a latent feature 10.

[0073] In an embodiment of the present invention, it further includes: a control decoder 11, configured to receive the image encoding feature vector and output a loss supervision image 12. Loss supervision is performed using the loss supervision image and the reference image; based on the result of the loss supervision between the loss supervision image and the reference image, the weights of the convolutional neural network 7 are updated.

[0074] In an embodiment of the present invention, the first acquisition module includes: a grayscale value acquisition module, configured to acquire the grayscale values of each pixel point in the reference image 14 and the grayscale values of each pixel point in the to-be-tested image 1; a grayscale difference acquisition module, configured to determine the difference between the grayscale values of each pixel point in the reference image 14 and the to-be-tested image 1; a target area determination module, configured to, when the difference is greater than a preset threshold, use the area of the pixel point corresponding to the difference in the to-be-tested image 1 as the target area of the to-be-tested image 1.

[0075] In this embodiment, the machine learning model 15 in the detection device is used to repair the defects in the to-be-tested image 1 of the to-be-tested sample to generate a target-free reference image 14, and the target area of the to-be-tested image 1 is obtained according to the difference between the reference image 14 and the to-be-tested image 1, which can improve the detection accuracy and thus reduce the mispart or missed inspection rate.

[0076] It should be noted that this device is the device corresponding to the above method, and all implementation manners in the above method embodiment are applicable to the embodiment of this device and can also achieve the same technical effect.

[0077] An embodiment of the present invention further provides a computing device, including: a processor and a memory storing a computer program, and when the computer program is run by the processor, it executes the method as described above. All implementation manners in the above method embodiment are applicable to this embodiment and can also achieve the same technical effect.

[0078] An embodiment of the present invention also provides a computer-readable storage medium, including instructions that, when run on a computer, cause the computer to execute the method described above. All implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0079] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0080] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0081] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical, mechanical, or other form.

[0082] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0083] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0084] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs.

[0085] In addition, it should be noted that in the devices and methods of the present invention, obviously, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present invention. And, the steps of performing the above series of processes can naturally be executed in chronological order according to the described order, but it is not necessary to execute them in chronological order. Some steps can be executed in parallel or independently of each other. For those of ordinary skill in the art, it can be understood that all or any steps or components of the methods and devices of the present invention can be implemented in any computing device (including processors, storage media, etc.) or in a network of computing devices in the form of hardware, firmware, software, or a combination thereof, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0086] Therefore, the object of the present invention can also be achieved by running a program or a set of programs on any computing device. The computing device can be a well-known general device. Therefore, the object of the present invention can also be achieved only by providing a program product containing program codes for implementing the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be noted that in the devices and methods of the present invention, obviously, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present invention. And, the steps of performing the above series of processes can naturally be executed in chronological order according to the described order, but it is not necessary to execute them in chronological order. Some steps can be executed in parallel or independently of each other.

[0087] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A detection method, characterized in that: include: Using a machine learning model to perform target repair on the image of the sample to be tested to generate a reference image without a target; A target area of ​​the image to be tested is obtained according to a difference between the reference image and the image to be tested.

2. The detection method according to claim 1, characterized in that: The machine learning model includes an information analysis module and a repair module; The machine learning model is used to perform target repair on the image of the sample to be tested to generate a reference image without a target, including: Acquire semantic embedding information and defect embedding information of the image to be tested through the information analysis module; The repair module repairs the target of the image to be tested according to the semantic embedding information and the defect embedding information to obtain a reference image without the target.

3. The detection method according to claim 2, characterized in that: The information analysis module includes an image encoder and a prompt processor, and obtains semantic embedding information and defect embedding information of the image to be tested, including: encoding the image to be tested using the image encoder; and processing the encoded image to be tested using the prompt processor to obtain the semantic embedding information and defect embedding information of the image to be tested.

4. The detection method according to claim 2, characterized in that: The repair module includes: a feature generator and an information perception decoder; the repair module repairs the target of the image to be tested according to the semantic embedding information and the defect embedding information, including: The semantic embedding information and the defect embedding information are respectively used as inputs of a feature generator of the repair module, so that the feature generator outputs latent features, wherein the latent features include high-level semantic information and guidance clues for detail repair; The latent features are input into an information-aware decoder to repair the target of the image to be tested.

5. The detection method according to claim 4, characterized in that: The feature generator includes a convolutional neural network and an image guided control module; the semantic embedding information and the defect embedding information are respectively used as inputs of the feature generator of the repair module, so that the feature generator outputs potential features, including: Inputting the image to be tested and the defect embedding information into an image guidance control module to obtain control information, wherein the control information is used to guide the convolutional neural network to regulate the target area; The semantic embedding information and the control information are input into the convolutional neural network, and potential features are output.

6. The detection method according to claim 5, characterized in that: The image guidance control module includes a control encoder and an image controller; The step of inputting the image to be tested and the defect embedding information into an image guidance control module to obtain control information includes: inputting the image to be tested and the defect embedding information into a control encoder to obtain an image encoding feature vector; inputting the image encoding feature vector into an image controller to obtain the control information; Among them, the semantic embedding information and the control information are input into the convolutional neural network, and the potential features are output, including: inputting the semantic embedding information and random noise into the encoding layer of the convolutional neural network to obtain a semantic embedding feature vector; inputting the semantic embedding feature vector, the semantic embedding information and the control information into the decoding layer of the convolutional neural network, and outputting the potential features.

7. The detection method according to any one of claims 1 to 6, characterized in that: The steps of obtaining the machine learning model include: Acquire multiple test images and standard images of the test sample to construct a training data set, wherein the standard image is a non-target image; The training data set is used to train the initial model to obtain the machine learning model.

8. The detection method according to claim 7, characterized in that: Acquiring multiple test images and standard images of the test sample, including: photographing the test sample to obtain multiple test images of the test sample; acquiring a standard sample corresponding to the test sample, photographing the standard sample to obtain a standard image, wherein the standard sample does not have the target; or, Acquiring multiple test images and standard images of a test sample, including: obtaining multiple test images of multiple identical test samples; matching the multiple test images so that pixels of the test images at the same position of the test sample correspond to each other to obtain a pixel group; acquiring the median of the pixel values ​​of at least part of the pixels in the pixel group as the pixel value of the same position in the standard image; The median includes one or more combinations of median and mean weighted values; the pixel value includes one or more combinations of grayscale value, light intensity, and charge.

9. The detection method according to claim 5, characterized in that: The machine learning model includes: an information analysis model, a repair module and a supervision model; The detection method further comprises: optimizing the repair module using a supervision model; Among them, the repair module is optimized by using a supervision model, including: obtaining semantic embedding information of a test image and defect embedding information of the test image through an information analysis module; the repair module repairs the target of the test image according to the semantic embedding information of the test image and the defect embedding information of the test image, and obtains a reference image corresponding to the test image; the test image and the defect embedding information of the test image are input into a control encoder to obtain an image coding feature vector of the test image; the image coding feature vector of the test image is input into a control decoder to obtain a loss supervision image; Using the loss supervision image and the reference image corresponding to the test image to perform loss supervision; Based on the result of loss supervision between the loss supervision image and the reference image corresponding to the test image, the weights of the convolutional neural network are updated.

10. The detection method according to claim 1, characterized in that: The step of acquiring the target area of ​​the image to be tested according to the difference between the reference image and the image to be tested comprises: Obtaining the grayscale value of each pixel in the reference image and the grayscale value of each pixel in the image to be tested; Determine the difference between the grayscale value of each pixel in the reference image and the image to be tested; When the difference is greater than a preset threshold, a region of pixels corresponding to the difference in the image to be tested is used as a target region of the image to be tested.

11. The detection method according to claim 1, characterized in that: The target is a defect, and the target area is a defect area.

12. A detection device, characterized in that: include: A generation module, used for performing target restoration on the image to be tested of the sample to be tested by using a machine learning model to generate a reference image without a target; The first acquisition module is used to acquire the target area of ​​the image to be tested according to the difference between the reference image and the image to be tested.

13. The detection device according to claim 12, characterized in that: The generation module comprises: A construction submodule is used to obtain a plurality of test images and standard images of the test sample to construct a training data set, wherein the standard image is a target-free image; The training submodule is used to train the initial model using the training data set to obtain the machine learning model.

14. The detection device according to claim 13, characterized in that: The construction submodule includes: a first acquisition module, used to photograph the test sample to obtain a plurality of test images of the test sample; a second acquisition module, used to photograph a standard sample corresponding to the test sample to obtain a standard image, wherein the standard sample does not have the target; Alternatively, the construction submodule includes: a first acquisition module, used to obtain multiple test images of multiple identical test samples; a matching module, used to match the multiple test images so that the pixels of the test images at the same position of the sample to be tested correspond to each other to obtain a pixel group; a second acquisition module, used to obtain the median value of the pixel values ​​of at least part of the pixels in the pixel group as the pixel value of the same position in the standard image; The median includes one or more combinations of median and mean weighted values; the pixel value includes one or more combinations of grayscale value, light intensity, and charge.

15. The detection device according to claim 12, characterized in that: The generating module also includes: An information analysis module, used to obtain semantic embedding information and defect embedding information of the image to be tested; The repair module is used to repair the target of the image to be tested according to the semantic embedding information and the defect embedding information to obtain a reference image without the target.

16. The detection device according to claim 15, characterized in that: The information analysis module includes: An image encoder, used for encoding the image to be tested; a prompt processor, used to process the encoded image to be tested, and obtain semantic embedding information and defect embedding information of the image to be tested; and / or, The repair module comprises: an output submodule, configured to respectively use the semantic embedding information and the defect embedding information as inputs of a feature generator, so that the feature generator outputs potential features, wherein the potential features include high-level semantic information and guidance clues for detail repair; and The repair submodule is used to input the potential features into the information perception decoder to repair the target of the image to be tested.

17. The detection device according to claim 16, characterized in that: The output submodule comprises: A first acquisition unit, configured to input the image to be tested and the defect embedding information into an image-guided control module to acquire control information, wherein the control information is used to guide a convolutional neural network to regulate a target area; and The second acquisition unit is used to input the semantic embedding information and the control information into a convolutional neural network and output potential features.

18. The detection device according to claim 17, characterized in that: The first acquiring unit includes: A first acquisition subunit is used to input the image to be tested and the defect embedding information into a control encoder to obtain an image coding feature vector; A second acquisition subunit is used to input the image coding feature vector into an image controller to acquire the control information; and / or The second acquiring unit includes: A third acquisition subunit is used to input the semantic embedding information and random noise into the encoding layer of the convolutional neural network to obtain a semantic embedding feature vector; The fourth acquisition subunit is used to input the semantic embedding feature vector, the semantic embedding information and the control information into the decoding layer of the convolutional neural network, and output the potential feature.

19. A processing device, characterized in that: include: A processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the detection method according to any one of claims 1 to 11.

20. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the detection method according to any one of claims 1 to 11 is implemented.

Images