Defect detection method and device, electronic equipment and storage medium

Abstract

The invention provides a defect detection method and device, electronic equipment and a storage medium. The method comprises the following steps: performing feature extraction operation on a to-be-detected image of a to-be-detected wafer grain to obtain a first image feature; and based on the first image feature, the reference image feature and the first query information, through a trained language model, generating a detection result of the to-be-detected wafer grain. On one hand, compared with a mode of representing the detection result by using a prediction label output by a deep learning model, the detection result output by the language model can be more flexible, and is beneficial to adapting to a scene with complex and numerous defects in a wafer or a wafer grain; and on the other hand, the reference image feature is used as one of the inputs of the language model, so that the language model can focus on the difference between the to-be-detected wafer grain and the reference wafer grain, and the accuracy of the detection result is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically to a defect detection method, a defect detection device, an electronic device, a storage medium and a computer program product. Background Art

[0002] In the industrial field, with the continuous development of production technology, people have put forward higher requirements for the production quality and production efficiency of industrial products. With the development of image processing technology, more and more product inspections are based on images.

[0003] Since there may be many types of defects in wafers or wafer grains, the process of defect detection based on the image of wafers or wafer grains is relatively complicated. If the deep learning model in the related art is used to detect defects in wafers or wafer grains, as the number of defect types that the deep learning model can detect increases, its detection ability for a single defect type will decrease. In addition, the deep learning model can only output the defect type, which is difficult to accurately represent the complex and numerous defects in wafers or wafer grains. Therefore, how to detect wafers or wafer grains more accurately is a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0004] The present invention is proposed in view of the above problems. The present invention provides a defect detection method, a defect detection device, an electronic device, a storage medium and a computer program product.

[0005] According to one aspect of the present invention, a defect detection method is provided, the detection method comprising:

[0006] Performing a feature extraction operation on the image to be tested of the wafer to be tested to obtain a first image feature;

[0007] Based on the first image feature, the reference image feature and the first query information, the detection result of the wafer grain to be tested is generated through the trained language model, wherein the reference image feature is the feature obtained by the reference image after the feature extraction operation, and the reference wafer grain in the reference image does not have a defective part. The first query information is used to guide the generation of the detection result, and the detection result is used to describe the defective part of the wafer grain to be tested.

[0008] Exemplarily, the detection method further comprises:

[0009] Comparing the reference image with the image to be tested to determine a region of interest in the image to be tested, wherein a difference between a pixel value of at least one pixel located in the region of interest and a pixel corresponding to the pixel in the reference image is greater than a preset difference;

[0010] Performing a sampling operation on feature points corresponding to the focus area in the first image feature to obtain a first defect feature;

[0011] Based on the first image feature, the reference image feature and the first query information, a detection result of the wafer to be tested is generated through the trained language model, including:

[0012] Based on the first image feature, the reference image feature, the first query information and the first defect feature, a detection result of the wafer to be tested is generated through the trained language model.

[0013] Exemplarily, the trained language model is obtained by the following steps:

[0014] Performing a feature extraction operation on a training image of a training wafer die to obtain a second image feature;

[0015] Based on the second image feature, the reference image feature and the second query information, a prediction result of the training wafer die is generated by an untrained language model, wherein the second query information is used to guide the generation of the prediction result, and the prediction result is used to describe the defective part of the training wafer die;

[0016] Based on the difference between the prediction results and the actual defect information of the training wafer particles, the model parameters of the untrained language model are adjusted.

[0017] Exemplarily, the defective portion of the training wafer corresponds to a defective area in the training image, and based on the second image feature, the reference image feature and the second query information, a prediction result of the training wafer is generated by an untrained language model, including:

[0018] Performing a sampling operation on feature points corresponding to the defect area in the second image feature to obtain a second defect feature;

[0019] Determine defect description features based on real defect information;

[0020] Performing a feature fusion operation on the second defect feature and the defect description feature to obtain a fused feature;

[0021] Based on the second image features, the reference image features, the second query information and the fusion features, a prediction result of the training wafer grain is generated through an untrained language model.

[0022] Exemplarily, the defect area is obtained by any of the following steps:

[0023] A defective pixel corresponding to the defective part of the training wafer in the training image is used as the center of the circle and a circle is drawn with a preset radius as the radius to obtain a defective area;

[0024] Determine the circumscribed rectangle of the defective pixels corresponding to the defective portion of the training wafer die in the training image as the defective area;

[0025] The defective pixels corresponding to the defective parts of the training wafer in the training image are directly used as the pixels in the defective area.

[0026] Exemplarily, the second query information is used to query the defect position of the defective part of the first wafer die in the training image, and the real defect information is used to describe that the first wafer die does not have the defective part, wherein the first wafer die is a training wafer die without the defective part; and / or

[0027] The second query information is used to query whether the first position of the defective part of the training wafer die in the training image is correct, and the real defect information is used to describe that the first position is incorrect and the defective position of the defective part of the training wafer die in the training image, wherein the first position is the position of the normal part of the training wafer die in the training image; and / or

[0028] The second query information is used to query the first method of the defective part of the training wafer grain, and the real defect information is used to describe that the first method is incorrect and the correct method of the defective part of the training wafer grain, wherein the first method is an incorrect processing method for the defective part, and the correct method is a correct processing method for the defective part.

[0029] Exemplarily, the second query information is used to query the defect type of the defective part of the training wafer die, and the real defect information is used to describe the defect type of the defective part of the training wafer die; and / or

[0030] The second query information is used to query the defect position of the defective part of the training wafer die in the training image, and the real defect information is used to describe the defect position of the defective part of the training wafer die in the training image; and / or

[0031] The second query information is used to query the positional relationship between the defective part of the training wafer die and the functional area in the training wafer die, and the real defect information is used to describe the positional relationship between the defective part of the training wafer die and the functional area in the training wafer die; and / or

[0032] The second query information is used to query the visual characteristics of the defective part of the training wafer die, and the real defect information is used to describe the visual characteristics of the defective part of the training wafer die.

[0033] Exemplarily, the second query information is used to query defect information of the training wafer die in the preset area, and the real defect information is used to describe the defect information of the training wafer die in the preset area; and / or

[0034] The second query information includes defect information, and the second query information is used to query the position of the part of the training wafer grain that matches the defect information in the training image. The real defect information is used to describe the position of the part of the training wafer grain that matches the defect information in the training image.

[0035] Exemplarily, the second query information is used to query the defect type and cause of the defective part of the training wafer die, and the real defect information is used to describe the defect type and cause of the defective part of the training wafer die; and / or

[0036] The second query information is used to query the processing method of the defective part of the training wafer die, and the real defect information is used to describe the processing method of the defective part of the training wafer die; and / or

[0037] The second query information is used to query the wafer processing flow improvement suggestions for the defective part of the training wafer, and the real defect information is used to describe the wafer processing flow improvement suggestions for the defective part of the training wafer.

[0038] According to yet another aspect of the present invention, a defect detection device is provided. The detection device includes: a first image feature extraction module and a detection result generation module.

[0039] A first image feature extraction module is used to perform a feature extraction operation on the image to be tested of the wafer to be tested to obtain a first image feature;

[0040] The detection result generation module is used to generate the detection result of the wafer grain to be tested based on the first image feature, the reference image feature and the first query information through the trained language model, wherein the reference image feature is the feature obtained by the reference image after the feature extraction operation, and the reference wafer grain in the reference image does not have a defective part. The first query information is used to guide the generation of the detection result, and the detection result is used to describe the defects of the wafer grain to be tested.

[0041] According to another aspect of the present invention, an electronic device is provided, which includes a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used to execute the above-mentioned defect detection method when the processor is running.

[0042] According to another aspect of the present invention, a storage medium is provided, on which program instructions are stored, and the program instructions are used to execute the above-mentioned defect detection method when running.

[0043] According to another aspect of the present invention, a computer program product is also provided, wherein the computer program product comprises computer program instructions, and the computer program instructions are used to execute the above-mentioned defect detection method when running.

[0044] According to the above scheme of the embodiment of the present invention, a feature extraction operation can be performed on the image to be tested of the wafer grain to be tested to obtain the first image feature. Then, based on the first image feature, the reference image feature and the first query information, the detection result of the wafer grain to be tested is generated through the trained language model. On the one hand, compared with the method of using the prediction label output by the deep learning model to represent the detection result, the detection result in the above scheme outputted by the language model can be more flexible, which is conducive to adapting to the scene with complex and numerous defects in the wafer or wafer grain; on the other hand, the above scheme also uses the reference image feature as one of the inputs of the language model, so that the language model can focus on the difference between the wafer grain to be tested and the reference wafer grain, which is conducive to improving the accuracy of the detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and other purposes, features and advantages of the present invention will become more apparent by describing the embodiments of the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0046] Figure 1 A schematic flow chart of a defect detection method according to an embodiment of the present invention is shown;

[0047] Figure 2 A schematic diagram showing a training process of a language model according to an embodiment of the present invention is shown;

[0048] Figure 3 A schematic diagram showing a training process of a language model according to an embodiment of the present invention is shown;

[0049] Figure 4 A schematic block diagram showing a defect detection device according to an embodiment of the present invention; and

[0050] Figure 5 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

[0052] In the field of component manufacturing, if defective industrial components (such as wafers) are packaged and tested, it will waste manpower and material resources, so it is necessary to inspect the industrial components so that unqualified industrial components can be detected and eliminated. However, there may be a variety of defects on industrial components. For example, taking wafers as an example, there may be defects such as foreign matter and cracks on the wafer. For another example, taking the wafer grains on the wafer as an example, there may be defects such as pin prints out of bounds, too many pin prints, and foreign matter on the wafer grains. However, the defect types that can be output by the defect detection model in the prior art are limited, which is not conducive to adapting to the actual scenario where there are complex and numerous defects in wafers or wafer grains.

[0053] In order to at least partially solve the above problem, an embodiment of the present invention provides a defect detection method. Figure 1 FIG. 1 is a schematic flow chart of a defect detection method according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps S110 to S120.

[0054] In step S110 , a feature extraction operation is performed on the image to be tested of the wafer die to obtain a first image feature.

[0055] The image to be tested of the wafer grain to be tested can be a static image or any video frame in a dynamic video. The image to be tested of the wafer grain to be tested can be the original image collected by the image acquisition device (for example, the original image collected by the image sensor in the camera), or it can be an image obtained after preprocessing the original image (such as digitization, normalization, smoothing, etc.). In one example, a wafer image of the wafer to be tested can be collected, and then the wafer image can be divided into multiple images of the wafer grains to be tested. For example, the wafer grain area to be tested in the wafer image can be determined by matching the wafer image with the reference wafer grain image of the reference wafer grain. Then, the wafer grain area to be tested in the wafer image can be cropped to obtain the image to be tested of the wafer grain to be tested.

[0056] A feature extraction operation can be performed on the image to be tested of the wafer grain to be tested by a feature extraction algorithm or a feature extraction model to obtain a first image feature. For example, a Haar-like Features extraction algorithm in the related art, a feature extraction module in a convolutional neural network, a feature extraction module in a recurrent neural network, etc. can be used to perform a feature extraction operation on the image to be tested to obtain a first image feature. It should be understood that the image to be tested can also be input into an image encoder to obtain an image embedding of the image to be tested, and the image embedding can be used as the first image feature. The above-mentioned image encoder can also be pre-trained.

[0057] In step S120 , based on the first image feature, the reference image feature and the first query information, a detection result of the wafer die to be tested is generated by using the trained language model.

[0058] The reference image feature is a feature obtained by a feature extraction operation on the reference image. The feature extraction algorithm or feature extraction model described above can be used to perform a feature extraction operation based on the reference image to obtain the reference image feature. The reference wafer grain in the reference image may not have defective parts. For example, an image acquisition device can be used to capture an image of a reference wafer grain without defective parts to obtain the above-mentioned reference image. Then, a feature extraction operation can be performed on the reference image through a feature extraction algorithm or a feature extraction model to obtain the reference image feature. For example, the Haar feature (Haar-like Features) extraction algorithm in the related art, the feature extraction module in the convolutional neural network, the feature extraction module in the recurrent neural network, etc. can be used to perform a feature extraction operation on the reference image to obtain the reference image feature. It should be understood that the reference image can also be input into an image encoder to obtain an image embedding of the reference image, and the image embedding can be used as the reference image feature.

[0059] The first query information is used to guide the generation of the test results, and the test results are used to describe the defective part of the wafer grain to be tested. The above-mentioned first query information can be input by the user through an input device, for example, the above-mentioned input device may include a keyboard, a microphone, etc. The first query information can be directly input through the keyboard, or a voice can be input through the microphone, and then the voice is converted into the first query information. The first query information can be used to perform defect-related queries on the image to be tested, and the test results can be used to describe the defects related to the relevant content in the image to be tested based on the semantics of the first query information.

[0060] The language model in the embodiment of the present invention may be any language model with language question-answering capabilities. For example, the language model may be a model such as BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer), or LLMMA (Large Language Model Meta AI).

[0061] The above-mentioned language model may include at least an encoder and a decoder. The above-mentioned encoder may be used to convert the first query information into a first query code. Then, the first query code, the first image feature, and the reference image feature may be input into the decoder together after feature processing (for example, feature splicing, feature fusion, feature enhancement, etc.) to generate a detection result that conforms to natural language. Combined with the actual scenario, the first query information may be "Does the wafer grain in this picture have an abnormal area, and what is the type of abnormality", and the detection result may be "There is an abnormal area in the wafer grain in this picture, and the type of the abnormal area is scratches".

[0062] In one example, other relevant information about the wafer to be tested may also be determined and converted into a code. Then, the information is input into a decoder after feature processing together with the first query code, the first image feature, and the reference image feature to generate the above-mentioned detection result. The specific content of the relevant information may depend on the actual situation. For example, the relevant information may include: a text description related to the visual characteristics of the wafer to be tested, a text description related to the processing stage of the wafer to be tested, etc.

[0063] According to the above scheme of the embodiment of the present invention, a feature extraction operation can be performed on the image to be tested of the wafer grain to be tested to obtain the first image feature. Then, based on the first image feature, the reference image feature and the first query information, the detection result of the wafer grain to be tested is generated through the trained language model. On the one hand, compared with the method of using the prediction label output by the deep learning model to represent the detection result, the detection result in the above scheme outputted by the language model can be more flexible, which is conducive to adapting to the scene with complex and numerous defects in the wafer or wafer grain; on the other hand, the above scheme also uses the reference image feature as one of the inputs of the language model, so that the language model can focus on the difference between the wafer grain to be tested and the reference wafer grain, which is conducive to improving the accuracy of the detection result.

[0064] Exemplarily, the trained language model is obtained by training through the following steps S210 to S230.

[0065] In step S210 , a feature extraction operation is performed on the training image of the training wafer die to obtain a second image feature.

[0066] The training image of the training wafer grain can be a static image or any video frame in a dynamic video. The training image of the training wafer grain can be the original image captured by the image acquisition device (for example, the original image captured by the image sensor in the camera), or it can be an image obtained after preprocessing the original image (such as digitization, normalization, smoothing, etc.). In one example, a wafer image of a training wafer grain can be collected, and then the wafer image can be divided into multiple images of training wafer grains. For example, the training wafer grain area in the wafer image can be determined by matching the wafer image with the reference wafer grain image of the reference wafer grain, and then the training wafer grain area in the wafer image can be cropped to obtain a training image of the training wafer grain.

[0067] A feature extraction operation may be performed on the training image of the training wafer grain by a feature extraction algorithm or a feature extraction model to obtain the second image feature. For example, a Haar-like Features extraction algorithm in the related art, a feature extraction module in a convolutional neural network, a feature extraction module in a recurrent neural network, etc. may be used to perform a feature extraction operation on the training image to obtain the second image feature. It should be understood that the training image may also be input into an image encoder to obtain an image embedding of the training image, and the image embedding may be used as the second image feature.

[0068] In step S220 , based on the second image feature, the reference image feature and the second query information, a prediction result of the training wafer die is generated by using an untrained language model.

[0069] The second query information is used to guide the generation of prediction results, and the prediction results are used to describe the defective part of the training wafer. The second query information can be input by the user through an input device, for example, the input device can include a keyboard, a microphone, etc. The second query information can be directly input through the keyboard, or a voice can be input through the microphone, and then the voice is converted into the second query information. The second query information can be used to query the training image related to defects, and the detection results can be used to describe the semantics of the second query information and the relevant content in the training image related to defects.

[0070] The above-mentioned untrained language model may include at least an encoder and a decoder. It should be understood that the developer may also adjust the structure of the language model according to actual needs. For example, a prompt learning module may be added to the language model to implement prompt learning for the language model.

[0071] The second query information may be converted into a second query code (the second query code may be generated by an existing encoding algorithm or other encoder, or by an encoder in an untrained language model). Then, the second image feature, the reference image feature, and the second query code may be input into a decoder after feature processing (e.g., feature concatenation, feature fusion, feature enhancement, etc.) to generate a prediction result of a training wafer grain that conforms to natural language.

[0072] In one example, other relevant information about the training wafer grains may also be determined and converted into a code. Then, the information is input into a decoder after feature processing together with the second image feature, the reference image feature, and the second query code to generate the above-mentioned prediction result. The specific content of the relevant information may depend on the actual situation. For example, the relevant information may include: a text description related to the visual characteristics of the training wafer grains, a text description related to the processing stage of the training wafer grains, etc.

[0073] In step S230 , based on the difference between the prediction result and the actual defect information of the training wafer dies, the model parameters of the untrained language model are adjusted.

[0074] The real defect information of the training wafer die may represent the semantics of the real defect of the training wafer die. For example, if there is a scratch in the pad area of ​​the training wafer die, the real defect information of the training wafer die may be the relevant semantics indicating that there is a scratch in the pad area of ​​the training wafer die. For another example, if there is a transparent foreign body in the surface area of ​​the training wafer die, the real defect information of the training wafer die may be the relevant semantics indicating that there is a transparent foreign body in the surface area of ​​the training wafer die.

[0075] In an example, the difference between the prediction result and the actual defect information of the training wafer grains can be represented by mean square error, cross entropy, etc., and the above difference can also be calculated based on the loss function in the relevant technology, which is not limited in this embodiment of the present invention.

[0076] The difference between the prediction result of the above-mentioned training wafer grain and the actual defect information of the training wafer grain can be used as part of the loss value of the untrained language model, and the training goal of the language model may include minimizing the loss value. Therefore, the language model trained by the above scheme can continuously reduce the difference between the prediction result and the actual defect information, so that the prediction result of the iterative output is closer and closer to the relevant description of the actual defect part of the training wafer grain. It should be understood that the difference between the prediction result and the actual defect information of the training wafer grain can be used as part of the loss value, and the loss value can also include the loss value obtained by other loss functions in the relevant technology, which is not limited in the embodiment of the present invention. The model parameters of the untrained language model can be adjusted by gradient descent, back propagation and other methods to obtain a trained language model. In one example, the untrained language model can also be trained in combination with supervised fine-tuning, reinforcement learning with human feedback and other training methods.

[0077] According to the above scheme of the embodiment of the present invention, a feature extraction operation can be performed on the training image of the training wafer grain to obtain the second image feature. Then, based on the second image feature, the reference image feature and the second query information, the prediction result of the training wafer grain can be generated through the untrained language model. Finally, based on the difference between the prediction result and the real defect information of the training wafer grain, the model parameters of the untrained language model are adjusted. The untrained language model in the above scheme can continuously adjust its own model parameters based on the difference between the prediction result and the real defect information. The trained language model obtained in this way can generate a more accurate detection result of the wafer grain to be tested based on the first image feature, the reference image feature and the first query information of the image to be tested.

[0078] Exemplarily, the above defect detection method may further include steps S310 to S320.

[0079] In step S310 , the reference image is compared with the image to be tested to determine a focus area in the image to be tested.

[0080] The pixels in the image to be tested can be compared with the pixels corresponding to the pixels in the reference image to determine the region of interest in the image to be tested. The difference between the pixel value of at least one pixel in the region of interest and the pixel corresponding to the pixel in the reference image is higher than the preset difference. For example, the region of interest in the image to be tested can be determined by calculating the absolute value of the difference between the pixel value of the pixel in the reference image and the pixel value of the pixel corresponding to the pixel in the reference image. Specifically, for example, the pixel corresponding to each pixel in the image to be tested in the reference image is determined according to the position information of each pixel in the image to be tested and the position information of each pixel in the reference image. The difference between the pixel value of each pixel in the image to be tested and the pixel value of the pixel corresponding to the pixel in the reference image is calculated to obtain the pixel difference of each pixel in the image to be tested. The pixels whose absolute value of the pixel difference in the image to be tested is greater than the preset difference are clustered to obtain at least one cluster region. In one example, the at least one cluster region can be used as the region of interest in the image to be tested. In another example, the cluster region that meets the preset conditions in the at least one cluster region can be used as the region of interest in the image to be tested. The preset difference and the preset conditions can be determined according to actual conditions. For example, the preset condition may be that the area of ​​the cluster region is greater than a preset area value, the cluster region is located in a focus area (such as a pad area, a trimming area) in the image to be tested, etc. The present disclosure does not limit this. For example, the image to be tested includes pixels P1, P2 to P3. n , where n is the total number of pixels in the image to be tested. Subtract the pixel values ​​of each of the above pixels from the pixel values ​​of the pixels at the corresponding position in the reference image. If the pixel at the corresponding position of pixel P1 in the reference image is P1', then subtract the pixel value of pixel P1 from the pixel value of pixel P1' to obtain the pixel difference value X1 corresponding to pixel P1, and so on, the pixel difference value X1 from pixel P1 to pixel P1 can be obtained. n The corresponding pixel difference X1 to pixel difference X n . Traverse pixel difference X1 to pixel difference X n , determine the pixel difference X1 to the pixel difference X in the image to be tested n The absolute value of the pixel difference in the pixel value is greater than the pixel difference corresponding to the preset difference. 10 To pixel difference X 20 And the pixel difference X 40 To pixel difference X 50 When the differences are greater than the preset differences, the pixels P in the image to be tested corresponding to the above pixel differences can be 10 To pixel P 20 And pixel P 40 To pixel P 50Clustering is performed respectively to obtain cluster region Q1 and cluster region Q2. If the preset condition is that the area of ​​the cluster region is greater than the preset area value, then when the area of ​​the cluster region Q1 is greater than the preset area value, the cluster region Q1 is used as the focus region in the image to be tested.

[0081] In step S320, a sampling operation is performed on feature points corresponding to the focus area in the first image feature to obtain a first defect feature.

[0082] The focus area in the image to be tested can be marked as 1, and the non-focus area can be marked as 0 to generate a binary image of the image to be tested. Then, according to the binary image of the image to be tested, the feature points corresponding to the focus area in the first image feature are determined. It should be understood that the first image feature is obtained by feature extraction based on the image to be tested, so there may be a corresponding relationship between the feature points in the first image feature and the pixels in the image to be tested. In combination with the actual scene, if the total number of feature points in the first image feature is less than the total number of pixels in the image to be tested, each feature point in the first image feature may correspond to multiple pixels in the image to be tested. The feature points corresponding to the focus area in the image to be tested can be sampled, and the first defect feature can be obtained according to the sampling result. For example, the feature points corresponding to the focus area in the first image feature are sampled. The average value or weighted average value of the feature points of the sampled first image feature is calculated, and the calculation result is used as the above-mentioned first defect feature. For another example, the feature points corresponding to the focus area in the first image feature are sampled. The feature points obtained by sampling are subjected to feature splicing or feature fusion operation, and the splicing result or fusion result is used as the above-mentioned first defect feature. Optionally, sampling operations may be performed on feature points corresponding to the focus area in the image to be tested by random sampling operations, uniform sampling operations, maximum pooling sampling operations, average pooling sampling operations, importance-based sampling operations, and clustering sampling operations, etc., and the embodiments of the present invention are not limited here. In one example, the first defect feature may be a feature of a preset feature size, so that the trained language model focuses on the feature distribution of the first defect feature, rather than focusing on the influence of different feature sizes of the first defect feature on the feature distribution.

[0083] In this example, the above step S120 generates the detection result of the wafer grain to be tested based on the first image feature, the reference image feature and the first query information through the trained language model, and may include step S121.

[0084] In step S121, based on the first image feature, the reference image feature, the first query information and the first defect feature, a detection result of the wafer to be tested is generated through the trained language model.

[0085] The trained language model may include at least an encoder and a decoder. The encoder may be used to convert the first query information into a first query code. Then, the first query code, the first image feature, the reference image feature, and the first defect feature may be input into the decoder together after feature processing (e.g., feature splicing, feature fusion, feature enhancement, etc.) to generate a detection result of the wafer to be tested that conforms to natural language.

[0086] In one example, other relevant information of the wafer to be tested may also be determined and converted into a code. Then, the information is input into a decoder after feature processing (e.g., feature splicing, feature fusion, feature enhancement, etc.) together with the first query code, the first image feature, the reference image feature, and the first defect feature to generate the above-mentioned detection result. The specific content of the relevant information may depend on the actual situation. For example, the relevant information may include: a text description related to the visual characteristics of the wafer to be tested, a text description related to the processing stage of the wafer to be tested, etc.

[0087] According to the above scheme of the embodiment of the present invention, the reference image can be compared with the image to be tested to determine the area of ​​interest in the image to be tested. Then, the feature points corresponding to the area of ​​interest in the first image feature are sampled to obtain the first defect feature. Finally, based on the first image feature, the reference image feature, the first query information and the first defect feature, the detection result of the wafer grain to be tested is generated through the trained language model. On the one hand, the above scheme can sample the feature points corresponding to the area of ​​interest in the first image feature to obtain the first defect feature, thereby reducing the amount of data that needs to be processed later; on the other hand, the above scheme can integrate the first image feature, the reference image feature, the first query information and the first defect feature, and generate the detection result of the wafer grain to be tested based on the trained language model, thereby improving the degree of attention of the trained language model to the area of ​​interest. The area of ​​interest is usually related to the abnormality in the wafer grain, so the above scheme can improve the accuracy of the detection result.

[0088] Exemplarily, a defective portion of a training wafer die corresponds to a defective region in a training image.

[0089] The defect area can be determined by the relevant steps of step S310 above, or obtained by manual annotation, or generated by a defect area detection model in the related art based on the training image, which is not limited in the embodiment of the present invention. The above defect area can be represented by the coordinates of the vertices of the area in the training image.

[0090] In this example, step S220 may include generating prediction results of training wafer grains based on the second image features, the reference image features and the second query information through an untrained language model, and may include steps S221 to S224.

[0091] In step S221, a sampling operation is performed on feature points corresponding to the defect area in the second image feature to obtain a second defect feature.

[0092] The second image feature is obtained by extracting features based on the image to be tested, so the feature points in the second image feature may correspond to the pixels in the training image. In combination with the actual scene, if the total number of feature points in the second image feature is less than the total number of pixels in the training image, each feature point in the second image feature may correspond to multiple pixels in the training image.

[0093] Optionally, sampling operations may be performed on feature points corresponding to defective areas in the training image by random sampling operations, uniform sampling operations, maximum pooling sampling operations, average pooling sampling operations, importance-based sampling operations, and cluster sampling operations, so as to obtain a second defect feature. It should be understood that the sampling operation in this step may be the same as the sampling operation used in step S320 above. In one example, the second defect feature may be a feature of a preset feature size as described above, so that the trained language model focuses on the feature distribution of the second defect feature, rather than on the influence of different feature sizes of the second defect feature on the feature distribution.

[0094] In step S222, the defect description features are determined based on the actual defect information.

[0095] In one example, the real defect information can be converted into defect description features by an encoder in an untrained language model. In another example, a frozen encoder in another model can be directly used as an encoder in a language model to convert the real defect information into defect description features. In this case, the parameters of the encoder may not be iteratively adjusted. In another example, a feature extraction algorithm in the related art can be used to perform feature extraction operations on the real defect information to obtain defect description features.

[0096] In step S223, a feature fusion operation is performed on the second defect feature and the defect description feature to obtain a fused feature.

[0097] The above-mentioned feature fusion operation may include concatenation, average summation, dot product, etc., which is not limited in the embodiment of the present invention.

[0098] In step S224, based on the second image features, the reference image features, the second query information and the fusion features, a prediction result of the training wafer grain is generated through an untrained language model.

[0099] The untrained language model may include at least an encoder and a decoder. The encoder may be used to convert the second query information into a second query code. Then, the second query code, the second image feature, the reference image feature, and the fused feature may be input into the decoder together after feature processing (e.g., feature concatenation, feature fusion, feature enhancement, etc.) to generate a prediction result of the training wafer grain that conforms to natural language.

[0100] In one example, other relevant information about the training wafer grains may also be determined and converted into a code. Then, the information is input into a decoder after feature processing (e.g., feature splicing, feature fusion, feature enhancement, etc.) together with the second query code, the second image feature, the reference image feature, and the fusion feature to generate the above-mentioned prediction result. The specific content of the relevant information may depend on the actual situation. For example, the relevant information may include: a text description related to the visual characteristics of the training wafer grains, a text description related to the processing stage of the training wafer grains, etc.

[0101] See also Figure 2 , Figure 3 , Figure 2 , Figure 3 FIG. 1 is a schematic diagram showing a training process of a language model according to an embodiment of the present invention. Figure 2 and Figure 3 , the reference image can be passed through the image encoder ( Figure 3 The first image encoder in the image is used to perform feature extraction to obtain the benchmark image features. The training image is passed through the image encoder ( Figure 3 The second image encoder in the text encoder) (the reference image and the training image may use the same or different image encoders) performs a feature extraction operation to obtain a second image feature. The second query information is input into the text encoder to obtain a second query code. The second image feature is sampled through the relevant process of step S221 above (obtained through the regional image encoder, combined with the defective area of ​​the training wafer grain), and the fused feature is obtained through the relevant process of step S223 above (refer to the connection between the regional image encoder and the text encoder). Then, the second query code, the second image feature, the reference image feature and the fused feature are subjected to a feature fusion operation (refer to Figure 3 The feature fusion module in the feature fusion operation is then used to input the features obtained by the feature fusion operation into the untrained language model to obtain the prediction result.

[0102] According to the above scheme of the embodiment of the present invention, the feature points corresponding to the defect area in the second image feature can be sampled to obtain the second defect feature. Then, the defect description feature can be determined based on the real defect information. The second defect feature and the defect description feature are then subjected to feature fusion operation to obtain the fusion feature. Finally, the prediction result of the training wafer grain can be generated based on the second image feature, the reference image feature, the second query information and the fusion feature through the untrained language model. On the one hand, the above scheme can sample the feature points corresponding to the defect area in the second image feature to obtain the second defect feature, thereby reducing the amount of data to be processed later; on the other hand, the above scheme can combine the second image feature, the reference image feature, the second query information and the fusion feature, and generate the prediction result of the training wafer grain based on the untrained language model. Then, the model parameters can be continuously adjusted based on the difference between the prediction result and the real defect information, so as to improve the attention of the trained language model to the defect area, which is conducive to improving the accuracy of the detection result of the wafer grain to be tested.

[0103] Exemplarily, the defective region may be obtained by the following steps: taking the defective pixel corresponding to the defective part of the training wafer in the training image as the center of the circle and drawing a circle with a preset radius as the radius to obtain the defective region.

[0104] By comparing with the reference image, pixels whose pixel value difference is greater than the preset difference value can be regarded as defective pixels. In one example, only one defective pixel can be selected as the center of the circle, and a circle with a preset radius can be drawn, and the circular area can be regarded as the defective area. In combination with the actual scene, the distribution of defective pixels is usually more concentrated, so using a preset radius to draw a circle can cover most defective pixels, and this type of implementation method requires less computing power. In another example, for each defective pixel, the defective pixel is used as the center of the circle, and a circle with a preset radius is drawn as the radius, and the union area of ​​the circular area corresponding to each defective pixel is used as the defective area. In combination with the actual scene, the above method can include all defective pixels obtained by comparison in the defective area, so the representativeness of the defective area can be improved. In addition, since there may be omissions in the generation of defective pixels, and multiple defective pixels are usually more concentrated, the circular area obtained by drawing a circle with defective pixels can, to a certain extent, include defective pixels that were missed after comparison in the above text, so as to further improve the representativeness of the defective area.

[0105] Exemplarily, the defect region may also be obtained by the following steps: determining a circumscribed rectangle of defective pixels corresponding to a defective portion of a training wafer die in a training image as the defect region.

[0106] By comparing with the reference image, pixels whose pixel value difference is greater than a preset difference value can be regarded as defective pixels. Combined with the actual scene, the above method can include all defective pixels obtained by comparison in the defective area, so the representativeness of the defective area can be improved. In addition, since there may be omissions in the generation of defective pixels, and multiple defective pixels are usually concentrated, the circumscribed rectangle of all defective pixels obtained by comparison is used as the defective area, which can include the defective pixels missed after the comparison above to a certain extent, so as to further improve the representativeness of the defective area.

[0107] Exemplarily, the defective region may also be obtained by the following steps: defective pixels corresponding to defective parts of the training wafer die in the training image are directly used as pixels in the defective region.

[0108] By comparing with the reference image, pixels with pixel values ​​greater than a preset difference value can be regarded as defective pixels. Then, the defective pixels can be integrated to obtain the defective area. The above method has lower requirements for computing power and is conducive to improving the generation efficiency of the inspection results of the wafer to be tested.

[0109] It should be understood that the above different methods for obtaining the defective area may be selected one by one or used in combination, and the embodiment of the present invention is not limited thereto.

[0110] According to the above-mentioned scheme provided in the embodiment of the present invention, defect areas can be generated in a variety of ways to improve the flexibility of defect area generation, and then adapt to different categories of defects in wafer grains, which can improve the language model's ability to recognize different categories of defects in training wafer grains, and is beneficial to improving the accuracy of the detection results of the wafer grains to be tested.

[0111] Exemplarily, the second query information may be used to query the defect type of the defective portion of the training wafer die, and the real defect information is used to describe the defect type of the defective portion of the training wafer die.

[0112] In combination with actual scenarios, the second query information may be "what defects exist in this wafer die", and the real defect information may be "there are cracks in the wafer die". By using the above second query information and real defect information, the detection results output by the trained language model can be used to represent the defect type of the defective part of the wafer die to be tested.

[0113] Exemplarily, the second query information may be used to query the defect position of the defective portion of the training wafer die in the training image, and the real defect information may be used to describe the defect position of the defective portion of the training wafer die in the training image.

[0114] The defect position can be represented by coordinates, or by generalized text, such as the upper left corner, the middle, etc. In combination with the actual scenario, the second query information can be "where is the defect position of this wafer grain", and the real defect information can be "the defective part of this wafer grain is located in the upper left corner of the image". In this example, the defect position is the upper left corner. By using the above second query information and the real defect information, the detection result output by the trained language model can be used to indicate the defect position of the defective part of the wafer grain to be tested in the image to be tested.

[0115] Exemplarily, the second query information can be used to query the positional relationship between the defective part of the training wafer die and the functional area in the training wafer die, and the real defect information can be used to describe the positional relationship between the defective part of the training wafer die and the functional area in the training wafer die.

[0116] The above-mentioned functional areas may be areas where the wafer grain performs specific logical functions, such as pad areas, trimming areas, surface areas, etc. In combination with actual scenarios, the second query information may be "whether the defective part of the wafer grain is located in the pad area", and the real defect information may be "the defective part of the wafer grain is located in the pad area". For another example, the second query information may be "where the defective part of the wafer grain is compared to the pad area", and the real defect information may be "the defective part is located above the pad area". By using the above second query information and real defect information, the detection results output by the trained language model can be used to represent the positional relationship between the defective part of the wafer grain to be tested and the functional area of ​​the wafer grain to be tested.

[0117] Exemplarily, the second query information may be used to query the visual characteristics of the defective portion of the training wafer die, and the real defect information may be used to describe the visual characteristics of the defective portion of the training wafer die.

[0118] In combination with actual scenarios, the second query information may be "What are the visual characteristics of the defective part in the wafer die?", and the real defect information may be "The defective part in the wafer die is a stripe defect." By using the above second query information and the real defect information, the detection results output by the trained language model can be used to represent the visual characteristics of the defective part of the wafer die to be tested.

[0119] It should be understood that the above different examples for the second query information and the real defect information may be selected for use in any one or in combination, and the embodiment of the present invention does not limit this.

[0120] According to the above scheme of an embodiment of the present invention, an untrained language model can be trained with the second query information at different query dimensions and the real defect labels corresponding to the second query information, so that the trained language model can generate detection results for the first query information at different query dimensions, which is conducive to adapting to different application scenarios.

[0121] Exemplarily, the second query information is used to query defect information of the training wafer die in a preset area, and the real defect information is used to describe the defect information of the training wafer die in the preset area.

[0122] The above-mentioned preset area may be a specific coordinate or a generalized text description, such as the upper left corner, the middle, etc. of the image. It should be understood that the preset area may be different in each second query information. Defect information can be used to represent defect-related information, such as whether there is a defective part, the defect type of the defective part, etc. In combination with the actual scenario, the second query information may be "whether there is a defective part in the upper left corner", and the real defect information may be "there is a defective part in the upper left corner". By using the above second query information and the real defect information to train the untrained language model, the detection ability of the trained language model for the preset area can be improved, which is conducive to improving the accuracy of the detection results of the wafer grains to be tested. Combined with the detection scenario of the wafer grains to be tested, the trained language model can be enabled to output the defect information in the preset area of ​​the image to be tested when the preset area is included in the first query information.

[0123] Exemplarily, the second query information includes defect information, and the second query information is used to query the position of the part of the training wafer grain that matches the defect information in the training image, and the real defect information is used to describe the position of the part of the training wafer grain that matches the defect information in the training image. In combination with the actual scenario, the second query information may be "where is the position of the crack", and the real defect information may be "the position of the crack is in the upper left corner". In this example, the position of the part that matches the defect information in the training image is the upper left corner. By using the above second query information and real defect information to train the untrained language model, the query ability of the trained language model for the position corresponding to the defect information in the image can be improved, which is conducive to improving the accuracy of the detection results of the wafer grain to be tested. In combination with the detection scenario of the wafer grain to be tested, the trained language model can be enabled to output the position of the part that matches the defect information in the image to be tested when the defect information is included in the first query information.

[0124] It should be understood that the above different examples for the second query information and the real defect information may be selected for use in any one or in combination, and the embodiment of the present invention does not limit this.

[0125] According to the above scheme of an embodiment of the present invention, an untrained language model can be trained with the second query information at different query dimensions and the real defect labels corresponding to the second query information, so that the trained language model can generate detection results for the first query information at different query dimensions, which is conducive to adapting to different application scenarios.

[0126] Exemplarily, the second query information is used to query the defect type and cause of the defective part of the training wafer die, and the real defect information is used to describe the defect type and cause of the defective part of the training wafer die.

[0127] Combined with the actual scenario, the second query information can be "What is the defect type of the defective part and how is it produced", and the real defect information can be "The defect type is a foreign body, which may be caused by dust." By using the above second query information and real defect information to train the untrained language model, the trained language model's query ability for the cause of the defective part can be improved, which is conducive to improving the accuracy of the detection results of the wafer grains to be tested. Combined with the detection scenario of the wafer grains to be tested, the trained language model can output the defect type and cause of the defective part when the first query information is used to query the cause of the defective part.

[0128] Exemplarily, the second query information is used to query the processing method of the defective part of the training wafer die, and the real defect information is used to describe the processing method of the defective part of the training wafer die.

[0129] The above-mentioned processing method can be used to indicate how to process the defective part to repair the defective part. Combined with the actual scenario, the second query information may be "how to remove the foreign matter", and the real defect information may be "the foreign matter can be removed by dry cleaning". By using the above second query information and the real defect information to train the untrained language model, the query capability of the trained language model for the processing method of the defective part can be improved, which is beneficial to improve the accuracy of the detection results of the wafer grains to be tested. Combined with the detection scenario of the wafer grains to be tested, the trained language model can output the processing method of the defective part when the first query information is used to query the processing method of the defective part, so as to guide the user to repair the defective part.

[0130] Exemplarily, the second query information is used to query the wafer processing flow improvement suggestions for the defective part of the training wafer, and the real defect information is used to describe the wafer processing flow improvement suggestions for the defective part of the training wafer.

[0131] Combined with the actual scenario, the second query information may be “how to improve the wafer processing flow to avoid foreign matter in the wafers”, and the real defect information may be “you can keep the workshop clean when processing wafers, or clean the platform carrying the wafers regularly”. By using the above second query information and real defect information to train the untrained language model, the query capability of the trained language model for suggestions for improving the wafer processing flow can be improved, which is beneficial to improving the accuracy of the detection results of the wafers to be tested. Combined with the detection scenario of the wafers to be tested, the trained language model can output suggestions for improving the wafer processing flow when the first query information is used to query suggestions for improving the wafer processing flow, so as to guide users to improve the wafer processing flow and avoid more defective parts in the wafers to be processed.

[0132] It should be understood that the above different examples for the second query information and the real defect information may be selected for use in any one or in combination, and the embodiment of the present invention does not limit this.

[0133] According to the above scheme of an embodiment of the present invention, an untrained language model can be trained with the second query information at different query dimensions and the real defect labels corresponding to the second query information, so that the trained language model can generate detection results for the first query information at different query dimensions, which is conducive to adapting to different application scenarios.

[0134] Exemplarily, the second query information may be used to query the defect position of the defective portion of the first wafer die in the training image, and the real defect information is used to describe that the first wafer die does not have a defective portion.

[0135] The first wafer grain is a training wafer grain without defective parts. The above defect position can be a specific coordinate or a generalized text description, such as the upper left corner, the middle of the image, etc. Combined with the actual scene, the second query information can be "where is the position of the crack in the picture". Since there is no defective part in the first wafer grain, the real defect information can be "there is no crack or other defective part in the wafer grain". Since the above second query information has implied the existence of crack defects in the wafer grain in semantics, it may affect the judgment of the trained language model on the wafer grain that actually does not have defective parts. By using the above second query information and real defect information, the independent detection ability of the trained language model for the defects of the wafer grain can be improved, which is conducive to improving the accuracy of the detection results of the wafer grain to be tested. Combined with the detection scenario of the wafer grain to be tested, the trained language model can be able to output natural language text that the wafer grain to be tested does not have defects without being adversely affected by the first query information when the wafer grain to be tested does not have defects.

[0136] Exemplarily, the second query information is used to query whether the first position of the defective part of the training wafer grain in the training image is correct, and the real defect information is used to describe that the first position is incorrect and the defective position of the defective part of the training wafer grain in the training image.

[0137] The first position is the position of the normal part of the training wafer in the training image. Combined with the actual scene, the second query information may be "whether the foreign matter in the wafer is located in the upper left corner", and the real defect information may be "the foreign matter in the wafer is not located in the upper left corner, but in the lower right corner". In this example, the first position is "lower right corner". Since the above second query information implies in the semantics that the location of the foreign matter defect in the wafer is in the upper left corner, it may affect the trained language model's misjudgment of the location of the defective part. By using the above second query information and the real defect information to train the untrained language model, the independent detection capability of the trained language model for the defects of the wafer can be improved, which is conducive to improving the accuracy of the detection results of the wafer to be tested. Combined with the detection scenario of the wafer to be tested, the trained language model can output the correct location of the defective part in the image to be tested when the wrong location of the defective part is included in the first query information.

[0138] Exemplarily, the second query information may be used to query a first method of a defective portion of a training wafer die, and the true defect information is used to describe that the first method is incorrect and a correct method of a defective portion of a training wafer die.

[0139] The first method is an incorrect processing method for the defective part, and the correct method is a correct processing method for the defective part. Combined with the actual scenario, the second query information may be "Can foreign matter in the wafer grain be soaked with chemical cleaning liquid?" The real defect information may be "The above method is incorrect and should be removed by dry cleaning." In this example, the first method is soaking with chemical cleaning liquid, and the correct method is to remove it by dry cleaning. Since the above second query information semantically implies the use of chemical cleaning liquid to soak to remove foreign matter, it may affect the trained language model's misjudgment of the processing method for the defective part. By using the above second query information and the real defect information to train the untrained language model, the trained language model can improve its independent selection ability for the processing method of the wafer grain defect, which is conducive to improving the accuracy of the detection results of the wafer grain to be tested. Combined with the detection scenario of the wafer grain to be tested, the trained language model can output the correct processing method for the defective part in the wafer grain to be tested when the first query information includes an incorrect processing method for the defective part.

[0140] It should be understood that the above different examples for the second query information and the real defect information may be selected for use in any one or in combination, and the embodiment of the present invention does not limit this.

[0141] According to the above scheme of an embodiment of the present invention, an untrained language model can be trained with the second query information at different query dimensions and the real defect labels corresponding to the second query information, so that the trained language model can generate detection results for the first query information at different query dimensions, which is conducive to adapting to different application scenarios.

[0142] The embodiment of the present invention also provides a defect detection device. Figure 4 FIG. 4 is a schematic block diagram of a defect detection device 400 according to an embodiment of the present invention. Figure 4 As shown, the generating device 400 may include a first image feature extraction module 410 and a detection result generating module 420 .

[0143] The first image feature extraction module 410 is used to perform a feature extraction operation on the image to be tested of the wafer grain to be tested to obtain the first image feature. The detection result generation module 420 is used to generate the detection result of the wafer grain to be tested based on the first image feature, the reference image feature and the first query information through the trained language model, wherein the reference image feature is the feature obtained by the reference image after the feature extraction operation, and the reference wafer grain in the reference image does not have a defective part, and the first query information is used to guide the generation of the detection result, and the detection result is used to describe the defect of the wafer grain to be tested.

[0144] Exemplarily, the detection device 400 further includes: a focus area determination module and a first defect feature determination module.

[0145] The focus area determination module is used to compare the reference image with the image to be tested to determine the focus area in the image to be tested, wherein the difference between the pixel value of at least one pixel located in the focus area and the pixel corresponding to the pixel in the reference image is higher than a preset difference. The first defect feature determination module is used to perform a sampling operation on the feature points corresponding to the focus area in the first image feature to obtain the first defect feature.

[0146] In this example, the detection result generation module 420 may include a first detection result generation module.

[0147] The first detection result generation module is used to generate the detection result of the wafer grain to be tested based on the first image feature, the reference image feature, the first query information and the first defect feature through the trained language model.

[0148] Exemplarily, the detection device 400 further includes: a second image feature extraction module, a first prediction result generation module and a parameter adjustment module.

[0149] The second image feature extraction module is used to perform feature extraction operations on the training image of the training wafer grain to obtain the second image feature. The first prediction result generation module is used to generate the prediction result of the training wafer grain through the untrained language model based on the second image feature, the reference image feature and the second query information, wherein the second query information is used to guide the generation of the prediction result, and the prediction result is used to describe the defective part of the training wafer grain. The parameter adjustment module is used to adjust the model parameters of the untrained language model based on the difference between the prediction result and the actual defect information of the training wafer grain.

[0150] Exemplarily, the defective portion of the training wafer grain corresponds to a defective area in the training image, and the first prediction result generation module may include: a second defect feature generation module, a defect description feature generation module, a fusion feature generation module and a second prediction result generation module.

[0151] The second defect feature generation module is used to perform a sampling operation on the feature points corresponding to the defect area in the second image feature to obtain the second defect feature. The defect description feature generation module is used to determine the defect description feature based on the real defect information. The fusion feature generation module is used to perform a feature fusion operation on the second defect feature and the defect description feature to obtain a fusion feature. The second prediction result generation module is used to generate a prediction result of the training wafer grain based on the second image feature, the reference image feature, the second query information and the fusion feature through an untrained language model.

[0152] Exemplarily, the detection device 400 further includes at least one of the following: a first region generating module, a second region generating module, and a third region generating module.

[0153] The first region generation module is used to draw a circle with a preset radius and a defective pixel corresponding to the defective part of the training wafer grain in the training image as the center of the circle to obtain the defective region. The second region generation module is used to determine the circumscribed rectangle of the defective pixel corresponding to the defective part of the training wafer grain in the training image as the defective region. The third region generation module is used to directly use the defective pixel corresponding to the defective part of the training wafer grain in the training image as the pixel in the defective region.

[0154] Exemplarily, the second query information is used to query the defect position of the defective part of the first wafer die in the training image, and the real defect information is used to describe that the first wafer die does not have the defective part, wherein the first wafer die is a training wafer die without the defective part; and / or

[0155] The second query information is used to query whether the first position of the defective part of the training wafer die in the training image is correct, and the real defect information is used to describe that the first position is incorrect and the defective position of the defective part of the training wafer die in the training image, wherein the first position is the position of the normal part of the training wafer die in the training image; and / or

[0156] The second query information is used to query the first method of the defective part of the training wafer grain, and the real defect information is used to describe that the first method is incorrect and the correct method of the defective part of the training wafer grain, wherein the first method is an incorrect processing method for the defective part, and the correct method is a correct processing method for the defective part.

[0157] Exemplarily, the second query information is used to query the defect type of the defective part of the training wafer die, and the real defect information is used to describe the defect type of the defective part of the training wafer die; and / or

[0158] The second query information is used to query the defect position of the defective part of the training wafer die in the training image, and the real defect information is used to describe the defect position of the defective part of the training wafer die in the training image; and / or

[0159] The second query information is used to query the positional relationship between the defective part of the training wafer die and the functional area in the training wafer die, and the real defect information is used to describe the positional relationship between the defective part of the training wafer die and the functional area in the training wafer die; and / or

[0160] The second query information is used to query the visual characteristics of the defective part of the training wafer die, and the real defect information is used to describe the visual characteristics of the defective part of the training wafer die.

[0161] Exemplarily, the second query information is used to query defect information of the training wafer die in the preset area, and the real defect information is used to describe the defect information of the training wafer die in the preset area; and / or

[0162] The second query information includes defect information, and the second query information is used to query the position of the part of the training wafer grain that matches the defect information in the training image. The real defect information is used to describe the position of the part of the training wafer grain that matches the defect information in the training image.

[0163] Exemplarily, the second query information is used to query the defect type and cause of the defective part of the training wafer die, and the real defect information is used to describe the defect type and cause of the defective part of the training wafer die; and / or

[0164] The second query information is used to query the processing method of the defective part of the training wafer die, and the real defect information is used to describe the processing method of the defective part of the training wafer die; and / or

[0165] The second query information is used to query the wafer processing flow improvement suggestions for the defective part of the training wafer, and the real defect information is used to describe the wafer processing flow improvement suggestions for the defective part of the training wafer.

[0166] According to another aspect of the present invention, an electronic device is also provided. Figure 5 FIG. 5 shows a schematic block diagram of an electronic device 500 according to an embodiment of the present invention. Figure 5 As shown, the electronic device 500 includes a processor 510 and a memory 520. The memory 520 stores a computer program. When the computer program instructions are executed by the processor 510, they are used to execute the above-mentioned defect detection method.

[0167] In addition, according to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, and when the program instructions are executed by a computer or a processor, the computer or the processor executes the corresponding steps of the above-mentioned defect detection method of the embodiment of the present invention, and is used to implement the corresponding modules in the above-mentioned defect detection device or the corresponding modules in the above-mentioned electronic device according to the embodiment of the present invention. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above-mentioned storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media. According to another aspect of the present invention, a computer program product is also provided, including computer program instructions, and when the above-mentioned computer program instructions are executed by a computer or a processor, the computer or the processor executes the corresponding steps of the above-mentioned defect detection method.

[0168] A person skilled in the art may understand the specific implementation scheme of the above-mentioned electronic device and storage medium by reading the above-mentioned description of the defect detection method, which will not be described here for the sake of brevity.

[0169] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.

[0170] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0171] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0172] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some examples, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0173] Similarly, it should be understood that in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

[0174] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0175] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0176] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some modules in the defect detection device according to an embodiment of the present invention. The present invention may also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0177] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0178] The above is only a specific embodiment of the present invention or an explanation of a specific embodiment. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A defect detection method, characterized in that: The method comprises: Performing a feature extraction operation on the image to be tested of the wafer to be tested to obtain a first image feature; Based on the first image feature, the reference image feature and the first query information, the detection result of the wafer grain to be tested is generated through the trained language model, wherein the reference image feature is the feature obtained by the reference image after a feature extraction operation, and the reference wafer grain in the reference image does not have a defective part. The first query information is used to guide the generation of the detection result, and the detection result is used to describe the defective part of the wafer grain to be tested.

2. The method according to claim 1, characterized in that The method further comprises: Comparing a reference image with the image to be tested to determine a region of interest in the image to be tested, wherein a difference between a pixel value of at least one pixel in the region of interest and a pixel corresponding to the pixel in the reference image is greater than a preset difference; Performing a sampling operation on feature points corresponding to the focus area in the first image feature to obtain a first defect feature; The step of generating the detection result of the wafer to be tested based on the first image feature, the reference image feature and the first query information through the trained language model includes: Based on the first image feature, the reference image feature, the first query information and the first defect feature, a detection result of the wafer to be tested is generated through a trained language model.

3. The method according to claim 1, characterized in that The trained language model is obtained by training in the following steps: Performing a feature extraction operation on a training image of a training wafer die to obtain a second image feature; Based on the second image feature, the reference image feature and the second query information, generating a prediction result of the training wafer die through an untrained language model, wherein the second query information is used to guide the generation of the prediction result, and the prediction result is used to describe the defective part of the training wafer die; Based on the difference between the prediction result and the actual defect information of the training wafer particles, the model parameters of the untrained language model are adjusted.

4. The method according to claim 3, characterized in that The defective part of the training wafer corresponds to a defective area in the training image, and the prediction result of the training wafer is generated by an untrained language model based on the second image feature, the reference image feature and the second query information, including: Performing a sampling operation on feature points corresponding to the defect area in the second image feature to obtain a second defect feature; Determining defect description features based on the real defect information; Performing a feature fusion operation on the second defect feature and the defect description feature to obtain a fused feature; Based on the second image feature, the reference image feature, the second query information and the fusion feature, a prediction result of the training wafer grain is generated through an untrained language model.

5. The method according to claim 4, characterized in that The defect area is obtained by any of the following steps: Taking the defective pixel corresponding to the defective part of the training wafer in the training image as the center of the circle and taking the preset radius as the radius, a circle is drawn to obtain the defective area; Determine a circumscribed rectangle of defective pixels corresponding to a defective portion of the training wafer die in the training image as a defective area; The defective pixels corresponding to the defective part of the training wafer in the training image are directly used as pixels in the defective area.

6. The method according to claim 3, characterized in that The second query information is used to query the defect position of the defective part of the first wafer die in the training image, and the real defect information is used to describe that the first wafer die does not have a defective part, wherein the first wafer die is a training wafer die without a defective part; and / or The second query information is used to query whether a first position of the defective part of the training wafer die in the training image is correct, and the real defect information is used to describe that the first position is incorrect and the defective position of the defective part of the training wafer die in the training image, wherein the first position is the position of a normal part of the training wafer die in the training image; and / or The second query information is used to query a first method of the defective part of the training wafer grain, and the real defect information is used to describe that the first method is incorrect, and a correct method of the defective part of the training wafer grain, wherein the first method is an incorrect processing method for the defective part, and the correct method is a correct processing method for the defective part.

7. The method according to claim 3, characterized in that The second query information is used to query the defect type of the defective part of the training wafer die, and the real defect information is used to describe the defect type of the defective part of the training wafer die; and / or The second query information is used to query the defect position of the defective part of the training wafer die in the training image, and the real defect information is used to describe the defect position of the defective part of the training wafer die in the training image; and / or The second query information is used to query the positional relationship between the defective portion of the training wafer die and the functional area in the training wafer die, and the real defect information is used to describe the positional relationship between the defective portion of the training wafer die and the functional area in the training wafer die; and / or The second query information is used to query the visual characteristics of the defective part of the training wafer die, and the real defect information is used to describe the visual characteristics of the defective part of the training wafer die.

8. The method according to claim 3, characterized in that The second query information is used to query the defect information of the training wafer in the preset area, and the real defect information is used to describe the defect information of the training wafer in the preset area; and / or The second query information includes defect information, and the second query information is used to query the position of the part of the training wafer grain that matches the defect information in the training image, and the real defect information is used to describe the position of the part of the training wafer grain that matches the defect information in the training image.

9. The method according to claim 3, characterized in that The second query information is used to query the defect type and cause of the defective part of the training wafer die, and the real defect information is used to describe the defect type and cause of the defective part of the training wafer die; and / or The second query information is used to query the processing method of the defective part of the training wafer die, and the real defect information is used to describe the processing method of the defective part of the training wafer die; and / or The second query information is used to query the wafer processing flow improvement suggestions for the defective part of the training wafer, and the real defect information is used to describe the wafer processing flow improvement suggestions for the defective part of the training wafer.

10. A defect detection device, characterized in that: The device comprises: A first image feature extraction module is used to perform a feature extraction operation on the image to be tested of the wafer to be tested to obtain a first image feature; The detection result generation module is used to generate the detection result of the wafer grain to be tested based on the first image feature, the reference image feature and the first query information through the trained language model, wherein the reference image feature is the feature obtained by the reference image after the feature extraction operation, and the reference wafer grain in the reference image does not have a defective part. The first query information is used to guide the generation of the detection result, and the detection result is used to describe the defect of the wafer grain to be tested.

11. An electronic device, characterized in that: It comprises a memory and a processor, wherein: the memory is used to store a computer program; the processor is used to execute the computer program to implement the defect detection method according to any one of claims 1 to 9.

12. A storage medium storing a computer program / instruction, characterized in that: The computer program / instructions are used to execute the defect detection method according to any one of claims 1 to 9 when running.

13. A computer program product comprising computer program instructions, characterized in that The computer program instructions are used to execute the defect detection method according to any one of claims 1 to 9 when running.

Images