Defect detection method and apparatus, electronic device and storage medium

Abstract

A defect detection method and apparatus, an electronic device and a storage medium. The method comprises: performing feature extraction on an image under test of a wafer die under test, so as to obtain a global image feature of said image (S110); with regard to each preset text, performing feature extraction on the preset text to obtain a preset text feature of the preset text, and performing similarity calculation on the preset text feature of the preset text and the global image feature to obtain the similarity between the preset text feature and the global image feature (S120); and, on the basis of the similarity between each preset text feature and the global image feature, determining a preset text matching said image to serve as a text description related to a defect present in said wafer die (S130). The solution achieves more flexible text descriptions of defects, and can be adapted to actual wafer die detection scenarios of complex and variable defect categories.

Description

Defect detection methods and devices, electronic equipment, and storage media

[0001] This application claims priority to Chinese Patent Application No. 202411880295.6, filed on December 19, 2024, entitled "Method and Apparatus for Defect Detection, Electronic Equipment and Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data processing technology, and more specifically to a defect detection method, a defect detection device, an electronic device, a storage medium, and a computer program product. Background Technology

[0003] In the industrial sector, with the continuous development of production technology, people have placed higher demands on the production quality and efficiency of industrial products. Furthermore, with the development of image processing technology, more and more product inspections are based on images. However, if the detection accuracy is low, the practicality of image-based inspection methods will decrease.

[0004] For example, in the wafer inspection process, automated optical inspection (AOI) equipment is typically used to visually inspect the wafer and determine the types of defects on it. However, different defects on a wafer are highly random and come in many categories. Therefore, how to accurately detect defect types is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This application is made in view of the above-mentioned problems. This application provides a defect detection method, a defect detection device, an electronic device, a storage medium, and a computer program product.

[0006] According to a first aspect of this application, a method for detecting defects is provided, the method comprising:

[0007] Feature extraction is performed on the image of the wafer grains to be tested to obtain the global image features of the image;

[0008] For each preset text, feature extraction is performed on the preset text to obtain the preset text features of the preset text, and the similarity between the preset text features of the preset text and the global image features is calculated to obtain the similarity between the preset text features and the global image features. The preset text is a text description related to the defects existing in the reference wafer.

[0009] Based on the similarity between each preset text feature and the global image features, the preset text that matches the image under test is determined as a text description related to the defects present in the wafer grains under test.

[0010] In one possible implementation, feature extraction is performed on the image of the wafer to be tested to obtain global image features of the image, including:

[0011] The image to be tested is input into the trained image feature extraction module to obtain the global image features of the image to be tested.

[0012] In one possible implementation, the trained image feature extraction module includes multiple image feature extraction sub-modules connected in sequence. The image to be tested is input into the trained image feature extraction module to obtain global image features of the image to be tested, including:

[0013] For each image feature extraction submodule, feature extraction processing is performed on the first intermediate features output by the image to be tested or other image feature extraction submodules connected to the image feature extraction submodule to obtain the first intermediate features output by the image feature extraction submodule, and the first intermediate features output by the last image feature extraction submodule are used as global image features.

[0014] In one possible implementation, the detection method further includes:

[0015] Preset text to be matched against the image to be tested.

[0016] The similarity between the preset text features and each feature point in each first intermediate feature is calculated to obtain the first similarity matrix between the preset text and each first intermediate feature.

[0017] The first similarity matrices of the preset text are fused together to obtain the fused similarity matrix of the preset text;

[0018] Based on the target location information of each target similarity in the fusion similarity matrix of the preset text, the pixels corresponding to each target location are marked in the image to be tested as part of the defect region of the image to be tested, wherein the target similarity is greater than the preset similarity.

[0019] In one possible implementation, the detection method further includes:

[0020] For each preset text,

[0021] The similarity between the preset text features and each feature point in each first intermediate feature is calculated to obtain the second similarity matrix between the preset text and each first intermediate feature.

[0022] The second similarity matrices of the preset text are fused together to obtain the fused similarity matrix of the preset text;

[0023] Based on the similarity between each preset text feature and the global image features, preset text matching the image under test is determined as a text description related to defects present in the wafer grains under test, including:

[0024] For each preset text, the degree of matching between the preset text and the image to be tested is determined based on the similarity between the preset text features and the global image features and the fusion similarity matrix of the preset text.

[0025] Based on the degree of matching between each preset text and the image under test, the preset text that matches the image under test is determined as a text description related to the defects present in the wafer grains under test.

[0026] In one possible implementation, the trained image feature extraction module is obtained through at least the following steps:

[0027] The training image of the training wafer is input into the image feature extraction module that has not been trained to obtain the global image features of the training image. The training image corresponds to the training text, which is a text description related to the defects existing in the training wafer.

[0028] The training text corresponding to the training image is input into the text feature extraction module to obtain the training text features corresponding to the training image;

[0029] Based on the global image features and training text features of the training images, the predicted similarity is determined, and the module parameters of the image feature extraction module that has not been trained are adjusted based on the difference between the predicted similarity and the preset maximum similarity.

[0030] In one possible implementation, the untrained image feature extraction module includes multiple image feature extraction sub-modules connected in sequence. The training image of the training wafer is input to the untrained image feature extraction module to obtain the global image features of the training image, including:

[0031] For each untrained image feature extraction submodule, feature extraction processing is performed on the training image or the second intermediate features output by other untrained image feature extraction submodules connected to the untrained image feature extraction submodule to obtain the second intermediate features output by the untrained image feature extraction submodule, and the second intermediate features output by the last untrained image feature extraction submodule are used as the global image features of the training image.

[0032] In one possible implementation, adjusting the module parameters of the untrained image feature extraction module based on the difference between the predicted similarity and the preset maximum similarity is further achieved through the following steps:

[0033] Based on the training images, a baseline similarity matrix is ​​generated, where the baseline similarity corresponding to the defect area in the baseline similarity matrix is ​​the preset highest similarity, and the baseline similarity corresponding to other areas is the preset lowest similarity.

[0034] The similarity between the training text features and each feature point in each second intermediate feature is calculated to obtain the third similarity matrix between the training text and each second intermediate feature.

[0035] All third similarity matrices of the training text are fused to obtain the fused similarity matrix of the training image;

[0036] The parameters are adjusted based on the difference between the predicted similarity and the preset maximum similarity, and the difference between the fused similarity matrix of the training images and the benchmark similarity matrix.

[0037] In one possible implementation, the preset text includes: a text description related to whether there are defects in the reference wafer, a text description related to the region where defects exist in the reference wafer, and a text description related to the type of defects present in the reference wafer.

[0038] According to a second aspect of this application, a defect detection device is also provided, the device comprising: a first feature extraction module, a matching module, and a defect description module.

[0039] The first feature extraction module is used to extract features from the image of the wafer grains to be tested, so as to obtain the global image features of the image to be tested.

[0040] The matching module is used to extract features from each preset text to obtain the preset text features, and to calculate the similarity between the preset text features and global image features to obtain the similarity between the preset text features and global image features. The preset text is a text description related to the defects present in the reference wafer.

[0041] The defect description module is used to determine the preset text that matches the image under test based on the similarity between each preset text feature and the global image features, so as to serve as a text description related to the defects present in the wafer grains under test.

[0042] According to a third aspect of this application, an electronic device is also provided. The electronic device includes a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, are used to perform the aforementioned defect detection method.

[0043] According to a fourth aspect of this application, a storage medium is also provided. Program instructions are stored on this storage medium, which, when executed, are used to perform the aforementioned defect detection method.

[0044] According to a fifth aspect of this application, a computer program product is also provided. This computer program product includes computer program instructions that, when executed, perform the aforementioned defect detection method.

[0045] According to the above-described scheme of the embodiments of this application, feature extraction can be performed on the image of the wafer to be tested to obtain global image features of the image. Then, for each preset text, feature extraction is performed on the preset text to obtain preset text features, and the similarity between the preset text features and the global image features is calculated to obtain the similarity between the preset text features and the global image features. Finally, based on the similarity between each preset text feature and the global image features, the preset text matching the image to be tested is determined as a text description related to the defects existing in the wafer to be tested. Compared with the method of obtaining defect labels using a defect classification model, the above scheme can describe the defects existing in the wafer to be tested using preset text. Therefore, the text description of defects in the above scheme is more flexible and can be adapted to the actual scenario of complex and varied defect categories in wafer inspection. Attached Figure Description

[0046] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0047] Figure 1 shows a schematic flowchart of a defect detection method according to an embodiment of this application;

[0048] Figure 2 shows a schematic diagram of defects in a wafer grain according to an embodiment of this application;

[0049] Figure 3 shows a schematic diagram of the training process of an image feature extraction module according to an embodiment of this application;

[0050] Figure 4 shows a schematic block diagram of a defect detection apparatus according to an embodiment of this application; and

[0051] Figure 5 shows a schematic block diagram of an electronic device according to an embodiment of the present application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application more apparent, exemplary embodiments according to this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein. Based on the embodiments of this application described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of this application.

[0053] In the field of component manufacturing, packaging and testing defective industrial components (such as wafer dies) would be a waste of manpower and resources. Therefore, it is necessary to inspect industrial components to detect and reject unqualified ones. However, industrial components may have various defects. For example, wafers may have defects such as foreign objects or cracks. Similarly, wafer dies may have defects such as out-of-bounds pin marks, excessive pin marks, or foreign objects. Existing defect detection models can only output a limited number of defect types, making them unsuitable for real-world scenarios with complex and numerous defects in wafers or wafer dies.

[0054] To at least partially address the aforementioned problems, according to a first aspect of this application, embodiments of this application provide a method for detecting defects. Figure 1 shows a schematic flowchart of a defect detection method according to an embodiment of this application. As shown in Figure 1, the method may include the following steps S110 to S130.

[0055] In step S110, feature extraction is performed on the image of the wafer to be tested to obtain the global image features of the image to be tested.

[0056] The image to be tested for the wafer under test can be a static image of the wafer under test or any video frame from a dynamic video of the wafer under test. The image to be tested for the wafer under test can be the original image of the wafer under test acquired by an image acquisition device (e.g., the original image acquired by the image sensor in a camera), or it can be an image obtained after preprocessing the original image (such as digitization, normalization, smoothing, etc.).

[0057] The aforementioned feature extraction operations can be obtained by processing the image of the wafer under test using feature extraction algorithms or feature extraction modules. For example, the image under test can be processed using feature extraction algorithms such as principal component analysis and linear discriminant analysis to obtain the global image features. Alternatively, the image of the wafer under test can be processed using feature extraction modules such as feature extraction layers in convolutional neural networks or recurrent neural networks to obtain the global image features.

[0058] In step S120, for each preset text, feature extraction is performed on the preset text to obtain the preset text features, and the similarity between the preset text features and the global image features is calculated to obtain the similarity between the preset text features and the global image features.

[0059] Referring to Figure 2, Figure 2 illustrates a schematic diagram of defects in a wafer die according to an embodiment of this application. In conjunction with Figure 2, defect 1 may be the presence of foreign matter in the wafer die, and defect 2 may be the presence of pin marks outside the wafer die boundary. The preset text is a text description related to the defects present in the reference wafer die. The preset text may include any text used to describe defects that may exist on the reference wafer die. For example, the preset text may include text indicating the presence of foreign matter in the pad area, text indicating pin marks outside the pad area, text indicating excessive pin marks in the pad area, text indicating scratches in the pad area, text indicating foreign matter in the surface area, text indicating scratches in the surface area, text indicating foreign matter in the trimming area, text indicating scratches in the trimming area, etc. Multiple reference wafer dies with different defects may exist, and preset text describing the defect can be generated for each reference wafer die. It should be understood that the preset text may also include text describing the absence of defects on the reference wafer die.

[0060] The preset text features are obtained by performing feature extraction on the preset text. For example, the preset text can be input into the text encoder in a language model to obtain the text embedding of the preset text, and this text embedding can be used as the preset text features.

[0061] For each preset text, the similarity between the global image features and the preset text features of that preset text can be calculated to obtain the similarity between the global image features and the preset text features. Based on this, the similarity between the global image features and the preset text features of each preset text can be calculated, thus obtaining the similarity between the global image features and the preset text features of each preset text. Taking the calculation of the similarity between the global image features and the preset text features of preset text T1 as an example, for each feature point in the global image features, the similarity between that feature point and the preset text features of preset text T1 can be calculated to obtain the similarity between that feature point and the preset text features of preset text T1. Based on this, the similarity corresponding to each feature point in the global image features can be obtained, forming a similarity matrix. Similarly, the similarity matrix between the global image features and the preset text features of the remaining preset texts can be calculated, thus obtaining the similarity matrix between the global image features and the preset text features of each preset text. It should be understood that the similarity matrix can be adapted to the dimension of the global image features. For example, if the global image features are converted into one-dimensional vectors, the similarity matrix can be converted into a one-dimensional similarity matrix; if the global image features are two-dimensional feature maps, the similarity matrix can also be a two-dimensional similarity matrix. For each preset text, the highest similarity between the preset text features and the global image features in the similarity matrix can be used as the similarity between the global image features and the preset text features. The above similarity can be obtained by calculating Euclidean distance, cosine similarity, etc., and this application embodiment does not impose any limitations on this. It should be understood that similarity calculation operations between the global image features and the preset text features of the preset text can also be performed using attention mechanisms, contrastive learning, etc., and this application embodiment does not impose any limitations on this. In some implementations, the similarity between the global image features and the preset text features can also be determined using deep learning, multimodal contrastive learning, etc.

[0062] In step S130, based on the similarity between each preset text feature and the global image features, a preset text matching the image to be tested is determined as a text description related to the defects present in the wafer grains to be tested.

[0063] The preset text corresponding to the preset text feature with the highest similarity to the global image features can be used as the preset text to match the image under test. The preset text matching the image under test is used as the text description related to defects existing in the wafer under test. For example, taking the similarity matrix in step S120 where each preset text feature corresponds to a global image feature, the similarity matrix S1 corresponds to the preset text feature of preset text T1, the similarity matrix S2 corresponds to the preset text feature of preset text T2, and the similarity matrix S3 corresponds to the preset text feature of preset text T3. The maximum value in similarity matrix S1 is 0.8, the maximum value in similarity matrix S2 is 0.7, and the maximum value in similarity matrix S3 is 0.6. Therefore, the preset text T1 corresponding to similarity matrix S1 is the preset text matching the image under test, and can be used as the preset text matching the image under test. When preset text T1 indicates that the pad area pin marks are outside the boundary, the text description related to defects existing in the wafer under test indicates that the pad area pin marks are outside the boundary. In another example, for each preset text feature, the average of multiple representative similarities in the similarity matrix corresponding to that preset text feature can be calculated, and this average value can be used as the similarity between the preset text and the global image features. This determines the preset text that matches the image to be tested. It should be understood that the preset text with the highest similarity, a preset number of such texts, can also be used as relevant text descriptions of potential defects in the wafer to be tested, and then the actual defects in the wafer to be tested can be determined manually or through other defect detection methods.

[0064] According to the above-described scheme of the embodiments of this application, feature extraction can be performed on the image of the wafer to be tested to obtain global image features of the image. Then, for each preset text, feature extraction is performed on the preset text to obtain preset text features, and the similarity between the preset text features and the global image features is calculated to obtain the similarity between the preset text features and the global image features. Finally, based on the similarity between each preset text feature and the global image features, a preset text matching the image to be tested is determined as a text description related to defects present in the wafer to be tested. Compared to using a defect classification model to obtain defect labels, the above scheme can use preset text to describe defects present in the wafer to be tested. Therefore, the text description of defects in the above scheme is more flexible and can be adapted to the complex and varied defect categories in wafer inspection.

[0065] In one possible implementation, the preset text includes: a text description related to whether there are defects in the reference wafer, a text description related to the region where defects exist in the reference wafer, and a text description related to the type of defects present in the reference wafer.

[0066] In one possible implementation, the preset text may be "In [x] region, [w] defect, [y] present". Here, "In [x] region" corresponds to the text description related to the region where a defect exists in the reference wafer, "[w] defect" corresponds to the text description related to whether a defect exists in the reference wafer, and "[y] present" corresponds to the text description related to the type of defect present in the reference wafer. For example, the text description related to the region where a defect exists in the reference wafer (i.e., "In [x] region") may include: "In [pad] region", "In [non-pad] region", "In [adjustment] region", "In [surface] region", etc. The text description related to whether a defect exists in the reference wafer (i.e., "[w] defect") may include: "[present] defect", "[no] defect", etc. The text description related to the type of defect present in the reference wafer (i.e., "[y] present") may include: "[scratches]", "[transparent foreign matter]", "[opaque foreign matter]", "[pin marks]", etc. This disclosure does not limit the specific content of the text descriptions related to the region where defects exist in the reference wafer, the presence of defects in the reference wafer, and the type of defects present in the reference wafer. It should be understood that the preset text may also include at least one of the above text descriptions, or other text descriptions related to defects, and this application embodiment does not impose any limitations on this.

[0067] In one example, relevant content in the preset text can be added or removed as needed. For instance, the preset text may include visual characteristics of the reference wafer image. Specifically, the preset text may include "No defects in the [non-pad] area" and "This image consists of alternating red and white lines." In this preset text, "In the [non-pad] area" is a text description related to the area where defects exist in the reference wafer. "No defects" is a text description related to whether defects exist in the reference wafer. "This image consists of alternating red and white lines" is a visual characteristic of the reference wafer image.

[0068] According to the above-described scheme of the embodiments of this application, by limiting the preset text, the detection result can include a variety of text descriptions related to defects in the image of the wafer to be tested, thereby describing the defects of the wafer to be tested from different perspectives, which is beneficial to improving the breadth of information in the detection result.

[0069] In one possible implementation, step S110 above, which extracts features from the image of the wafer to be tested to obtain global image features of the image to be tested, may include step S111.

[0070] In step S111, the image to be tested is input into the trained image feature extraction module to obtain the global image features of the image to be tested.

[0071] The image feature extraction module can be an image encoder. Examples include convolutional neural networks, Transformer-based encoders, and hybrid model architectures combining convolutional neural networks and Transformer-based encoders. The trained image feature extraction module can extract features from the test image, and its output can be used as the global image features of the test image.

[0072] According to the above-described scheme of the embodiments of this application, the image to be tested can be input into a trained image feature extraction module to obtain the global image features of the image to be tested. Since the above scheme can obtain the global image features of the image to be tested through the trained image feature extraction module, compared with the method of obtaining the global image features of the image to be tested through an algorithm, the above scheme can enhance the robustness of the global image features.

[0073] In one possible implementation, the trained image feature extraction module includes multiple image feature extraction sub-modules connected in sequence.

[0074] The image feature extraction submodule described above can be a feature extraction module in any machine learning model. For example, it can be a convolutional feature extraction module in a convolutional neural network, a recurrent feature extraction module in a recurrent neural network, or a graph convolutional feature extraction module in a graph neural network. Here, we take the image feature extraction submodule as an example of a feature extraction module in a convolutional feature extraction network. The image feature extraction submodule may include sequentially connected convolutional layers, activation layers, and pooling layers. The convolutional layer convolves the input data of the image feature extraction submodule and then inputs it to the activation layer. The activation layer processes the convolutional input data using a preset activation function to obtain activation data, which is then input to the pooling layer. The pooling layer processes the activation data using max pooling, average pooling, or other methods to serve as the output of the image feature extraction submodule.

[0075] In this example, step S111 above also includes step S1111.

[0076] In step S1111, for each image feature extraction submodule, the first intermediate feature output by the image to be tested or other image feature extraction submodules connected to the image feature extraction submodule is processed to obtain the first intermediate feature output by the image feature extraction submodule, and the first intermediate feature output by the last image feature extraction submodule is used as the global image feature.

[0077] Taking the trained image feature extraction module, which includes image feature extraction submodules block1, block2, and block3, as an example, the image to be tested is input into image feature extraction submodule block1. Image feature extraction submodule block1 performs feature extraction operations on the image to be tested, thereby generating and outputting the first intermediate feature f1 to image feature extraction submodule block2. Refer to the above description of using this image feature extraction submodule to perform feature extraction processing on the image to be tested. Image feature extraction submodule block2 performs feature extraction operations on the first intermediate feature f1, thereby generating and outputting the first intermediate feature f2 to image feature extraction submodule block3. Refer to the above description of using this image feature extraction submodule to perform feature extraction processing on the first intermediate features output by other image feature extraction submodules connected to this image feature extraction submodule. Image feature extraction submodule block3 performs feature extraction operations on the first intermediate feature f2, thereby generating and outputting the first intermediate feature f3. The first intermediate feature f3 output by image feature extraction submodule block3 can be used as the global image feature. Referring to the above, the first intermediate feature output by the last image feature extraction submodule can be used as the global image feature. It should be understood that the feature sizes of the first intermediate features output by different image feature extraction submodules may differ. Therefore, different first intermediate features can be converted into first intermediate features with the same feature size to facilitate subsequent calculations.

[0078] According to the above-described scheme of the embodiments of this application, the image to be tested can be input into sequentially connected image feature extraction sub-modules, so that the last image feature extraction sub-module outputs global image features. The above scheme uses multiple image feature extraction sub-modules to extract features from the image to be tested, which enriches the feature hierarchy of the obtained global image features, thereby improving the representativeness of the global image features and enhancing the accuracy of the detection results.

[0079] In one possible implementation, the method for detecting the above-mentioned defects may include steps S210 to S230.

[0080] In step S210, for the preset text matched to the image to be tested, the preset text features of the preset text are compared with each feature point in each first intermediate feature to obtain the first similarity matrix between the preset text and each first intermediate feature.

[0081] The similarity between the preset text features of the preset text and each feature point in the first intermediate feature output by each image feature extraction submodule can be calculated. For example, taking the image feature extraction submodules block1, block2, and block3 above as examples, the similarity between the preset text features of the preset text and each feature point in the first intermediate feature f1 output by image feature extraction submodule block1 is calculated to obtain the first similarity matrix S10. The similarity between the preset text features of the preset text and each feature point in the first intermediate feature f2 output by image feature extraction submodule block2 is calculated to obtain the first similarity matrix S11. The similarity between the preset text features of the preset text and each feature point in the first intermediate feature f3 output by image feature extraction submodule block3 is calculated to obtain the first similarity matrix S12.

[0082] The aforementioned similarity calculation can be obtained by calculating Euclidean distance, cosine similarity, etc. The above scheme calculates the similarity between each feature point in the first intermediate feature and a preset text feature, obtaining the similarity corresponding to each feature point in the first intermediate feature, and thus obtaining the first similarity matrix of the first intermediate feature. This determines whether it matches the preset text feature. If there are high similarity values ​​in the first similarity matrix, it proves that the feature point corresponding to that similarity value is more consistent with the semantics of the preset text. In practical scenarios, feature points corresponding to high similarity values ​​can be considered as potentially defective feature points.

[0083] In one example, the similarity between each first intermediate feature and a preset text feature can be directly calculated. If the feature sizes of each first intermediate feature are the same, the similarity between each first intermediate feature and the preset text feature can be directly calculated. The similarity calculation result between the first intermediate feature and the preset text feature is then used as the first similarity matrix of that first intermediate feature. The similarity calculation results obtained in this way are of the same size, facilitating subsequent calculations.

[0084] In another example, at least one first intermediate feature can also undergo other feature-related operations to meet actual needs. If there is a first intermediate feature with a different feature size than the other first intermediate features, each first intermediate feature can be converted into an image feature of a preset size, and then a similarity calculation operation can be performed with a preset text feature. The similarity calculation result between the first intermediate feature and the preset text feature is used as the first similarity matrix of the first intermediate feature. If the feature size of the first intermediate feature changes, feature points in the first intermediate feature can be added or deleted. Specifically, if it is necessary to add feature points, the first intermediate feature can be processed using interpolation algorithms in related technologies.

[0085] In another example, multiple first intermediate features can be compared with preset text features to calculate their similarity, and then the similarity calculation results can be converted to a preset size. This embodiment of the application does not impose any limitations on this. The similarity calculation results converted to the preset size are used as the first similarity matrix of the first intermediate feature. If the size of the first similarity matrix changes, data in the first similarity matrix can be added or deleted. Specifically, if it is necessary to add data to the first similarity matrix, the first similarity matrix can be processed using the interpolation algorithm described above.

[0086] In one example, the similarity matrix obtained from the first intermediate feature can be directly used as the first similarity matrix. In another example, for each first intermediate feature, the similarity corresponding to the feature point can be used as a weight to perform a calculation with the feature of that feature point to obtain a new feature for that feature point, which is then used as part of the first similarity matrix corresponding to the first intermediate feature.

[0087] In step S220, for the preset text matched by the image to be tested, all the first similarity matrices of the preset text are fused to obtain the fused similarity matrix of the preset text.

[0088] The first similarity matrices can be fused using methods such as linear averaging and rule-based fusion in related technologies to obtain the fused similarity matrix of the preset text. It should be understood that the value of each similarity in the fused similarity matrix can range from 0 to 1. If the value exceeds this range after fusion, it can be normalized. This will not be elaborated on in the embodiments of this application.

[0089] In step S230, for the preset text matched to the image under test, based on the target location information of each target similarity in the fusion similarity matrix of the preset text, the pixels corresponding to each target location in the image under test are marked as part of the defect region of the image under test.

[0090] The target similarity is greater than the preset similarity. For example, the fused similarity matrix contains similarities a, b, c, d, e, f, g, and h. Since similarities b, c, and d are greater than the preset similarity, they are taken as the target similarities. The target position information of each target similarity in the fused similarity matrix is ​​determined. Based on the correspondence between the position information of each similarity in the fused similarity matrix and the position information of each feature point in the first intermediate feature of the image under test, the feature points corresponding to each target similarity are determined. These feature points can represent feature points in the first intermediate feature of the image under test that conform to the semantics of the preset text. Based on the semantics of the preset text, these feature points can be considered as feature points that may have defects. The pixels corresponding to these feature points in the image under test can be marked as part of the defective region of the image under test.

[0091] According to the above-described scheme of the embodiments of this application, for each image feature extraction submodule's output first intermediate feature, the similarity between each feature point in the first intermediate feature and a preset text feature is determined to obtain a first similarity matrix between the first intermediate feature and the preset text. Then, all first similarity matrices are fused to obtain a fused similarity matrix of the preset text. Finally, based on the target location information of each target similarity in the fused similarity matrix of the preset text, the pixels corresponding to each target location are marked in the image to be tested as part of the defect region of the image to be tested. The above scheme, by determining the defect region of the image to be tested, can improve the breadth of information for defect detection, which is beneficial to improving the comprehensiveness of defect detection.

[0092] In one possible implementation, the method for detecting the above-mentioned defects further includes steps S310 to S320.

[0093] In step S310, for each preset text, the preset text features of the preset text are compared with each feature point in each first intermediate feature to obtain a second similarity matrix between the preset text and each first intermediate feature.

[0094] The preset text features of the preset text can be used to calculate the similarity between each feature point in the first intermediate features output by each image feature extraction submodule. The generation process of the first similarity matrix can be referred to above, and will not be repeated here in the embodiments of this application.

[0095] The above scheme calculates the similarity between each feature point in the first intermediate feature and a preset text feature, obtaining the similarity corresponding to each feature point in the first intermediate feature, and thus obtaining the second similarity matrix of the first intermediate feature. This determines whether it matches the preset text feature. If there are high similarity values ​​in the second similarity matrix, it proves that the feature point corresponding to that similarity value is more consistent with the semantics of the preset text. In practical scenarios, feature points corresponding to high similarity values ​​can be considered as potentially defective feature points. It should be understood that the feature sizes of the first intermediate features output by different image feature extraction submodules may be different. Multiple first intermediate features can be converted into first intermediate features with the same feature size through the relevant process in step S210 above, to facilitate subsequent calculations.

[0096] In step S320, for each preset text, all the second similarity matrices of the preset text are fused to obtain the fused similarity matrix of the preset text.

[0097] The second similarity matrices can be fused using methods such as linear averaging and rule-based fusion from related technologies to obtain the fused similarity matrix of the preset text.

[0098] In this example, step S130 determines the preset text that matches the image under test based on the similarity between each preset text feature and the global image features, as a text description related to the defects present in the wafer grains under test, and may include steps S131 to S132.

[0099] In step S131, for each preset text, the matching degree between the preset text and the image to be tested is determined based on the similarity between the preset text features and the global image features and the fusion similarity matrix of the preset text.

[0100] The matching degree between the preset text and the image to be tested can be determined based on the similarity between the preset text features and the global image features, and the fusion similarity matrix of the preset text. For example, the matching degree between the preset text and the image to be tested can be determined based on the maximum similarity value in the fusion similarity matrix of the preset text and the similarity between the preset text features and the global image features. Specifically, if the maximum similarity value in the fusion similarity matrix of the preset text is 0.8 and the similarity between the preset text features and the global image features is 0.9, the average value of the maximum similarity value in the fusion similarity matrix of the preset text and the similarity between the preset text features and the global image features (i.e., 0.85) can be calculated, and the result can be used as the matching degree between the preset text and the image to be tested. It should be understood that a weighted average value can also be calculated according to actual needs, and this embodiment of the application does not impose any limitations on this.

[0101] In step S132, based on the degree of matching between each preset text and the image to be tested, the preset text that matches the image to be tested is determined as a text description related to the defects present in the wafer grains to be tested.

[0102] The preset text with the highest matching degree between each preset text and the image under test can be used as the preset text for matching the image under test. This preset text is then used as the text description related to the defects present in the wafer under test. For example, if the matching degree between preset text T5 and the image under test is 0.8, preset text T6 is 0.3, preset text T7 is 0.6, and preset text T8 is 0.2, then preset text T5 is used as the preset text for matching the image under test. If preset text T5 is "In the [non-pad] area, there is a defect, with [transparent] [foreign matter]", then the text description related to the defects present in the wafer under test can be "In the [non-pad] area, there is a defect, with [transparent] [foreign matter]".

[0103] According to the above-described scheme of the embodiments of this application, for each preset text, the preset text features of the preset text can be compared with each feature point in each first intermediate feature to obtain a second similarity matrix between the preset text and each first intermediate feature. Then, all the second similarity matrices of the preset text are fused to obtain a fused similarity matrix of the preset text. Furthermore, for each preset text, the matching degree between the preset text and the image to be tested can be determined based on the similarity between the preset text features of the preset text and the global image features and the fused similarity matrix of the preset text. Then, based on the matching degree between each preset text and the image to be tested, the preset text matching the image to be tested is determined as a text description related to defects existing in the wafer grains to be tested. The above scheme can comprehensively determine the preset text matching the image to be tested based on the fused similarity matrix of the preset text and the similarity between the preset text features of the preset text and the global image features, thus improving the representativeness of the matched preset text and improving the detection accuracy of the image to be tested.

[0104] In one possible implementation, the trained image feature extraction module in the above-mentioned defect detection method is obtained at least through steps S410 to S430.

[0105] In step S410, the training image of the training wafer is input into the image feature extraction module that has not yet completed training to obtain the global image features of the training image.

[0106] Each training image corresponds to training text, which is a textual description related to defects present in the training wafer. For example, the training text can be used to describe information related to defects in the training object, such as whether the training object has defects, the defect area of ​​the defects, and the defect type. The specific content of the training text can be set by the developer, and this embodiment does not impose any limitations. In one example, the training text may be the preset text mentioned above.

[0107] In step S420, the training text corresponding to the training image is input into the text feature extraction module to obtain the training text features corresponding to the training image.

[0108] The training text can be input into the text encoder of a language model to obtain the text embedding of the training text, and this text embedding can be used as the training text feature corresponding to the training image. The training text can be associated with defects in the training image. It should be understood that the aforementioned training image can include images with defects or images without defects. The training text associated with the defect type of an image without defects can include a text description describing that the image does not have defects.

[0109] In step S430, based on the global image features and training text features of the training image, the predicted similarity is determined, and based on the difference between the predicted similarity and the preset highest similarity, the module parameters of the image feature extraction module that has not been trained are adjusted.

[0110] Based on the global image features and training text features of the training image, the predicted similarity can be determined through the relevant steps in step S120 above. Since the training text features are obtained from training text related to the training text through feature extraction operations, the similarity between the training text features and the global image features should be the highest. Therefore, the difference between the predicted similarity and the preset highest similarity can be used as the adjustment benchmark for the module parameters of the image feature extraction module that has not yet been fully trained.

[0111] Based on the difference between the predicted similarity and the preset maximum similarity, the module parameters of the untrained image feature extraction module are adjusted. For example, if the preset maximum similarity is 1 and the predicted similarity is 0.8, the module parameters of the untrained image feature extraction module are adjusted according to the difference between the predicted similarity of 0.8 and 1. Specifically, the difference between the predicted similarity and the preset maximum similarity can be used as part of the loss value of the untrained image feature extraction module. The training objective of the image feature extraction module may include minimizing the loss value. Therefore, the image feature extraction module trained by the above scheme can continuously reduce the difference between the predicted similarity and the preset maximum similarity, thereby narrowing the matching degree between the training image and the training text. Therefore, after training, it is possible to accurately match the text related to defects in the test image. In one example, the difference between the predicted similarity and the preset maximum similarity can be represented by mean squared error, cross-entropy, etc. It should be understood that the difference between the predicted similarity and the preset maximum similarity can be used as part of the loss value, which may also include the loss value obtained by other loss functions in related technologies. This application embodiment does not impose any limitations on this. Gradient descent, backpropagation, and other methods can be used to adjust the parameters of the untrained image feature extraction module to obtain a trained module. It should be understood that the training images can also correspond to a defective region labeling matrix; defective regions in the training images can be labeled as 1, and non-defective regions as 0. The difference between this defective region labeling matrix and the similarity matrix can then be used as part of the loss value.

[0112] According to the above-described scheme of the embodiments of this application, the training image of the training wafer can be input into the image feature extraction module that is not yet fully trained to obtain the global image features of the training image. Then, the training text corresponding to the training image is input into the text feature extraction module to obtain the training text features corresponding to the training image. Finally, based on the global image features and training text features of the training image, a predicted similarity is determined, and based on the difference between the predicted similarity and a preset maximum similarity, the module parameters of the image feature extraction module that is not yet fully trained are adjusted. The above scheme can adjust the module parameters of the image feature extraction module that is not yet fully trained based on the difference between the predicted similarity and the preset maximum similarity, so that the trained image feature extraction module can focus on the features of the wafer to be tested that are related to the preset text in the image to be tested, which is beneficial to improving the accuracy of matching the preset text to the image to be tested.

[0113] In one possible implementation, the untrained image feature extraction module includes multiple image feature extraction sub-modules connected in sequence.

[0114] The above step S410 inputs the training image of the training wafer to the image feature extraction module that has not been trained to obtain the global image features of the training image, and may include step S411.

[0115] In step S411, for each untrained image feature extraction submodule, feature extraction processing is performed on the training image or the second intermediate features output by other untrained image feature extraction submodules connected to the untrained image feature extraction submodule to obtain the second intermediate features output by the untrained image feature extraction submodule, and the second intermediate features output by the last untrained image feature extraction submodule are used as the global image features of the training image.

[0116] Referring to Figure 3, Figure 3 illustrates a schematic diagram of the training process of an image feature extraction module according to an embodiment of this application. Referring to Figure 3, training text can be input to a text encoder to obtain training text features. A training image can be input to the image feature extraction module to output second intermediate features through the image feature extraction submodule. The second intermediate features output by the last image feature extraction submodule are used as the global image features of the training image, and a predicted similarity is generated with the training text features. The predicted similarity can be compared with the annotation of the training image (which may include the preset highest similarity mentioned above) to calculate the difference, thereby obtaining a global loss.

[0117] According to the above-described scheme of the embodiments of this application, the training image can be input into sequentially connected image feature extraction sub-modules that have completed training, and the last image feature extraction sub-module that has completed training outputs the global image features. The above scheme uses multiple image feature extraction sub-modules to extract features from the training image, which enriches the feature hierarchy of the obtained global image features, thereby improving the representativeness of the global image features and enhancing the accuracy of the detection results.

[0118] In one possible implementation, step S430 involves adjusting the module parameters of the untrained image feature extraction module based on the difference between the predicted similarity and the preset maximum similarity, which may include steps S431 to S434.

[0119] In step S431, a benchmark similarity matrix is ​​generated based on the training images.

[0120] In the baseline similarity matrix, the baseline similarity corresponding to the defective region can be the preset highest similarity, and the baseline similarity corresponding to other regions can be the preset lowest similarity. For example, to determine the defective and non-defective regions in a training image, pixels located in the defective region are marked as 1, and pixels located in the non-defective region are marked as 0. The marked training image is used as the baseline similarity matrix. As another example, image feature extraction is performed on the training image to obtain training image features. Feature points corresponding to the defective and non-defective regions in the training image features are determined. Feature points located in the defective region are marked as 1, and feature points located in the non-defective region are marked as 0. The marked training image features are used as the baseline similarity matrix.

[0121] In step S432, the similarity between the training text features and each feature point in each second intermediate feature is calculated to obtain the third similarity matrix between the training text and each second intermediate feature.

[0122] The aforementioned similarity calculation can be represented using methods such as Euclidean distance and cosine similarity. The above scheme calculates the similarity between each feature point in the second intermediate feature and the training text features, obtaining the similarity corresponding to each feature point in the second intermediate feature, and thus obtaining the third similarity matrix of the second intermediate feature. Based on this, it can be determined whether the second intermediate feature matches the training text features. If the similarity is high, it can be considered that the feature point is more consistent with the semantics of the training text. In practical scenarios, feature points with high similarity can be considered as feature points that may contain defects in the wafer.

[0123] In step S433, all the third similarity matrices of the training text are fused to obtain the fused similarity matrix of the training image.

[0124] The fused similarity matrix of the training image can be obtained by fusing all the third similarity matrices using methods such as linear averaging and rule-based fusion.

[0125] In step S434, the parameters are adjusted based on the difference between the predicted similarity and the preset highest similarity and the difference between the fusion similarity matrix of the training images and the benchmark similarity matrix.

[0126] The difference between the fused similarity matrix and the reference similarity matrix can be determined based on the correspondence between the position of each similarity in the fused similarity matrix of the training image and the position of each similarity in the reference similarity matrix. For example, if the reference similarity matrix is ​​the training image labeled in step S431, the size of the reference similarity matrix can be adjusted to the same size as the fused similarity matrix. Then, the difference between the adjusted reference similarity matrix (which may be a part labeled in Figure 3) and the fused similarity matrix can be determined as a local loss. The calculation process of the difference between the predicted similarity and the preset highest similarity can refer to the relevant process of step S430 above, and will not be repeated here.

[0127] According to the above scheme of the embodiments of this application, a benchmark similarity matrix can be generated based on the defective and non-defective regions in the training image. Then, the similarity of the training text features with each feature point in each second intermediate feature is calculated to obtain a third similarity matrix between the training text and each second intermediate feature. Then, all the third similarity matrices of the training text are fused to obtain a fused similarity matrix of the training image. Finally, the parameters are adjusted based on the difference between the predicted similarity and the preset maximum similarity, and the difference between the fused similarity matrix of the training image and the benchmark similarity matrix. In the above scheme, the module parameters of the trained image extraction module are adjusted based on the difference between the predicted similarity and the preset maximum similarity, and the difference between the fused similarity matrix of the training image and the benchmark similarity matrix. This allows the trained image feature extraction module to comprehensively consider the matching degree between the test image and the preset text based on the similarity between the defective regions in the test image and the preset text, as well as the global similarity between the test image and the preset text, which helps to improve the accuracy of the matched preset text.

[0128] According to a second aspect of this application, embodiments of this application also provide a defect detection device. FIG4 shows a schematic block diagram of a defect detection device 500 according to an embodiment of this application. As shown in FIG4, the detection device 500 may include: a first feature extraction module 510, a matching module 520, and a defect description module 530.

[0129] The first feature extraction module 510 is used to extract features from the image of the wafer to be tested to obtain global image features. The matching module 520 is used to extract features from each preset text to obtain preset text features, and calculate the similarity between the preset text features and the global image features to obtain the similarity between the preset text features and the global image features. The preset text is a text description related to defects existing in the reference wafer. The defect description module 530 is used to determine the preset text matching the image to be tested based on the similarity between each preset text feature and the global image features, as a text description related to defects existing in the wafer to be tested.

[0130] In one possible implementation, the first feature extraction module 510 may include a global image feature generation module.

[0131] The global image feature generation module is used to input the image to be tested into the trained image feature extraction module to obtain the global image features of the image to be tested.

[0132] In one possible implementation, the trained image feature extraction module may include multiple image feature extraction sub-modules connected in sequence, and the global image feature generation module may include a first intermediate feature generation module.

[0133] The first intermediate feature generation module is used to perform feature extraction processing on the first intermediate features output by the image to be tested or other image feature extraction submodules connected to the image feature extraction submodule for each image feature extraction submodule, so as to obtain the first intermediate features output by the image feature extraction submodule, and use the first intermediate features output by the last image feature extraction submodule as the global image features.

[0134] In one possible implementation, the detection device 500 may further include: a first similarity matrix generation module, a first similarity matrix fusion module, and a defect region determination module.

[0135] The first similarity matrix generation module is used to perform similarity calculations between the preset text features of the preset text and each feature point in each first intermediate feature for the preset text to be matched with the image to be tested, so as to obtain the first similarity matrix between the preset text and each first intermediate feature.

[0136] The first similarity matrix fusion module is used to fuse all the first similarity matrices of the preset text matched by the image to be tested, so as to obtain the fused similarity matrix of the preset text.

[0137] The defect region determination module is used to match the preset text to the image under test. Based on the target location information of each target similarity in the fusion similarity matrix of the preset text, the module marks the corresponding pixels of each target location in the image under test as part of the defect region of the image under test. The target similarity is greater than the preset similarity.

[0138] In one possible implementation, the detection device 500 may further include: a second similarity matrix generation module and a second similarity matrix fusion module.

[0139] For each preset text, the similarity between the preset text features and each feature point in each first intermediate feature is calculated to obtain the second similarity matrix between the preset text and each first intermediate feature.

[0140] For each preset text, all the second similarity matrices of the preset text are fused to obtain the fused similarity matrix of the preset text.

[0141] The defect description module 530 may include: a first matching module and a second matching module.

[0142] The first matching module is used to determine the degree of matching between each preset text and the image under test based on the similarity between the preset text features and the global image features, and the fusion similarity matrix of the preset text. The second matching module is used to determine the preset text that matches the image under test based on the degree of matching between each preset text and the image under test, so as to serve as a text description related to the defects present in the wafer grains under test.

[0143] In one possible implementation, the detection device 500 may further include: a training image feature generation module, a training text feature generation module, and a first difference module.

[0144] The training image feature generation module inputs the training image of the training wafer dies into the untrained image feature extraction module to obtain the global image features of the training image. Each training image corresponds to training text, which is a textual description related to defects present in the training wafer dies. The training text feature generation module inputs the training text corresponding to the training image into the text feature extraction module to obtain the training text features corresponding to the training image. The first difference module determines the predicted similarity based on the global image features and training text features of the training image, and adjusts the module parameters of the untrained image feature extraction module based on the difference between the predicted similarity and the preset maximum similarity.

[0145] In one possible implementation, the untrained image feature extraction module includes multiple image feature extraction sub-modules connected in sequence, and the trained image feature generation module may include a second intermediate feature generation module.

[0146] The second intermediate feature generation module is used to perform feature extraction processing on the training image or the second intermediate features output by other untrained image feature extraction sub-modules connected to the untrained image feature extraction sub-module for each untrained image feature extraction sub-module, so as to obtain the second intermediate features output by the untrained image feature extraction sub-module, and use the second intermediate features output by the last untrained image feature extraction sub-module as the global image features of the training image.

[0147] In one possible implementation, the first difference module further includes: a baseline similarity matrix generation module, a third similarity matrix generation module, a third similarity matrix fusion module, and an adjustment module.

[0148] The baseline similarity matrix generation module generates a baseline similarity matrix based on the training image. In this matrix, the baseline similarity corresponding to defective regions is the preset highest similarity, and the baseline similarity corresponding to other regions is the preset lowest similarity. The third similarity matrix generation module calculates the similarity between each feature point of each second intermediate feature and the training text features, obtaining a third similarity matrix between the training text and each second intermediate feature. The third similarity matrix fusion module fuses all the third similarity matrices of the training text to obtain a fused similarity matrix for the training image. The adjustment module adjusts the parameters based on the difference between the predicted similarity and the preset highest similarity, and the difference between the fused similarity matrix of the training image and the baseline similarity matrix.

[0149] In one possible implementation, the preset text includes: a text description related to whether there are defects in the reference wafer, a text description related to the region where defects exist in the reference wafer, and a text description related to the type of defects present in the reference wafer.

[0150] According to a third aspect of this application, an electronic device is also provided. FIG5 shows a schematic block diagram of an electronic device 600 according to an embodiment of this application. As shown in FIG5, the electronic device 600 includes a processor 610 and a memory 620, wherein the memory 620 stores a computer program, and the computer program instructions are executed by the processor 610 to perform the aforementioned defect detection method.

[0151] Furthermore, according to a fourth aspect of this application, a storage medium is provided, on which program instructions are stored. When the program instructions are executed by a computer or processor, the computer or processor performs corresponding steps of the defect detection method described in the embodiments of this application, and is used to implement corresponding modules in the defect detection apparatus or electronic device described in the embodiments of this application. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media. According to a fifth aspect of this application, a computer program product is also provided, including computer program instructions. When the computer program instructions are executed by a computer or processor, the computer or processor performs corresponding steps of the defect detection method described in the embodiments of this application.

[0152] Those skilled in the art can understand the specific implementation schemes of the above-mentioned electronic devices and storage media by reading the relevant descriptions of the defect detection methods. For the sake of brevity, they will not be described in detail here.

[0153] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

[0154] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0156] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0157] Similarly, it should be understood that, in order to streamline this application and aid in understanding one or more of the various inventive aspects, features of this application may sometimes be grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of this application. However, this approach should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with features fewer than all features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0158] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0159] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0160] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the defect detection apparatus according to embodiments of this application. This application can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0161] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0162] The above description is merely a specific embodiment or illustration of the embodiments of this application. The scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. The scope of protection of this application shall be determined by the scope of the claims.

Claims

1. A method of detecting defects, characterized by, The method includes: Feature extraction is performed on the image of the wafer to be tested to obtain the global image features of the image to be tested; For each preset text, feature extraction is performed on the preset text to obtain the preset text features of the preset text, and the similarity between the preset text features of the preset text and the global image features is calculated to obtain the similarity between the preset text features and the global image features. The preset text is a text description related to the defects present in the reference wafer. Based on the similarity between each preset text feature and the global image features, a preset text matching the image under test is determined as a text description related to defects present in the wafer grains under test.

2. The method of claim 1, wherein, The process of extracting features from the image of the wafer to be tested to obtain global image features includes: The image to be tested is input into the trained image feature extraction module to obtain the global image features of the image to be tested.

3. The method of claim 2, wherein, The trained image feature extraction module includes multiple image feature extraction sub-modules connected in sequence. The step of inputting the image to be tested into the trained image feature extraction module to obtain the global image features of the image to be tested includes: For each image feature extraction submodule, feature extraction processing is performed on the first intermediate feature output by the image to be tested or other image feature extraction submodules connected to the image feature extraction submodule to obtain the first intermediate feature output by the image feature extraction submodule, and the first intermediate feature output by the last image feature extraction submodule is used as the global image feature.

4. The method of claim 3, wherein, The method further includes: For the preset text matched to the image under test, The similarity between the preset text features and each feature point in each first intermediate feature is calculated to obtain the first similarity matrix between the preset text and each first intermediate feature. The first similarity matrices of the preset text are fused together to obtain the fused similarity matrix of the preset text; Based on the target location information of each target similarity in the fusion similarity matrix of the preset text, the pixels corresponding to each target location are marked in the image to be tested as part of the defect region of the image to be tested, wherein the target similarity is greater than the preset similarity.

5. The method of claim 3, wherein, The method further includes: For each preset text, The similarity between the preset text features and each feature point in each first intermediate feature is calculated to obtain the second similarity matrix between the preset text and each first intermediate feature. The second similarity matrices of the preset text are fused together to obtain the fused similarity matrix of the preset text; The step of determining the preset text matching the image under test based on the similarity between each preset text feature and the global image features, and using it as a text description related to defects present in the wafer under test, includes: For each preset text, the degree of matching between the preset text and the image to be tested is determined based on the similarity between the preset text features and the global image features and the fusion similarity matrix of the preset text. Based on the degree of matching between each preset text and the image under test, the preset text that matches the image under test is determined as a text description related to the defects present in the wafer grains under test.

6. The method of claim 2, wherein, The trained image feature extraction module is obtained through at least the following steps: The training image of the training wafer is input into the image feature extraction module that has not been trained to obtain the global image features of the training image. The training image corresponds to training text, which is a text description related to the defects present in the training wafer. The training text corresponding to the training image is input into the text feature extraction module to obtain the training text features corresponding to the training image; Based on the global image features of the training images and the training text features, a predicted similarity is determined, and based on the difference between the predicted similarity and the preset maximum similarity, the module parameters of the image feature extraction module that has not been fully trained are adjusted.

7. The method of claim 6, wherein, The untrained image feature extraction module includes multiple image feature extraction sub-modules connected in sequence. The step of inputting the training image of the training wafer into the untrained image feature extraction module to obtain the global image features of the training image includes: For each untrained image feature extraction submodule, feature extraction processing is performed on the training image or the second intermediate features output by other untrained image feature extraction submodules connected to the untrained image feature extraction submodule to obtain the second intermediate features output by the untrained image feature extraction submodule, and the second intermediate features output by the last untrained image feature extraction submodule are used as the global image features of the training image.

8. The method of claim 7, wherein, The adjustment of the module parameters of the untrained image feature extraction module based on the difference between the predicted similarity and the preset maximum similarity is further achieved through the following steps: Based on the training images, a benchmark similarity matrix is ​​generated, wherein the benchmark similarity corresponding to the defect region in the benchmark similarity matrix is ​​the preset highest similarity, and the benchmark similarity corresponding to other regions is the preset lowest similarity. The similarity between the training text features and each feature point in each second intermediate feature is calculated to obtain the third similarity matrix between the training text and each second intermediate feature. The third similarity matrices of the training text are fused to obtain the fused similarity matrix of the training image; The parameters are adjusted based on the difference between the predicted similarity and the preset maximum similarity, and the difference between the fused similarity matrix of the training images and the benchmark similarity matrix.

9. The method of claim 1, wherein, The preset text includes: text descriptions related to whether there are defects in the reference wafer, text descriptions related to the regions in the reference wafer where defects exist, and text descriptions related to the types of defects present in the reference wafer.

10. A defect detection apparatus characterized by comprising: The device includes: The first feature extraction module is used to extract features from the image of the wafer to be tested, so as to obtain the global image features of the image to be tested. The matching module is used to extract features from each preset text to obtain preset text features, and to calculate the similarity between the preset text features and the global image features to obtain the similarity between the preset text features and the global image features. The preset text is a text description related to defects existing in the reference wafer. The defect description module is used to determine the preset text that matches the image under test based on the similarity between each preset text feature and the global image feature, so as to serve as a text description related to the defects present in the wafer grains under test.

11. An electronic device, comprising: The device includes a memory and a processor, wherein: the memory is used to store a computer program; and the processor is used to execute the computer program to implement the defect detection method as described in any one of claims 1-9.

12. A storage medium storing a computer program / instructions, characterized by The computer program / instructions are used to perform the defect detection method as described in any one of claims 1-9 when the program is executed.

13. A computer program product comprising computer program instructions, characterised in that, The computer program instructions, when executed, are used to perform the defect detection method as described in any one of claims 1-9.

Images