Method and device for generating defect category and defect description based on wafer graph

Abstract

The invention relates to a method and a device for generating defect categories and defect descriptions based on a wafer graph. The method comprises the following steps: acquiring a defect view, an amplified view and a top view of a target wafer; respectively inputting the defect view, the magnified view and the top view into an image encoder of a CLIP model to obtain a first image representation, a second image representation and a third image representation respectively corresponding to the defect view, the magnified view and the top view; fusing the first image representation, the second image representation and the third image representation to obtain a fused image representation; a text encoder of the CLIP model is used for encoding a plurality of preset defect tags to obtain a plurality of tag text representations; respectively carrying out similarity comparison on the fused image representation and each label text representation, and determining a defect category of the target wafer according to a similarity comparison result; and inputting the fused image representation into a pre-trained LSTM model to obtain a first defect description of the target wafer.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor manufacturing, and in particular to a method and device for generating defect categories and defect descriptions based on a wafer image. Background Art

[0002] Wafer is an indispensable core material in semiconductor manufacturing. It is made of silicon or other semiconductor materials in the form of a round thin sheet, and is mainly used to produce integrated circuit chips (die). Wafers must have extremely high purity and flatness. However, during the semiconductor production process, defects such as scratches and voids may appear on the surface of the wafer due to various factors. These defects may have an adverse effect on the performance and reliability of the final chip. Therefore, detecting and analyzing wafer defects is crucial to discovering problem links and equipment abnormalities in the production process.

[0003] Traditional defect classification methods rely on manual inspection or single-angle image processing technology, which can only provide classification information but not other effective descriptive information, which limits the effectiveness and automation of detection algorithms in defect analysis and diagnosis. At the same time, single-angle image processing makes it difficult to detect wafer defects more comprehensively. Therefore, a method is needed to better classify defects on wafers and generate effective descriptive information. Summary of the invention

[0004] The present invention describes a method and device for generating defect categories and defect descriptions based on a wafer image, which can more accurately classify wafer defects and generate detailed defect description information for wafer defects.

[0005] In a first aspect, a method for generating defect categories and defect descriptions based on a wafer image is provided, comprising: Obtain defect view, magnified view and top view of target wafer; The defect view, the magnified view and the top view are respectively input into the image encoder of the CLIP model to obtain the first image representation, the second image representation and the third image representation corresponding to each other; the CLIP model is fine-tuned by a sample set including a sample wafer image and its corresponding sample defect label; fusing the first image representation, the second image representation, and the third image representation to obtain a fused image representation; Use the text encoder of the CLIP model to encode multiple preset defect labels respectively to obtain multiple label text representations; Comparing the fused image representation with each label text representation for similarity, and determining the defect category of the target wafer according to the similarity comparison result; The fused image representation is input into a pre-trained LSTM model to obtain a first defect description of the target wafer.

[0006] In a possible implementation, fusing the first image representation, the second image representation, and the third image representation to obtain a fused image representation includes: The first image representation, the second image representation and the third image representation are spliced ​​and input into a multi-layer perceptron to obtain a fused image representation.

[0007] In a possible implementation, fusing the first image representation, the second image representation, and the third image representation to obtain a fused image representation includes: A pooling operation is performed on the first image representation, the second image representation, and the third image representation to obtain a fused image representation.

[0008] In a possible implementation, a text encoder of the CLIP model is used to encode a plurality of preset defect labels respectively to obtain a plurality of label text representations, including: Fill the label text of each defect label into the preset prompt word template respectively to obtain the defect label prompt word corresponding to each defect label; Input each defect label prompt word into the text encoder of the CLIP model respectively to obtain the label text representation corresponding to each defect label.

[0009] In a possible implementation, the fused image representation and the label text representation have the same dimension; and the fused image representation and each label text representation are respectively compared for similarity, and the defect category of the target wafer is determined according to the similarity comparison result, including: The cosine similarities between the fused image representation and each label text representation are calculated respectively, and the defect label corresponding to the label text representation with the largest cosine similarity value is determined as the defect category of the target wafer.

[0010] In a possible implementation, the method further includes: The first defect description is input into a large language model, and the large language model is enabled to adjust the text content of the first defect description to obtain a second defect description.

[0011] In a possible implementation, the CLIP model and the LSTM model are trained by the following steps: Acquire a sample set, including a plurality of sample wafer images containing defects and their corresponding sample defect labels, and sample defect descriptions for the sample wafer images; the sample wafer images include a sample defect view, a sample magnified view, and a sample top view of the sample wafer; Inputting the sample defect view, the sample magnified view and the sample top view into the image encoder of the CLIP model respectively, to obtain the fourth image representation, the fifth image representation and the sixth image representation respectively; fusing the fourth image representation, the fifth image representation and the sixth image representation to obtain a sample fused image representation; Determining a first training loss of a CLIP model according to the sample fusion image representation, the sample defect label, and the label text representation of each defect label; Determining a second training loss of the LSTM model according to the sample fusion image representation and the sample defect description; According to the total training loss determined by the first training loss and the second training loss, parameter values ​​of the CLIP model and the LSTM model are adjusted.

[0012] In a second aspect, a device for generating defect categories and defect descriptions based on a wafer image is provided, comprising: An acquisition unit configured to acquire a defect view, a magnified view, and a top view of a target wafer; An image encoding unit is configured to input the defect view, the magnified view and the top view into an image encoder of a CLIP model respectively to obtain a first image representation, a second image representation and a third image representation corresponding to each other; the CLIP model is fine-tuned by a sample set including a sample wafer image and its corresponding sample defect label; a feature fusion unit configured to fuse the first image representation, the second image representation and the third image representation to obtain a fused image representation; A label encoding unit is configured to respectively encode a plurality of preset defect labels using a text encoder of the CLIP model to obtain a plurality of label text representations; A defect category generating unit is configured to compare the fused image representation with each label text representation respectively for similarity, and determine the defect category of the target wafer according to the similarity comparison result; The defect description generating unit is configured to input the fused image representation into a pre-trained LSTM model to obtain a first defect description of the target wafer.

[0013] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method of the first aspect.

[0014] In a fourth aspect, a computing device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method of the first aspect is implemented.

[0015] The present invention proposes a method and device for generating defect categories and defect descriptions based on wafer images. The method first obtains a defect view, an enlarged view, and a top view of the wafer, and then encodes the three views using a fine-tuned CLIP model (Contrastive Language-Image Pre-training) image encoder to obtain image representations of the three views. After the image representations of the three views are fused, a fused image representation is obtained. The use of fused image representations enables the model to integrate wafer images from multiple angles for defect recognition, so as to improve the accuracy of defect classification and defect description.

[0016] At the same time, the text encoder of the CLIP model is used to encode the preset defect labels to obtain several label text representations. The fused image representation is used to compare the similarity with each label text representation to determine the defect category of the wafer image. The fused image representation is input into the LSTM model (Long Short-Term Memory) to obtain the defect description generated by the LSTM model.

[0017] The solution of the present invention fuses multi-view wafer images to more accurately classify wafer defects and also generates a wafer defect description based on the fused image representation to help engineers better understand the wafer defect situation and optimize the semiconductor production process in a targeted manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the multiple embodiments disclosed in this specification, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only the multiple embodiments disclosed in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flowchart of a method for generating defect categories and defect descriptions based on a wafer image disclosed in an embodiment of the present invention; Figure 2 A flowchart of a method for generating defect categories and defect descriptions based on a wafer image disclosed in an embodiment of the present invention; Figure 3 A flowchart of the multi-task learning method disclosed in an embodiment of the present invention; Figure 4 A schematic block diagram of a device for generating defect categories and defect descriptions based on a wafer image disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] As mentioned earlier, in the semiconductor industry, wafer quality control is the core link in the entire production process. Any defects on the surface of the wafer may cause performance failures or reliability problems in the final product. Current defect detection methods mainly rely on manual visual inspection or single-angle image analysis, which causes a large amount of valuable image information from different perspectives to be ignored. Specifically, for the same wafer, the semiconductor machine can shoot it from multiple angles to obtain multiple wafer images. These mainly include defect views, zoomed views, and top views.

[0022] The defect view is an overall view of the detected defect area, which marks all defect locations on the wafer detected by the semiconductor machine. The magnified view is an enlarged view of the specific defect location marked in the defect view. When a suspicious area is found in the defect view, a higher magnification microscope is used to magnify the area for detailed observation of the specific shape, size and characteristics of the defect. The top view is a top view of the wafer surface, corresponding to the top view of the magnified view. Compared with the magnified view and the top view, the top view takes a wafer image from another angle, which can more comprehensively display the shape of the defect.

[0023] In addition to the fact that information from multi-angle views is not fully utilized, related technical solutions also fail to generate corresponding defect descriptions for defects on the wafer map, which makes it difficult for semiconductor engineers to obtain more detailed information about wafer defects.

[0024] In order to solve the above problems, the present invention proposes a method for generating defect categories and defect descriptions based on wafer images. Figure 1 A flowchart of a method for generating defect categories and defect descriptions based on a wafer image is disclosed in an embodiment of the present invention. Figure 1 In the example of , first, the defect view, magnified view, and top view of the wafer are respectively input into the image encoder of the CLIP model to obtain the corresponding image representation 1, image representation 2, and image representation 3. Then, image representation 1, image representation 2, and image representation 3 are fused to obtain a fused image representation.

[0025] The defect labels of each wafer defect prepared in advance are input into the text encoder of the CLIP model to obtain the label text representation corresponding to each defect label. Then the fused image representation is compared with each label text representation for similarity, and the defect label corresponding to the label text representation with the highest similarity is determined as the defect category of the wafer. At the same time, the fused image representation is input into the pre-trained LSTM model to obtain the defect description for the wafer defect output by the model.

[0026] The multi-task training process for the CLIP model and LSTM model will be described in detail in subsequent steps.

[0027] The following describes the specific implementation steps of the method for generating defect categories and defect descriptions based on wafer images in conjunction with specific embodiments.

[0028] Figure 2 The flowchart of a method for generating defect categories and defect descriptions based on wafer images disclosed in an embodiment of the present invention is as follows. Figure 2 As shown, the method at least includes: step 201, obtaining a defect view, an enlarged view and a top view of a target wafer; step 202, inputting the defect view, the enlarged view and the top view into the image encoder of the CLIP model respectively, and obtaining the first image representation, the second image representation and the third image representation corresponding to each other; the CLIP model is trained by a sample set including a sample wafer image and its corresponding sample defect label; step 203, fusing the first image representation, the second image representation and the third image representation to obtain a fused image representation; step 204, using the text encoder of the CLIP model to encode a plurality of preset defect labels respectively, and obtaining a plurality of label text representations; step 205, comparing the similarity between the fused image representation and each label text representation respectively, and determining the defect category of the target wafer according to the similarity comparison result; step 206, inputting the fused image representation into the pre-trained LSTM model to obtain the first defect description of the target wafer.

[0029] The specific execution process of each of the above steps is described below.

[0030] First, in step 201, a defect view, a magnified view, and a top view of a target wafer are acquired.

[0031] The defect view of the target wafer can be obtained by photographing the target wafer when the semiconductor machine detects that the target wafer has defects. The enlarged view can be obtained by further enlarging the defect position on the defect view. The top view can be obtained by using the same magnification and shooting coordinates as the enlarged view, but using a different shooting angle (top view).

[0032] Then, in step 202, the defect view, the magnified view and the top view are respectively input into the image encoder of the CLIP model to obtain the first image representation, the second image representation and the third image representation corresponding to each other; the CLIP model is trained by a sample set including sample wafer images and their corresponding sample defect labels.

[0033] The first image representation may correspond to the image representation of the defect view; the second image representation may correspond to the image representation of the enlarged view; and the third image representation may correspond to the image representation of the top view.

[0034] The CLIP model is a pre-trained multimodal model that is pre-trained with a large amount of paired image and text data to learn the alignment relationship between images and text. By using the contrastive learning method, the text and image are embedded in a common semantic space, so that the relevant text descriptions and image content are close to each other in this space, while the irrelevant ones are far away.

[0035] Due to the above characteristics of the CLIP model, the text encoder and image encoder of the CLIP model can be used to classify images. Since the CLIP model is only a pre-trained model, it is not proficient in the image and text knowledge related to the semiconductor field. Therefore, in some possible factual methods, before using the CLIP model to classify wafer defects, it is necessary to fine-tune the CLIP model using a sample set containing wafer defects to improve the accuracy of the CLIP model in wafer defect classification. The specific method of training the CLIP model will be explained in detail in the subsequent steps.

[0036] In step 202 , each image representation may be in the form of an embedding vector.

[0037] Next, in step 203, the first image representation, the second image representation and the third image representation are fused to obtain a fused image representation.

[0038] Since the defect view, the magnified view and the top view are essentially the same wafer defect photographed at different angles and different zoom factors, their corresponding image representations can be fused, and the fused representation has the image features of the three views. A variety of methods can be used to fuse the first image representation, the second image representation and the third image representation.

[0039] In one embodiment, step 203 includes: The first image representation, the second image representation and the third image representation are spliced ​​and input into a multi-layer perceptron to obtain a fused image representation.

[0040] The first image representation, the second image representation and the third image representation are spliced ​​to obtain a spliced ​​image representation, and then the spliced ​​image representation is input into a multi-layer perceptron, and after being processed and compressed by the multi-layer perceptron, a fused image representation is obtained.

[0041] In another embodiment, step 203 includes: A pooling operation is performed on the first image representation, the second image representation, and the third image representation to obtain a fused image representation.

[0042] The pooling operation may be any pooling operation, such as average pooling, maximum pooling, etc., which is not limited here.

[0043] When the pooling operation is average pooling, the first image representation, the second image representation, and the third image representation are averaged bit by bit to obtain an averaged fused image representation.

[0044] When the pooling operation is maximum pooling, the first image representation, the second image representation, and the third image representation are bitwise maximized to obtain an averaged fused image representation.

[0045] It can be understood that the dimension of the fused image representation after fusion is the same as the dimension of the individual image representations before fusion.

[0046] After obtaining the fused image representation that can represent the original wafer, in step 204, a text encoder of the CLIP model is used to encode the preset multiple defect labels respectively to obtain multiple label text representations.

[0047] You can pre-set multiple defect labels, such as crack, short circuit, open circuit, etc., and then use the text encoder of the CLIP model to encode them separately to obtain the label text representation corresponding to each defect label. Similar to the image representation, the label text representation is also in the form of an embedded vector.

[0048] The text encoder of the CLIP model receives descriptive text for the image content. Therefore, before the defect labels are input into the text encoder of the CLIP model, they need to be put into the preset prompt word templates to form their corresponding prompt words, and then input into the text encoder of the CLIP model.

[0049] In one embodiment, step 204 includes steps 2041 and 2042 .

[0050] In step 2041, the label texts of the defect labels are respectively filled into the preset prompt word templates to obtain the defect label prompt words corresponding to the defect labels.

[0051] The prompt word template can be a piece of text containing placeholders, and the placeholders are used to fill in the defect label. For example, in one example, the prompt word template can be: "This is a [defect type] defect." Among them, [defect type] is a placeholder for filling in a specific defect label to form a defect label prompt word. For example, fill the defect label "crack" into the prompt word template to obtain the defect label prompt word "This is a crack defect." Then, in step 2042, each defect label prompt word is input into the text encoder of the CLIP model to obtain a label text representation corresponding to each defect label.

[0052] Next, in step 205, the fused image representation is compared with each label text representation in terms of similarity, and the defect category of the target wafer is determined according to the similarity comparison result.

[0053] In one embodiment, the fusion image representation has the same dimension as the label text representation. Step 205 includes: The cosine similarities between the fused image representation and each label text representation are calculated respectively, and the defect label corresponding to the label text representation with the largest cosine similarity value is determined as the defect category of the target wafer.

[0054] In addition to using cosine similarity to calculate similarity, other similarity measurement methods such as Euclidean similarity may also be used, which are not limited here.

[0055] Finally, in step 206, the fused image representation is input into a pre-trained LSTM model to obtain a first defect description of the target wafer.

[0056] Compared with using the training set to fine-tune the Large Language Model (LLM) to generate defect descriptions, training the LSTM model is cheaper and saves more computing resources.

[0057] In some possible implementations, the LSTM model may be affected by the training data when generating descriptions, which may sometimes generate repetitive or grammatically / semantically incoherent content. To solve this problem, after the description generated by the LSTM, it is post-processed by a large language model to improve the accuracy and coherence of the description. In this implementation, the method further includes: The first defect description is input into a large language model, and the large language model is enabled to adjust the text content of the first defect description to obtain a second defect description.

[0058] The large language model may be instructed to adjust the text content of the first defect description by using a specific prompt word template, for example: "Please polish the following text about the semiconductor wafer defect description to make it semantically coherent: [defect description text]". [defect description text] is a placeholder for filling in the first defect description. Then, the prompt word after filling in the first defect description is input into the large language model to obtain a more accurate second defect description.

[0059] You can use any pre-trained large language model without any limitation.

[0060] In some possible implementations, a multi-task learning approach can be used to train the CLIP model and the LSTM model simultaneously. In this implementation, the CLIP model and the LSTM model are trained by Figure 3 steps for training. Figure 3 The present invention is a flowchart of a multi-task learning method disclosed in an embodiment of the present invention.

[0061] In step 301, a sample set is obtained, which includes a plurality of sample wafer images containing defects and their corresponding sample defect labels, and sample defect descriptions for the sample wafer images; the sample wafer images include a sample defect view, a sample magnified view and a sample top view of the sample wafer.

[0062] In step 302, the sample defect view, the sample magnified view and the sample top view are respectively input into the image encoder of the CLIP model to obtain the corresponding fourth image representation, fifth image representation and sixth image representation.

[0063] In step 303, the fourth image representation, the fifth image representation and the sixth image representation are fused to obtain a sample fused image representation.

[0064] The specific implementation methods of step 301 to step 303 may refer to the implementation methods of step 201 to step 203 respectively, and are not limited here.

[0065] In step 304, a first training loss of the CLIP model is determined based on the sample fusion image representation, the sample defect label, and the label text representation of each defect label.

[0066] In step 305, a second training loss of the LSTM model is determined according to the sample fusion image representation and the sample defect description.

[0067] The methods for calculating the training loss in step 304 and step 305 can refer to the conventional training loss calculation methods for training the CLIP model and training the LSTM model, respectively, and will not be described in detail here.

[0068] In step 306, the parameter values ​​of the CLIP model and the LSTM model are adjusted according to the total training loss determined by the first training loss and the second training loss.

[0069] The total training loss is determined by weighted summing of the first training loss and the second training loss. Then the parameters of the CLIP model and the LSTM model are adjusted based on the gradient descent method to train the two models.

[0070] The present invention proposes a method for generating defect categories and defect descriptions based on wafer images. In the feature extraction stage, features are extracted from multiple perspectives (defect view, magnified view, top view), and new fused features are formed by multi-perspective feature fusion. The results of multi-perspective fused visual features are used in combination with text features for classification, thereby improving the classification accuracy of the CLIP model.

[0071] Through feature reuse, fused image features can be used for classification and description generation; at the same time, through multi-task learning, image classification and description generation are performed simultaneously, and description labels can be used for CLIP fine-tuning and LSTM training at the same time, reducing the waste of computing resources and enabling the model to understand defects more comprehensively.

[0072] By using LSTM to generate defect descriptions and using a large language model to optimize the descriptions, the accuracy and coherence of the descriptions can be guaranteed while saving resources.

[0073] According to another embodiment, a device for generating defect categories and defect descriptions based on wafer images is also provided. FIG. 4 is a schematic block diagram of a device for generating defect categories and defect descriptions based on wafer images disclosed in an embodiment of the present invention. The device can be deployed in any device, platform or device cluster with computing and processing capabilities. Figure 4 As shown, the device 400 includes: An acquisition unit 401 is configured to acquire a defect view, an enlarged view, and a top view of a target wafer; The image encoding unit 402 is configured to input the defect view, the magnified view and the top view into an image encoder of a CLIP model respectively to obtain a first image representation, a second image representation and a third image representation corresponding to each other; the CLIP model is fine-tuned by a sample set including a sample wafer image and its corresponding sample defect labels; A feature fusion unit 403 is configured to fuse the first image representation, the second image representation and the third image representation to obtain a fused image representation; The label encoding unit 404 is configured to respectively encode a plurality of preset defect labels using a text encoder of the CLIP model to obtain a plurality of label text representations; The defect category generating unit 405 is configured to compare the similarity between the fused image representation and each label text representation, and determine the defect category of the target wafer according to the similarity comparison result; The defect description generating unit 406 is configured to input the fused image representation into a pre-trained LSTM model to obtain a first defect description of the target wafer.

[0074] In some possible implementations, the device 400 further includes: The second defect description generating unit 407 is configured to input the first defect description into a large language model, and enable the large language model to adjust the text content of the first defect description to obtain a second defect description.

[0075] According to another aspect of the embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in any of the above embodiments.

[0076] According to yet another embodiment, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in any one of the above embodiments is implemented.

[0077] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0078] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0079] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0080] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0081] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating defect categories and defect descriptions based on a wafer image, characterized in that: include: Obtain defect view, magnified view and top view of target wafer; The defect view, the magnified view and the top view are respectively input into the image encoder of the CLIP model to obtain the first image representation, the second image representation and the third image representation corresponding to each other; the CLIP model is trained by a sample set including a sample wafer image and its corresponding sample defect label; fusing the first image representation, the second image representation, and the third image representation to obtain a fused image representation; Use the text encoder of the CLIP model to encode multiple preset defect labels respectively to obtain multiple label text representations; Comparing the fused image representation with each label text representation for similarity, and determining the defect category of the target wafer according to the similarity comparison result; The fused image representation is input into a pre-trained LSTM model to obtain a first defect description of the target wafer.

2. The method according to claim 1, characterized in that The first image representation, the second image representation and the third image representation are fused to obtain a fused image representation, including: The first image representation, the second image representation and the third image representation are spliced ​​and input into a multi-layer perceptron to obtain a fused image representation.

3. The method according to claim 1, characterized in that The first image representation, the second image representation and the third image representation are fused to obtain a fused image representation, including: A pooling operation is performed on the first image representation, the second image representation, and the third image representation to obtain a fused image representation.

4. The method according to claim 1, characterized in that: The text encoder of the CLIP model is used to encode multiple preset defect labels respectively to obtain multiple label text representations, including: Fill the label text of each defect label into the preset prompt word template respectively to obtain the defect label prompt word corresponding to each defect label; Input each defect label prompt word into the text encoder of the CLIP model respectively to obtain the label text representation corresponding to each defect label.

5. The method according to claim 1, characterized in that The fused image representation and the label text representation have the same dimension; the fused image representation and each label text representation are respectively compared for similarity, and the defect category of the target wafer is determined according to the similarity comparison result, including: The cosine similarities between the fused image representation and each label text representation are calculated respectively, and the defect label corresponding to the label text representation with the largest cosine similarity value is determined as the defect category of the target wafer.

6. The method according to claim 1, characterized in that The method further comprises: The first defect description is input into a large language model, and the large language model is enabled to adjust the text content of the first defect description to obtain a second defect description.

7. The method according to claim 1, characterized in that The CLIP model and LSTM model are trained through the following steps: Acquire a sample set, including a plurality of sample wafer images containing defects and their corresponding sample defect labels, and sample defect descriptions for the sample wafer images; the sample wafer images include a sample defect view, a sample magnified view, and a sample top view of the sample wafer; Inputting the sample defect view, the sample magnified view and the sample top view into the image encoder of the CLIP model respectively, to obtain the fourth image representation, the fifth image representation and the sixth image representation respectively; fusing the fourth image representation, the fifth image representation and the sixth image representation to obtain a sample fused image representation; Determining a first training loss of a CLIP model according to the sample fusion image representation, the sample defect label, and the label text representation of each defect label; Determining a second training loss of the LSTM model according to the sample fusion image representation and the sample defect description; According to the total training loss determined by the first training loss and the second training loss, parameter values ​​of the CLIP model and the LSTM model are adjusted.

8. A device for generating defect categories and defect descriptions based on a wafer image, characterized in that: include: An acquisition unit configured to acquire a defect view, a magnified view, and a top view of a target wafer; An image encoding unit is configured to input the defect view, the magnified view and the top view into an image encoder of a CLIP model respectively to obtain a first image representation, a second image representation and a third image representation corresponding to each other; the CLIP model is fine-tuned by a sample set including a sample wafer image and its corresponding sample defect label; a feature fusion unit configured to fuse the first image representation, the second image representation and the third image representation to obtain a fused image representation; A label encoding unit is configured to respectively encode a plurality of preset defect labels using a text encoder of the CLIP model to obtain a plurality of label text representations; A defect category generating unit is configured to compare the fused image representation with each label text representation respectively for similarity, and determine the defect category of the target wafer according to the similarity comparison result; The defect description generating unit is configured to input the fused image representation into a pre-trained LSTM model to obtain a first defect description of the target wafer.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

10. A computing device comprising a memory and a processor, wherein: The memory stores executable code, and is characterized in that when the processor executes the executable code, the method according to any one of claims 1 to 7 is implemented.

Images