Wafer defect classification method and computer device

Abstract

The specification provides a wafer defect classification method and a computer device. The wafer defect classification method comprises: first inputting a wafer defect image into a classification model to obtain a plurality of wafer defect types to which the wafer defect image may fall; then inputting the wafer defect image into a multi-modal large model to obtain a description text generated by the multi-modal large model for the wafer defect image; finally inputting the plurality of wafer defect types output by the classification model and the description text into the multi-modal large model to obtain a final wafer defect classification result. By introducing the multi-modal large model, the classification result has interpretability. By combining the generation results of the multi-modal large model and the classification model for classification, the capabilities of the two models are integrated, so that a more accurate wafer defect classification result is obtained.

Description

Technical Field

[0001] This specification relates to the field of computer vision technology, and in particular to a method for classifying wafer defects and a computer device. Background Technology

[0002] Wafers are the fundamental raw material for manufacturing semiconductor products, and their quality directly impacts the performance and reliability of these products. During wafer manufacturing, various factors, such as material defects, process parameter fluctuations, and equipment malfunctions, often lead to defects on the wafer surface. These defects affect the wafer yield and the quality of the semiconductor materials manufactured from the wafer.

[0003] Different types of wafer defects are often caused by different reasons. Classifying wafer defects helps to quickly identify problems in the wafer manufacturing process, so that measures can be taken as soon as possible to reduce wafer defects and ensure the quality of finished products.

[0004] In recent years, with the development of computer vision (CV) technology, machine learning-based wafer defect classification methods have been increasingly used. However, existing CV models suffer from poor interpretability and low accuracy when performing classification. Summary of the Invention

[0005] To overcome the problems of poor interpretability and low accuracy of wafer classification results in related technologies, this specification provides a wafer defect classification method and computer equipment.

[0006] According to a first aspect of the embodiments of this specification, a wafer defect classification method is provided, comprising:

[0007] The wafer defect image is input into the classification model to obtain multiple wafer defect types that the wafer defect image may fall into;

[0008] The wafer defect image is input into a multimodal large model to obtain descriptive text for the wafer defect image;

[0009] The descriptive text and multiple possible wafer defect types are input into a multimodal large model to obtain the wafer defect classification result corresponding to the wafer defect image; the classification result includes at least the types of wafer defects present in the wafer defect image.

[0010] According to a second aspect of the embodiments of this specification, a wafer defect classification apparatus is provided, comprising:

[0011] The classification module is used to input wafer defect images into a classification model to obtain multiple wafer defect types that the wafer defect images may fall into;

[0012] The text generation module is used to input the wafer defect image into a multimodal large model to obtain descriptive text for the wafer defect image;

[0013] The classification result acquisition module is used to input the description text and multiple possible wafer defect types into a multimodal large model to obtain the wafer defect classification result corresponding to the wafer defect image; the classification result includes at least the types of wafer defects present in the wafer defect image.

[0014] According to a third aspect of the embodiments of this specification, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the wafer defect classification method as described in the first aspect of the embodiments of this specification.

[0015] According to a fourth aspect of the embodiments of this specification, a computer device is provided, the computer device comprising:

[0016] One or more processors;

[0017] Memory, used to store one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the wafer defect classification method as described in the first aspect of the embodiments of this specification.

[0019] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions that, when executed, implement the wafer defect classification method as described in the first aspect of the embodiments of this specification.

[0020] This specification provides a wafer defect classification method. First, a wafer defect image is input into a classification model to obtain multiple wafer defect types that the image may fall into. Then, the wafer defect image is input into a multimodal large-scale model to obtain descriptive text generated by the model for that image. Finally, the multiple wafer defect types and descriptive text output by the classification model are input into the multimodal large-scale model to obtain the final wafer defect classification result. By introducing a multimodal large-scale model, the classification result becomes interpretable. By combining the results generated by the multimodal large-scale model and the classification model, the capabilities of both models are integrated, resulting in a more accurate wafer defect classification result.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this specification and, together with the description, serve to explain the principles of this specification.

[0023] Figure 1 This is a flowchart illustrating a wafer defect classification method according to an exemplary embodiment of this specification.

[0024] Figure 2 This is an image of a wafer defect shown in this specification.

[0025] Figure 3 This is a flowchart illustrating a wafer defect classification method according to a specific embodiment of this specification.

[0026] Figure 4 This is a block diagram illustrating a wafer defect classification device according to an exemplary embodiment of this specification.

[0027] Figure 5 This is a hardware structure diagram of a computer device illustrated in this specification according to an exemplary embodiment. Detailed Implementation

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0029] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0030] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0031] For wafer defect images, CV (Computer Vision) models typically only output the type of wafer defect in the image, lacking semantic understanding and descriptive ability. This makes the classification results of CV models often lack interpretability and struggle to provide detailed defect attribution information. Furthermore, the classification accuracy and robustness of CV models still need improvement when dealing with complex defects.

[0032] Based on this, this specification provides a wafer defect classification method. First, the wafer defect image is input into a classification model to obtain multiple wafer defect types that the wafer defect image may fall into. Then, the wafer defect image is input into a multimodal large model to obtain descriptive text generated by the multimodal large model for the wafer defect image. Finally, the multiple wafer defect types and descriptive text output by the classification model are input into the multimodal large model to obtain the final wafer defect classification result.

[0033] By introducing a multimodal large model, the classification results become interpretable. Combining the results generated by the multimodal large model and the classification model for classification integrates the capabilities of both models, resulting in more accurate wafer defect classification results.

[0034] The embodiments described in this specification will now be described in detail.

[0035] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a multimodal large model fine-tuning method according to an exemplary embodiment of this specification. The method includes:

[0036] Step 101: Input the wafer defect image into the classification model to obtain multiple wafer defect types that the wafer defect image may fall into.

[0037] Specifically, in the process of classifying wafer defects, the wafer defect images can first be classified using a machine learning-based classification model.

[0038] The method described in this specification can classify multiple wafer defect images. It can be implemented through multiple iterations, with each iteration processing one wafer defect image.

[0039] Wafer defect images can be obtained using a scanning electron microscope (SEM). An SEM can photograph and scan a wafer to identify defects. However, it can only identify the presence and location of defects, not their specific types.

[0040] After obtaining a defective wafer image using SEM, the entire wafer image can be directly used as the wafer defect image. For easier subsequent processing, a pre-defined portion of the image at the defect location can also be used as the wafer defect image. Of course, wafer defect images can also be obtained using other devices capable of identifying wafer defects; this manual does not limit this approach.

[0041] The classification model can be a pre-trained neural network model used to classify wafer defects included in wafer defect images; in this paper, it can also be referred to as a CV model. As an optional example, the classification model can include two components: an encoding module and a classification module. The encoding module extracts features from the wafer defect image, that is, it encodes the wafer defect image to obtain a feature vector of the wafer defect image. The classification module processes the wafer defect image to determine the type of wafer defect included in the image, thereby enabling better root cause analysis and adjustment of problems in the wafer manufacturing process.

[0042] By using a classification model, wafer defect types can be identified. Different wafer defect types often correspond to different defect causes.

[0043] The classification model can output multiple wafer defect types. Since the output is not the final classification result, the model may produce inaccurate results. Outputting multiple wafer defect types allows for more accurate classification using the multimodal large-scale model discussed later.

[0044] The multiple wafer defect types can be output by the classification model, representing the N most likely classifications of the wafer defect image, where N is a preset value. Furthermore, to facilitate the multimodal large-scale model's judgment later, the classification model can optionally output the ranking of each wafer defect type among multiple wafer defect types, or the probability of each wafer defect type, etc., to enable more accurate classification by the multimodal large-scale model.

[0045] In an optional implementation, step 101 can obtain not only the wafer defect type but also the feature vector of the wafer defect image, which is the output of the encoding module mentioned above. This feature vector can be used to reduce computational load later, as detailed in the following description.

[0046] In other words, step 101 may include: inputting the wafer defect image into a classification model to obtain the wafer defect type corresponding to the wafer defect image and the feature vector of the wafer defect image.

[0047] Step 103: Input the wafer defect image into the multimodal large model to obtain a descriptive text for the wafer defect image.

[0048] Specifically, the multimodal large model can generate descriptive text based on wafer defect images. Therefore, wafer defect images can be input into the multimodal large model to obtain the output descriptive text.

[0049] The generated descriptive text is used to describe the specific details of the wafer defects in the wafer defect image. Specifically, the descriptive text may include information such as the attributes of the wafer defects in the image, describing their specific representation within the image. For example, it may include the location of the wafer defects in the image, the number of wafer defects, and so on.

[0050] To facilitate understanding, a specific example will be used to illustrate the descriptive text. For example... Figure 2 As shown, Figure 2 Pattern failures always exist, and in the image, there are some locations where elongated indentations were not etched. The descriptive text can describe the basic situation of the defects present in the wafer defect image, such as three wafer defects located in the upper center and lower right corner of the image, specifically elongated indentations that were not etched.

[0051] The multimodal large model can be a pre-trained, domain-specific multimodal large model. In another optional implementation, the multimodal large model is obtained by fine-tuning a pre-trained multimodal large model. Specifically, the fine-tuning is performed using a first training dataset, which includes multiple training data sets, each with an image as input and the corresponding descriptive text as a label.

[0052] The descriptive text in the first training dataset can be artificially generated descriptive text for wafer defect images.

[0053] During the fine-tuning of the multimodal large model, cue words were also input into the multimodal large model. These cue words can guide the multimodal large model to learn how to generate descriptive text based on wafer defect images, thereby enabling the fine-tuning of the multimodal large model.

[0054] By fine-tuning the multimodal large model, it can learn the relationship between wafer defect images and their corresponding initial descriptive text based on prompts, and then learn how to generate descriptive text from wafer defect images. This gives the multimodal large model wafer-level capabilities, making it easy to use.

[0055] In one alternative implementation, in addition to inputting a wafer defect image into the multimodal model, a first prompt word can also be input into the multimodal model. The first prompt word is used to instruct the multimodal model to generate descriptive text for the wafer defect image. The first prompt word guides the multimodal model to output the required content.

[0056] In other words, step 203 specifically includes: inputting the wafer defect image and the first prompt word into the multimodal large model to obtain descriptive text; the first prompt word is used to enable the multimodal large model to generate descriptive text for the wafer defect image.

[0057] When processing images of different wafer defects, the same first prompt word can be used. In an alternative embodiment, the first prompt word may also include a descriptive text format to enable the multimodal large model to generate the required descriptive text.

[0058] Furthermore, in an optional embodiment, the data used to characterize the wafer defect image input to the multimodal large model can specifically be the feature vector of the wafer defect image obtained in step 101. That is, the input to the multimodal large model here may not be the wafer defect image itself, but rather the feature vector of the wafer defect image. This feature vector can be generated by a classification model.

[0059] This saves computational resources and improves efficiency when processing large multimodal models. If images are directly input into a large multimodal model, the model also needs to encode the images to obtain feature vectors. However, by using a classification model to obtain the image's feature vectors, and then inputting the encoded feature vectors from the classification model into the large multimodal model, the computational load of the model can be reduced, accelerating the training process.

[0060] In other words, the output of the classification model also includes the feature vector of the target wafer defect image. Step 103 specifically includes: inputting the feature vector of the wafer defect image into a multimodal large model to obtain descriptive text for the wafer defect image.

[0061] Step 105: Input the description text and the multiple possible wafer defect types into the multimodal large model to obtain the wafer defect classification result corresponding to the wafer defect image.

[0062] The classification results include at least the types of wafer defects present in the wafer defect images.

[0063] Specifically, the descriptive text generated by the multimodal large model and the multiple wafer defect types output by the classification model can be input into the multimodal large model. The multimodal large model then processes the data to determine the accurate wafer defect type, i.e., the wafer defect classification result.

[0064] In this scenario, the classification model may output multiple wafer defect types. In such cases, the multimodal large model can combine descriptive text to select the more accurate wafer defect type from the multiple types, thereby achieving a more accurate classification. Alternatively, the classification model may output only one number of wafer defect types. In this case, the multimodal large model can determine the accuracy of the classification model's output based on learned knowledge, and if it is inaccurate, determine the accurate classification result.

[0065] This allows the wafer defect classification process to combine the capabilities of a multimodal large model and a classification model, resulting in more accurate classification results.

[0066] During step 105, a second prompt word can also be input into the multimodal large model. The second prompt word is used to enable the multimodal large model to classify the wafer defects in the wafer defect image by combining the descriptive text and multiple possible wafer defect types.

[0067] In one optional implementation, the second prompt word may include content guiding the multimodal large model to output the classification result corresponding to the wafer defect image, and may also include content guiding the multimodal large model to learn from the descriptive text and the classification result output by the classification model to generate an accurate final classification result. The second prompt word may be the same for different wafer defect images.

[0068] In other words, step 105 includes: inputting the descriptive text, the multiple wafer defect types that may fall into the category, and the second prompt word into a multimodal large model to obtain the classification result corresponding to the wafer defect image.

[0069] This allows the multimodal large model to combine the outputs of the two models for classification, resulting in a more accurate classification result.

[0070] Furthermore, after step 105, a knowledge base can be introduced, which stores the causes of various wafer defects. After completing the above steps, the classification results generated by the multimodal large model can also be input into the multimodal large model, and the multimodal large model result knowledge base generates the causes of the aforementioned wafer defects.

[0071] In one alternative implementation, the classification results may further include descriptive text for the wafer defect image. This not only allows users to more intuitively identify existing wafer defects, but also enables better matching with the knowledge base when inputting descriptive text into the multimodal large model during the cause-finding process.

[0072] In other words, the classification results can also be input into the multimodal large model to obtain the causes of wafer defects in the wafer defect image generated by the multimodal large model based on the knowledge base.

[0073] In the above process, a third cue word can also be input into the multimodal large model. This third cue word enables the multimodal large model to learn how to generate the cause corresponding to the wafer defect image based on the knowledge base. In this way, automatic cause tracing is achieved.

[0074] For specific cause generation methods, retrieval-augmented generation (RAG) can be used. Specifically, the above-mentioned cause generation process includes a multimodal large model retrieving descriptive text corresponding to the classification result from the knowledge base; and the multimodal large model generating the wafer defect cause in the wafer defect image based on the cause of the wafer defect corresponding to the retrieved descriptive text.

[0075] Specifically, the knowledge base may include: a descriptive text of the wafer defect (which may include information such as the type, number, and distribution of the defect), and the corresponding analytical conclusions for the wafer defect, that is, what is the most likely cause of the current defect.

[0076] Optionally, the knowledge base can be generated by inputting a large amount of expert experience text into a multimodal large model. The multimodal large model can interpret and learn from expert experience to generate the knowledge base, facilitating causal attribution.

[0077] In other words, the method for generating the knowledge base includes: acquiring a text set; the text set includes multiple texts, each text containing a description of a wafer defect and the corresponding cause of the wafer defect; inputting the text set into a multimodal large model to obtain the knowledge base. The text set includes expert experience.

[0078] When generating a knowledge base, a multimodal large model can first perform entity recognition to identify which parts of the text are descriptive and which are causal. Then, a structured knowledge base can be generated based on the recognized content.

[0079] In wafer defect detection and classification, expert experience is a valuable resource. However, traditional expert experience relies heavily on manual recording and transmission, which is inefficient and prone to errors. This invention utilizes a large-scale model to interpret and learn from expert experience, storing it in a dedicated expert experience knowledge base. Combined with defect classification and description information, it performs defect cause reasoning for subsequently encountered defects, improving the accuracy and reliability of defect cause analysis. The construction of the expert experience knowledge base not only enhances the system's intelligence level but also strengthens its robustness and scalability.

[0080] In summary, the above methods enhance the interpretability of the classification process and make the classification results easier to understand by introducing a multimodal large model. Specifically, traditional wafer defect classification mainly relies on single image feature extraction and classification, lacking the ability to semantically understand and describe defects. By introducing a multimodal large model, combining image features and textual descriptions, a comprehensive understanding and accurate classification of wafer defects is achieved. The multimodal large model can learn the relationship between image features and textual descriptions, generating more accurate and detailed defect descriptions. This multimodal learning approach not only improves the accuracy of classification but also enhances the interpretability of the system, making the classification results easier to understand and apply.

[0081] By combining a multimodal large model and a machine learning model, the accuracy of wafer defect classification has been improved. The CV classification model provides high-dimensional feature information, while the multimodal large model generates more accurate and detailed defect descriptions. The combination not only improves classification accuracy but also enhances the system's robustness. This approach enables the system to classify and reason about complex defects more accurately and reliably. Furthermore, compared to using only a multimodal large model or only a machine learning model, combining the results of both models leads to more accurate decomposition results.

[0082] The wafer defect classification method provided in this specification will now be illustrated through a specific embodiment.

[0083] like Figure 3 As shown, Figure 3 The circles in the diagram represent machine learning models, and the squares represent specific data. It should be noted that, although... Figure 3 There are multiple circles representing the same multimodal large model, but multiple circles can represent the same multimodal large model.

[0084] The specific process is as follows: Figure 3 As shown, the wafer defect image is input into the classification model to obtain feature vectors and Top N classifications. Top N classifications represent the N most likely classifications of the wafer defect image output by the classification model.

[0085] The feature vector corresponding to the wafer defect image is input into the multimodal large model to obtain the descriptive text. During this process, the first prompt word can also be input into the multimodal large model.

[0086] The descriptive text and the Top N categories are input into a multimodal large-scale model to obtain the classification results. A second cue word can also be input into the multimodal large-scale model during this process.

[0087] In addition, expert experience can be pre-input into the multimodal large model to generate a knowledge base, which stores the descriptive text of wafer defects and the corresponding relationship between the causes of wafer defects.

[0088] Furthermore, the classification results and knowledge base output by the multimodal large model can be input into the multimodal large model to obtain the defect causes corresponding to the wafer defect images.

[0089] Corresponding to the embodiments of the foregoing methods, this specification also provides embodiments of the apparatus and the computer equipment to which it is applied.

[0090] like Figure 4 As shown, Figure 4 This is a block diagram illustrating a wafer defect classification apparatus according to an exemplary embodiment of this specification, the apparatus comprising:

[0091] The classification module 410 is used to input the wafer defect image into the classification model to obtain multiple wafer defect types that the wafer defect image may fall into;

[0092] The text generation module 420 is used to input the wafer defect image into a multimodal large model to obtain descriptive text for the wafer defect image;

[0093] The classification result acquisition module 430 is used to input the description text and multiple possible wafer defect types into a multimodal large model to obtain the wafer defect classification result corresponding to the wafer defect image; the classification result includes at least the types of wafer defects present in the wafer defect image.

[0094] In one optional implementation, the classification module 410 is specifically used to input the wafer defect image into a classification model to obtain the wafer defect type corresponding to the wafer defect image and the feature vector of the wafer defect image. The text generation module 420 is specifically used to input the feature vector of the wafer defect image into a multimodal large model to obtain descriptive text for the wafer defect image.

[0095] In one optional embodiment, the multimodal large model is associated with a knowledge base, which includes descriptive text for multiple wafer defects and the causes of the wafer defects corresponding to each descriptive text; the classification result also includes descriptive text for the wafer defect image. The device further includes a cause-finding module 440 (not shown in the figure), used to input the classification result into the multimodal large model to obtain the causes of wafer defects in the wafer defect image generated by the multimodal large model based on the knowledge base.

[0096] In an optional embodiment, the device further includes a knowledge base generation module 400 (not shown in the figure) for acquiring a text set; the text set includes multiple texts, each text including a description of a wafer defect and the cause of the corresponding wafer defect; the text set is input into a multimodal large model to obtain the knowledge base.

[0097] In one optional implementation, the cause-finding module 440 is specifically used to retrieve descriptive text corresponding to the classification result in the knowledge base based on the classification result; and the multimodal large model generates the cause of wafer defects in the wafer defect image based on the cause of wafer defects corresponding to the retrieved descriptive text.

[0098] In one optional implementation, the multimodal large model is obtained by fine-tuning a pre-trained multimodal large model; the fine-tuning is accomplished using a first training dataset, which includes multiple training data, each training data taking an image as input data and the corresponding descriptive text of the image as a label.

[0099] In one optional embodiment, the text generation module 420 is specifically used to input the wafer defect image and the first prompt word into the multimodal large model to obtain descriptive text; the first prompt word is used to enable the multimodal large model to generate descriptive text for the wafer defect image.

[0100] In one optional embodiment, the classification result acquisition module 430 is specifically used to input the descriptive text, the multiple wafer defect types that may fall into the category, and the second prompt word into the multimodal large model to obtain the classification result corresponding to the wafer defect image; the second prompt word is used to enable the multimodal large model to combine the descriptive text and the multiple wafer defect types that may fall into the category to classify the wafer defects in the wafer defect image.

[0101] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0102] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0103] like Figure 5 As shown, Figure 5A hardware structure diagram of a computer device is shown. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are internally connected to each other via the bus 1050.

[0104] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0105] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0106] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0107] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0108] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0109] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0110] This specification also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the wafer defect classification method as described in the first aspect of this specification.

[0111] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0112] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0113] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.

[0114] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

[0115] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A wafer defect classification method, characterized by, The method comprises the following steps: inputting a wafer defect image into a classification model to obtain a plurality of wafer defect types to which the wafer defect image is likely to fall, and first information, wherein the first information comprises a ranking of each wafer defect type to which the wafer defect image is likely to fall in the plurality of wafer defect types, or a probability of each wafer defect type to which the wafer defect image is likely to fall; inputting the wafer defect image into a multi-modal large model to obtain a description text for the wafer defect image; inputting the description text and the plurality of wafer defect types to which the wafer defect image is likely to fall into the multi-modal large model, and obtaining a wafer defect classification result corresponding to the wafer defect image based on the description text, the plurality of wafer defect types to which the wafer defect image is likely to fall, and the first information, wherein the classification result at least comprises a type of wafer defect existing in the wafer defect image, the classification result comprises the type of wafer defect determined from the plurality of wafer defect types to which the wafer defect image is likely to fall, and the multi-modal large model is associated with a knowledge base, wherein the knowledge base comprises a plurality of description texts of wafer defects and causes of wafer defects corresponding to each description text; the multi-modal large model retrieves a description text corresponding to the classification result from the knowledge base according to the classification result, wherein the classification result further comprises a description text for the wafer defect image; the multi-modal large model generates a cause of wafer defect in the wafer defect image according to the cause of wafer defect corresponding to the retrieved description text.

2. The method of claim 1, wherein, The method comprises the following steps: inputting a wafer defect image into a classification model to obtain a plurality of wafer defect types to which the wafer defect image is likely to fall, and first information, wherein the first information comprises a ranking of each wafer defect type to which the wafer defect image is likely to fall in the plurality of wafer defect types, or a probability of each wafer defect type to which the wafer defect image is likely to fall; inputting the wafer defect image into a multi-modal large model to obtain a description text for the wafer defect image; inputting the description text and the plurality of wafer defect types to which the wafer defect image is likely to fall into the multi-modal large model, and obtaining a wafer defect classification result corresponding to the wafer defect image based on the description text, the plurality of wafer defect types to which the wafer defect image is likely to fall, and the first information, wherein the classification result at least comprises a type of wafer defect existing in the wafer defect image, the classification result comprises the type of wafer defect determined from the plurality of wafer defect types to which the wafer defect image is likely to fall, and the multi-modal large model is associated with a knowledge base, wherein the knowledge base comprises a plurality of description texts of wafer defects and causes of wafer defects corresponding to each description text; 3. The method of claim 1, wherein, the multi-modal large model retrieves a description text corresponding to the classification result from the knowledge base according to the classification result, wherein the classification result further comprises a description text for the wafer defect image; the multi-modal large model generates a cause of wafer defect in the wafer defect image according to the cause of wafer defect corresponding to the retrieved description text. The method comprises the following steps:

4. The method of claim 1, wherein, inputting a wafer defect image into a classification model to obtain a plurality of wafer defect types to which the wafer defect image is likely to fall, and first information, wherein the first information comprises a ranking of each wafer defect type to which the wafer defect image is likely to fall in the plurality of wafer defect types, or a probability of each wafer defect type to which the wafer defect image is likely to fall; 5. The method of claim 1, wherein, inputting the wafer defect image into a multi-modal large model to obtain a description text for the wafer defect image; inputting the description text and the plurality of wafer defect types to which the wafer defect image is likely to fall into the multi-modal large model, and obtaining a wafer defect classification result corresponding to the wafer defect image based on the description text, the plurality of wafer defect types to which the wafer defect image is likely to fall, and the first information, wherein the classification result at least comprises a type of wafer defect existing in the wafer defect image, the classification result comprises the type of wafer defect determined from the plurality of wafer defect types to which the wafer defect image is likely to fall, and the multi-modal large model is associated with a knowledge base, wherein the knowledge base comprises a plurality of description texts of wafer defects and causes of wafer defects corresponding to each description text; 6. The method of claim 1, wherein, the multi-modal large model retrieves a description text corresponding to the classification result from the knowledge base according to the classification result, wherein the classification result further comprises a description text for the wafer defect image; the multi-modal large model generates a cause of wafer defect in the wafer defect image according to the cause of wafer defect corresponding to the retrieved description text. The method comprises the following steps: inputting a wafer defect image into a classification model to obtain a plurality of wafer defect types to which the wafer defect image is likely to fall, and first information, wherein the first information comprises a ranking of each wafer defect type to which the wafer defect image is likely to fall in the plurality of wafer defect types, or a probability of each wafer defect type to which the wafer defect image is likely to fall; inputting the wafer defect image into a multi-modal large model to obtain a description text for the wafer defect image; inputting the description text and the plurality of wafer defect types to which the wafer defect image is likely to fall into the multi-modal large model, and obtaining a wafer defect classification result corresponding to the wafer defect image based on the description text, the plurality of wafer defect types to which the wafer defect image is likely to fall, and the first information, wherein the classification result at least comprises a type of wafer defect existing in the wafer defect image, the classification result comprises the type of wafer defect determined from the plurality of wafer defect types to which the wafer defect image is likely to fall, and the multi-modal large model is associated with a knowledge base, wherein the knowledge base comprises a plurality of description texts of wafer defects and causes of wafer defects corresponding to each description text; the multi-modal large model retrieves a description text corresponding to the classification result from the knowledge base according to the classification result, wherein the classification result further comprises a description text for the wafer defect image; the multi-modal large model generates a cause of wafer defect in the wafer defect image according to the cause of wafer defect corresponding to the retrieved description text. The description text, the wafer defect types possibly fallen into, and a second prompt word are input into a multi-modal large model to obtain a classification result corresponding to the wafer defect image; the second prompt word is used to make the multi-modal large model combine the description text and the wafer defect types possibly fallen into to classify the wafer defect in the wafer defect image.

7. A wafer defect classification apparatus characterized by comprising: Comprise: A classification module configured to input a wafer defect image into a classification model, obtain a plurality of wafer defect types to which the wafer defect image possibly falls, and first information, the first information comprising a ranking of each wafer defect type to which the wafer defect image possibly falls in the plurality of wafer defect types, or a probability of each wafer defect type to which the wafer defect image possibly falls; A text generation module configured to input the wafer defect image into a multi-modal large model to obtain a description text for the wafer defect image; A classification result acquisition module configured to input the description text and the wafer defect types possibly fallen into into a multi-modal large model, the multi-modal large model obtaining a wafer defect classification result corresponding to the wafer defect image based on the description text, the wafer defect types possibly fallen into, and the first information; the classification result at least comprises a type of wafer defect existing in the wafer defect image, the type of wafer defect being determined from the wafer defect types possibly fallen into; the multi-modal large model is associated with a knowledge base, the knowledge base comprising description texts of a plurality of wafer defects and causes of wafer defects corresponding to each description text; A cause tracing module configured to make the multi-modal large model retrieve a description text corresponding to the classification result in the knowledge base according to the classification result; the classification result further comprises a description text for the wafer defect image; the multi-modal large model generates a cause of a wafer defect in the wafer defect image according to a cause of a wafer defect corresponding to the retrieved description text.

8. A computer device, comprising: The computer device comprises: One or more processors; Memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 6.

Images