Wafer defect classification method and computer equipment

Abstract

The invention provides a wafer defect classification method and computer equipment. The wafer defect classification method comprises the following steps: firstly, inputting a wafer defect image into a classification model to obtain a plurality of wafer defect types in which the wafer defect image may fall; inputting the wafer defect image into a multi-modal large model to obtain a description text generated by the multi-modal large model for the wafer defect image; and finally, inputting a plurality of wafer defect types and description texts output by the classification model into a multi-modal large model to obtain a final wafer defect classification result. And by introducing a multi-modal large model, a classification result has interpretability. The generation results of the multi-modal large model and the classification model are combined for classification, and the capabilities of the two models are integrated, so that a more accurate wafer defect classification result is obtained.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a wafer defect classification method and computer equipment. Background Art

[0002] Wafers are the basic raw materials for manufacturing semiconductor products, and the quality of wafers will affect the performance and reliability of semiconductor products. During the wafer manufacturing process, various defects often appear on the wafer surface due to various factors, such as material defects, process parameter fluctuations, equipment failures, etc. These defects will affect the yield rate of the wafer and the quality of the semiconductor materials manufactured based on the wafer.

[0003] Different types of wafer defects are often caused by different reasons. Classifying wafer defects helps to quickly discover problems in the wafer defect manufacturing process, so that measures can be taken as soon as possible to reduce wafer defects and ensure the quality of finished products.

[0004] In recent years, with the development of computer vision (CV) technology, wafer defect classification methods based on machine learning have been increasingly used. However, the existing CV models have problems with poor interpretability and low accuracy when performing classification. Summary of the invention

[0005] In order to overcome the problems of poor interpretability and low accuracy of wafer classification results in the related art, this specification provides a wafer defect classification method and computer equipment.

[0006] According to a first aspect of an embodiment of this specification, a wafer defect classification method is provided, comprising: Inputting the wafer defect image into a classification model to obtain a plurality of wafer defect types that the wafer defect image may fall into; Inputting the wafer defect image into a multimodal large model to obtain a description text for the wafer defect image; The description text and a plurality of possible wafer defect types are input into a multimodal large model to obtain a wafer defect classification result corresponding to the wafer defect image; the classification result at least includes the type of wafer defects existing in the wafer defect image.

[0007] According to a second aspect of an embodiment of this specification, there is provided a wafer defect classification device, comprising: A classification module, used for inputting a wafer defect image into a classification model to obtain a plurality of wafer defect types that the wafer defect image may fall into; A text generation module, used for inputting the wafer defect image into the multimodal large model to obtain a description text for the wafer defect image; A classification result acquisition module is used to input the description text and multiple wafer defect types that may fall into the multimodal large model to obtain a wafer defect classification result corresponding to the wafer defect image; the classification result at least includes the type of wafer defects existing in the wafer defect image.

[0008] According to a third aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the wafer defect classification method as described in the first aspect of the embodiments of this specification is implemented.

[0009] According to a fourth aspect of the embodiments of this specification, a computer device is provided, the computer device comprising: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the wafer defect classification method as described in the first aspect of the embodiment of this specification.

[0010] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions, which, when executed, implement the wafer defect classification method as described in the first aspect of the embodiments of this specification.

[0011] This specification provides a wafer defect classification method, first inputting a wafer defect image into a classification model to obtain multiple wafer defect types that the wafer defect image may fall into; then inputting the wafer defect image into a multimodal large model to obtain a description text generated by the multimodal large model for the wafer defect image; finally, inputting multiple wafer defect types and description texts output by the classification model into the multimodal large model to obtain the final wafer defect classification result. By introducing a multimodal large model, the classification result is interpretable. By combining the generated results of the multimodal large model and the classification model for classification, the capabilities of the two models are combined to obtain a more accurate wafer defect classification result.

[0012] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0014] Figure 1 The flowchart of a wafer defect classification method is shown in this specification according to an exemplary embodiment.

[0015] Figure 2 This is a wafer defect image shown in this specification.

[0016] Figure 3 This is a flow chart of a wafer defect classification method shown in this specification according to a specific embodiment.

[0017] Figure 4 It is a block diagram of a wafer defect classification device according to an exemplary embodiment of the present specification.

[0018] Figure 5 It is a hardware structure diagram of a computer device shown in this specification according to an exemplary embodiment. DETAILED DESCRIPTION

[0019] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0020] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0021] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0022] For wafer defect images, CV models can generally only output the type of wafer defects in the wafer defect images, and lack the ability to understand and describe the semantics of wafer defects. This makes the classification results of CV models often lack of explanation and difficult to provide detailed defect tracing information. Moreover, the classification accuracy and robustness of CV models when dealing with complex defects still need to be improved.

[0023] Based on this, this specification provides a wafer defect classification method, which first inputs a wafer defect image into a classification model to obtain multiple wafer defect types that the wafer defect image may fall into; then inputs the wafer defect image into a multimodal large model to obtain a description text generated by the multimodal large model for the wafer defect image; finally, inputs the multiple wafer defect types and description texts output by the classification model into the multimodal large model to obtain the final wafer defect classification result.

[0024] By introducing a multimodal large model, the classification results are made interpretable. By combining the generation results of the multimodal large model and the classification model for classification, the capabilities of the two models are combined to obtain more accurate wafer defect classification results.

[0025] Next, the embodiments of this specification are described in detail.

[0026] like Figure 1 As shown, Figure 1 This is a flowchart of a multi-modal large model fine-tuning method according to an exemplary embodiment of this specification. The method includes: Step 101: input a wafer defect image into a classification model to obtain a plurality of wafer defect types that the wafer defect image may fall into.

[0027] Specifically, in the process of classifying wafer defects, the wafer defect images can first be classified through a classification model based on machine learning.

[0028] The method of the present specification can classify multiple wafer defect images, and can be carried out through multiple cycles, with each cycle processing one wafer defect image.

[0029] The wafer defect image may be a wafer image obtained by a scanning electron microscope (SEM). The scanning electron microscope can take pictures and scan the wafer and identify the defects therein. However, it can only identify the existence of defects and the location of the defects, but cannot specifically determine the type of defects.

[0030] After obtaining a defective wafer image through SEM, the entire wafer image can be directly used as a wafer defect image. In order to facilitate subsequent processing, an image of a preset size at the defective location can also be captured as a wafer defect image. Of course, the wafer defect image can also be obtained by taking pictures with other equipment that can identify wafer defects, and this manual does not limit this.

[0031] The classification model can be a pre-trained neural network model for classifying wafer defects included in a wafer defect image, which may also be referred to as a CV model in this article. As an optional example, the classification model may include two components, an encoding module and a classification module. The encoding module can perform feature extraction on the wafer defect image, that is, encode the wafer defect image to obtain a feature vector of the wafer defect image. The classification module can process the wafer defect image and determine the type of wafer defects included in the wafer defect image, so as to better trace the cause and adjust the problems existing in the wafer manufacturing process.

[0032] Through the classification model, the wafer defect type can be obtained. Different wafer defect types often correspond to different defect causes.

[0033] Among them, the wafer defect types output by the classification model may include multiple types. Since the classification model outputs not the final classification result, the classification model may have the problem of inaccurate classification results. By outputting multiple wafer defect types, the multimodal large model described later can perform more accurate classification.

[0034] Among them, multiple wafer defect types can be the most likely N classifications of the wafer defect image output by the classification model, where N is a preset value. In addition, in order to facilitate the judgment of the multimodal large model later, the classification model can optionally also output the ranking of each wafer defect type among multiple wafer defect types, or the possible probability of each wafer defect type, etc., so that the multimodal large model can perform more accurate classification.

[0035] In an optional implementation, in addition to obtaining the wafer defect type, step 101 can also obtain a feature vector of the wafer defect image, that is, the output of the encoding module mentioned above. The feature vector can be used to reduce the amount of calculation in the following text, as described in detail below.

[0036] In other words, step 101 may include: inputting the wafer defect image into a classification model, acquiring a wafer defect type corresponding to the wafer defect image, and a feature vector of the wafer defect image.

[0037] Step 103: input the wafer defect image into a multimodal large model to obtain a description text for the wafer defect image.

[0038] Specifically, the multimodal large model can generate description text based on the wafer defect image, so the wafer defect image can be input into the multimodal large model to obtain the output description text.

[0039] The generated description text is used to describe the specific situation of the wafer defect in the wafer defect image. Specifically, the description text may include information such as the attributes of the wafer defect in the wafer defect image, which is used to describe the specific representation of the wafer defect in the image. For example, the position of the wafer defect in the wafer defect image, the number of wafer defects in the wafer defect image, etc.

[0040] For ease of understanding, a specific example will be used to illustrate the description text. Figure 2 As shown, Figure 2 There are always pattern fail defects, and there are some locations in the image where long strip-shaped dents are not etched enough. Then the description text can explain the basic situation of the defects in the wafer defect image, such as there are 3 wafer defects in the middle upper and lower right corners of the image, and the specific defects are long strip-shaped dents that are not etched enough.

[0041] The multimodal large model may be a pre-trained multimodal large model commonly used in various fields. In another optional implementation, the multimodal large model is obtained by fine-tuning the pre-trained multimodal large model. Specifically, the fine-tuning is performed using a first training data set, wherein the first training data set includes a plurality of training data, each of which uses an image as input data and a description text corresponding to the image as a label.

[0042] The description text in the first training data set may be artificially generated description text for wafer defect images.

[0043] When fine-tuning the multimodal large model, prompt words are also input into the multimodal large model. The prompt words can guide the multimodal large model to learn how to generate descriptive text based on wafer defect images, thereby achieving fine-tuning of the multimodal large model.

[0044] By fine-tuning the multimodal large model, the multimodal large model can learn the relationship between the wafer defect image and its corresponding first description text according to the guidance of the prompt word, and then learn how to generate description text based on the wafer defect image. This enables the multimodal large model to have the ability in the wafer field and is easy to use.

[0045] In an optional embodiment, in addition to inputting the wafer defect image into the multimodal large model, a first prompt word can also be input into the multimodal large model. The first prompt word is used to enable the multimodal large model to generate a description text for the wafer defect image. Through the first prompt word, the multimodal large model can be guided to output the required content.

[0046] In other words, step 203 specifically includes: inputting the wafer defect image and the first prompt word into the multimodal large model to obtain a description text; the first prompt word is used to enable the multimodal large model to generate a description text for the wafer defect image.

[0047] When processing different wafer defect images, the same first prompt word may be used. In an optional implementation, the first prompt word may also include a format of a description text, so that the multimodal large model generates the required description text.

[0048] In addition, in an optional embodiment, the data input into the multimodal large model for characterizing the wafer defect image may be specifically the feature vector of the wafer defect image obtained in step 101. That is, what is input into the multimodal large model here may not be the wafer defect image, but the feature vector of the wafer defect image. The feature vector may be generated by a classification model.

[0049] This can save the amount of computation when processing the multimodal large model and improve efficiency. If the image is directly input into the multimodal large model, the multimodal large model also needs to encode the image to obtain the feature vector. However, the feature vector of the image can be obtained through the classification model, and the feature vector encoded by the classification model can be input into the multimodal large model, which can reduce the amount of computation of the multimodal large model and speed up the training process.

[0050] In other words, the output of the classification model also includes the feature vector of the target wafer defect image. Step 103 specifically includes: inputting the feature vector of the wafer defect image into the multimodal large model to obtain a description text for the wafer defect image.

[0051] Step 105 , input the description text and a plurality of possible wafer defect types into a multimodal large model to obtain a wafer defect classification result corresponding to the wafer defect image.

[0052] The classification result at least includes the type of wafer defects present in the wafer defect image.

[0053] Specifically, the description text generated by the multimodal large model and the multiple wafer defect types output by the classification model can be input into the multimodal large model, and the multimodal large model then processes them to determine the accurate wafer defect type, that is, the wafer defect classification result.

[0054] Among them, the wafer defect type output by the classification model may include multiple wafer defect types. In this case, the multimodal large model can combine the description text to select a more accurate wafer defect type from multiple wafer defect types, thereby achieving a more accurate classification purpose. In a possible implementation, the number of wafer defect types output by the classification model can also be one. In this case, the multimodal large model can determine whether the result output by the classification model is accurate based on the learned knowledge, and determine the accurate classification result if it is inaccurate.

[0055] This allows the wafer defect classification process to combine the capabilities of the multimodal large model and the classification model, thereby obtaining more accurate classification results.

[0056] During the execution of step 105, a second prompt word may also be input into the multimodal large model, where the second prompt word is used to enable the multimodal large model to classify the wafer defects in the wafer defect image in combination with the description text and a plurality of possible wafer defect types.

[0057] In an optional embodiment, the second prompt word may include content that guides the multimodal large model to output the classification result corresponding to the wafer defect image, and may also include content that guides the multimodal large model to learn to generate an accurate final classification result through the description text and the classification result output by the classification model. The second prompt words corresponding to different wafer defect images may be the same.

[0058] In other words, step 105 includes: inputting the description text, multiple possible wafer defect types and the second prompt word into the multimodal large model to obtain a classification result corresponding to the wafer defect image.

[0059] This can guide the large multimodal model to combine the output results of the two models for classification and obtain more accurate classification results.

[0060] In addition, after step 105, a knowledge base may be introduced, in which the causes corresponding to various wafer defects are stored. After executing the above steps, the classification results generated by the multimodal large model may be input into the multimodal large model, and the multimodal large model result knowledge base generates the causes of the above wafer defects.

[0061] In an optional implementation, the classification result may also include a description text for the wafer defect image, which not only allows the user to more intuitively identify the existing wafer defects, but also allows better matching with the knowledge base by inputting the description text into the multimodal large model during the tracing process.

[0062] In other words, the classification result may also be input into a multimodal large model to obtain the cause of the wafer defects in the wafer defect image generated by the multimodal large model based on the knowledge base.

[0063] In the above process, a third prompt word may also be input into the multimodal large model, and the third prompt word is used to enable the multimodal large model to learn how to generate the cause corresponding to the wafer defect image according to the knowledge base. In this way, automatic cause tracing is achieved.

[0064] The specific cause generation method can be completed through retrieval-augmented generation (RAG). Specifically, the above-mentioned cause generation process specifically includes the multimodal large model retrieving the description text corresponding to the classification result in the knowledge base according to the classification result; the multimodal large model generates the wafer defect cause in the wafer defect image according to the wafer defect cause corresponding to the retrieved description text.

[0065] Specifically, the knowledge base may include: description text of wafer defects (which may include information such as defect category, quantity and distribution), and analysis conclusions corresponding to the wafer defects, that is, what is the most likely cause of the current defect.

[0066] Optionally, the knowledge base can be a knowledge base generated by inputting a large amount of expert experience text into a multimodal large model. The multimodal large model can interpret and learn the expert experience to generate a knowledge base, which is convenient for tracing.

[0067] In other words, the method for generating the knowledge base includes: obtaining a text set; the text set includes multiple texts, each of which includes a description of a wafer defect and a cause of the corresponding wafer defect; and inputting the text set into a multimodal macro model to obtain the knowledge base. The text set includes expert experience.

[0068] When generating a knowledge base, the multimodal large model can first perform entity recognition to identify which part of the text is the description text and which part is the cause. Then, a structured knowledge base can be generated based on the recognized content.

[0069] Expert experience is a valuable resource in the process of wafer defect detection and classification. However, traditional expert experience mainly relies on manual recording and transmission, which is inefficient and prone to errors. The present invention uses the large model to interpret and learn expert experience, and stores the expert experience in a special expert experience knowledge base. Combined with defect classification and description information, defect tracing reasoning is performed on defects encountered later, which improves the accuracy and reliability of defect tracing. The construction of the expert experience knowledge base not only improves the intelligence level of the system, but also enhances the robustness and scalability of the system.

[0070] In summary, the above method enhances the interpretability of the classification process and makes the classification results easier to understand by introducing a multimodal large model. Specifically, traditional wafer defect classification mainly relies on single image feature extraction and classification, and lacks the ability to understand and describe the defects semantically. By introducing a multimodal large model, combined with image features and text descriptions, a comprehensive understanding and accurate classification of wafer defects is achieved. The multimodal large model can learn the relationship between image features and text descriptions to generate more accurate and detailed defect descriptions. This multimodal learning method not only improves the accuracy of classification, but also enhances the interpretability of the system, making the classification results easier to understand and apply.

[0071] The accuracy of wafer defect classification is improved by combining a multimodal large model with a machine learning model. The CV classification model can provide high-dimensional feature information, and the multimodal large model can generate more accurate and detailed defect descriptions. The combination of the two not only improves the accuracy of classification, but also enhances the robustness of the system. This combination enables the system to classify and reason more accurately and reliably when dealing with complex defects. Moreover, compared with processing with only a multimodal large model or only a machine learning model, combining the results of the two models can make the splitting results more accurate.

[0072] Next, the wafer defect classification method provided in this specification will be described through a specific embodiment.

[0073] like Figure 3 As shown, Figure 3 The round box in the figure represents the machine learning model, and the square box represents the specific data. Figure 3 There are multiple circular frames of multimodal large models, but multiple circular frames can represent the same multimodal large model.

[0074] The specific process is as follows Figure 3 As shown, the wafer defect image is input into the classification model to obtain the feature vector and the classification TopN. The classification TopN represents the most likely N classifications of the wafer defect image output by the classification model.

[0075] The feature vector corresponding to the wafer defect image is input into the multimodal large model to obtain the description text. In this process, the first prompt word can also be input into the multimodal large model.

[0076] The description text and the classification TopN are input into the multimodal large model to obtain the classification result. In this process, the second prompt word can also be input into the multimodal large model.

[0077] In addition, expert experience can be input into the multimodal large model in advance to generate a knowledge base, which stores the correspondence between the description text of wafer defects and the causes of wafer defects.

[0078] Furthermore, the classification results and knowledge base output by the multimodal large model can be input into the multimodal large model to obtain the defect causes corresponding to the wafer defect images.

[0079] Corresponding to the embodiments of the aforementioned method, this specification also provides embodiments of an apparatus and a computer device to which it is applied.

[0080] like Figure 4 As shown, Figure 4 1 is a block diagram of a wafer defect classification device according to an exemplary embodiment of the present specification, the device comprising: A classification module 410 is used to input the wafer defect image into a classification model to obtain a plurality of wafer defect types that the wafer defect image may fall into; A text generation module 420 is used to input the wafer defect image into the multimodal large model to obtain a description text for the wafer defect image; The classification result acquisition module 430 is used to input the description text and multiple wafer defect types that may fall into the multimodal large model to obtain the wafer defect classification result corresponding to the wafer defect image; the classification result at least includes the type of wafer defects existing in the wafer defect image.

[0081] In an optional implementation, the classification module 410 is specifically used to input the wafer defect image into the classification model to obtain the wafer defect type corresponding to the wafer defect image and the feature vector of the wafer defect image. The text generation module 420 is specifically used to input the feature vector of the wafer defect image into the multimodal large model to obtain a description text for the wafer defect image.

[0082] In an optional embodiment, the multimodal large model is associated with a knowledge base, the knowledge base includes a plurality of description texts of wafer defects and the causes of wafer defects corresponding to each description text; the classification result also includes a description text for the wafer defect image. The device also includes a tracing module 440 (not shown in the figure), which is used to input the classification result into the multimodal large model to obtain the causes of wafer defects in the wafer defect image generated by the multimodal large model based on the knowledge base.

[0083] In an optional embodiment, the device also includes a knowledge base generation module 400 (not shown in the figure), which is used to obtain a text set; the text set includes multiple texts, each text includes a description of wafer defects and the cause of the corresponding wafer defects; the text set is input into the multimodal large model to obtain the knowledge base.

[0084] In an optional embodiment, the tracing module 440 is specifically used for the multimodal large model to retrieve a description text corresponding to the classification result in the knowledge base according to the classification result; the multimodal large model generates the cause of the wafer defect in the wafer defect image according to the cause of the wafer defect corresponding to the retrieved description text.

[0085] In an optional embodiment, the multimodal large model is obtained by fine-tuning a pre-trained multimodal large model; the fine-tuning is completed using a first training data set, and the first training data set includes multiple training data, each training data uses an image as input data and a description text corresponding to the image as a label.

[0086] In an optional embodiment, the text generation module 420 is specifically used to input the wafer defect image and the first prompt word into the multimodal large model to obtain a description text; the first prompt word is used to enable the multimodal large model to generate a description text for the wafer defect image.

[0087] In an optional embodiment, the classification result acquisition module 430 is specifically used to input the description text, multiple possible wafer defect types and a second prompt word into the multimodal large model to obtain a classification result corresponding to the wafer defect image; the second prompt word is used to enable the multimodal large model to combine the description text and multiple possible wafer defect types to classify the wafer defects in the wafer defect image.

[0088] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0089] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0090] like Figure 5 As shown, Figure 5A hardware structure diagram of a computer device is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0091] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0092] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0093] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0094] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0095] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0096] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0097] The embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the wafer defect classification method as described in the first aspect of the embodiments of this specification is implemented.

[0098] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0099] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0100] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0101] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0102] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A wafer defect classification method, characterized in that: include: Inputting the wafer defect image into a classification model to obtain a plurality of wafer defect types that the wafer defect image may fall into; Inputting the wafer defect image into a multimodal large model to obtain a description text for the wafer defect image; The description text and a plurality of possible wafer defect types are input into a multimodal large model to obtain a wafer defect classification result corresponding to the wafer defect image; the classification result at least includes the type of wafer defects existing in the wafer defect image.

2. The method according to claim 1, characterized in that The step of inputting the wafer defect image into a classification model to obtain a plurality of wafer defect types that the wafer defect image may fall into includes: Inputting the wafer defect image into the classification model to obtain the wafer defect type corresponding to the wafer defect image and the feature vector of the wafer defect image; The step of inputting the wafer defect image into a multimodal large model to obtain a description text for the wafer defect image includes: The feature vector of the wafer defect image is input into a multimodal large model to obtain a description text for the wafer defect image.

3. The method according to claim 1, characterized in that The multimodal large model is associated with a knowledge base, wherein the knowledge base includes description texts of multiple wafer defects and causes of the wafer defects corresponding to each description text; the classification result also includes description text for the wafer defect image; The method further comprises: The classification result is input into the multimodal large model to obtain the cause of the wafer defect in the wafer defect image generated by the multimodal large model based on the knowledge base.

4. The method according to claim 3, characterized in that The method for generating the knowledge base includes: Acquire a text set; the text set includes a plurality of texts, each of which includes a description of a wafer defect and a cause of the corresponding wafer defect; The text set is input into a multimodal large model for entity recognition and structural processing to obtain the knowledge base.

5. The method according to claim 3, characterized in that: The step of inputting the classification result into the multimodal large model to obtain the cause of the wafer defect in the wafer defect image generated by the multimodal large model based on the knowledge base includes: The multimodal large model retrieves the description text corresponding to the classification result in the knowledge base according to the classification result; The multimodal large model generates the causes of wafer defects in the wafer defect image according to the causes of wafer defects corresponding to the retrieved description texts.

6. The method according to claim 1, characterized in that The multimodal large model is obtained by fine-tuning a pre-trained multimodal large model; the fine-tuning is completed using a first training data set, the first training data set includes multiple training data, each training data uses an image as input data and a description text corresponding to the image as a label.

7. The method according to claim 1, characterized in that The step of inputting the wafer defect image into a multimodal large model to obtain a description text for the wafer defect image includes: The wafer defect image and the first prompt word are input into the multimodal large model to obtain a description text; the first prompt word is used to enable the multimodal large model to generate a description text for the wafer defect image.

8. The method according to claim 1, characterized in that The step of inputting the description text and a plurality of possible wafer defect types into a multimodal large model to obtain a wafer defect classification result corresponding to the wafer defect image includes: The description text, multiple possible wafer defect types and a second prompt word are input into the multimodal large model to obtain a classification result corresponding to the wafer defect image; the second prompt word is used to enable the multimodal large model to combine the description text and multiple possible wafer defect types to classify the wafer defects in the wafer defect image.

9. A wafer defect classification device, characterized in that: include: A classification module, used for inputting a wafer defect image into a classification model to obtain a plurality of wafer defect types that the wafer defect image may fall into; A text generation module, used for inputting the wafer defect image into the multimodal large model to obtain a description text for the wafer defect image; A classification result acquisition module is used to input the description text and multiple wafer defect types that may fall into the multimodal large model to obtain a wafer defect classification result corresponding to the wafer defect image; the classification result at least includes the type of wafer defects existing in the wafer defect image.

10. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

Images