Wafer defect detection method, device, equipment, storage medium and program product
By introducing attention modules into the wafer defect detection model, the feature extraction capability is improved, the problem of low detection accuracy in the existing technology is solved, and more efficient wafer defect detection is achieved.
Patent Information
- Application Number
- CN202411779222.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-05
- Publication Date
- 2025-04-29
AI Technical Summary
The prior art has low detection accuracy in wafer defect detection, resulting in poor detection effect.
By adding an attention module between the encoder and the decoder in the wafer defect detection model, the feature extraction capability of the wafer detection model is improved, thereby improving the detection accuracy of the target image.
Improve the accuracy and accuracy of wafer defect detection and enhance the ability to classify wafer types.
Smart Images
Figure CN119904412A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image detection technology, and in particular to a wafer defect detection method, device, equipment, storage medium and program product. Background Art
[0002] Wafers are silicon chips used to make semiconductor silicon integrated circuits. They are widely used in the field of microelectronics and are the basis of important components such as chips, solar panels, and LED light sources. The process of producing wafers is very complicated and requires multiple steps. Therefore, the production process of wafers can easily lead to defects in wafers. Wafer defects will have a negative impact on the performance, reliability, and life of the chip. Therefore, wafer defects need to be detected during the production process.
[0003] Currently, wafer defect detection is usually performed through target detection algorithms or image classification algorithms. However, the existing algorithms have low detection accuracy and poor detection effect. Summary of the invention
[0004] Based on this, it is necessary to provide a wafer defect detection method, device, equipment, storage medium and program product that can improve the accuracy of wafer defect detection in response to the above technical problems.
[0005] In a first aspect, the present application provides a wafer defect detection method, comprising:
[0006] Acquire a target image of the wafer surface;
[0007] Inputting the target image into the wafer defect detection model to obtain an output result of the wafer defect detection model, wherein the wafer defect detection model includes an encoder and a decoder having a symmetrical relationship, and the encoder and the decoder are connected through an attention module;
[0008] The detection result of the target image is determined according to the output result, and the detection result includes the type of the wafer.
[0009] In the above embodiment, first, a target image of the wafer surface is acquired, and then the target image is input into a wafer defect detection model to obtain an output result of the wafer defect detection model, wherein the wafer defect detection model includes an encoder and a decoder having a symmetrical relationship, and the encoder and the decoder are connected through an attention module, and finally, the detection result of the target image is determined according to the output result, and the detection result includes the type of wafer. In this way, by adding an attention module between the encoder and the decoder in the wafer defect detection model, the feature extraction capability of the wafer detection model is improved, thereby improving the detection accuracy of the target image, that is, improving the accuracy of defect detection of the wafer.
[0010] In one embodiment, the target image is input into a wafer defect detection model to obtain an output result of the wafer defect detection model, including:
[0011] Input the target image into the encoder and obtain the output result of the encoder;
[0012] Input the output result of the encoder into the attention module to obtain the output result of the attention module;
[0013] The output of the attention module is input into the decoder to obtain the output of the wafer defect detection model.
[0014] In the above embodiment, the target image is passed through the encoder, attention module and decoder of the wafer defect detection model to obtain an output result. The classification result of the target image can be determined based on the output result, thereby realizing defect detection of the wafer.
[0015] In one embodiment, the encoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a first multi-layer perceptron, and a second multi-layer perceptron. The target image is input into the encoder to obtain an output result of the encoder, including:
[0016] The target image is input into the encoder, and the target image is downsampled and feature extracted through the first convolution layer, the second convolution layer and the third convolution layer;
[0017] The output of the third convolutional layer is input into the first multi-layer perceptron for processing, and the output of the first multi-layer perceptron is input into the second multi-layer perceptron for processing to obtain the processing result of the encoder.
[0018] In the above embodiment, the encoder includes three convolutional layers and two multi-layer perceptrons to implement downsampling and feature extraction of the target image.
[0019] In one embodiment, the decoder includes a third multi-layer perceptron, a fourth multi-layer perceptron, a fourth convolutional layer, a fifth convolutional layer, and a sixth convolutional layer, and the output result of the attention module is input into the decoder to obtain the output result of the wafer defect detection model, including:
[0020] Input the output of the attention module into the third multi-layer perceptron for processing, and input the output of the third multi-layer perceptron into the fourth multi-layer perceptron for processing, to obtain a processing result of the fourth multi-layer perceptron;
[0021] The processing result of the fourth multi-layer perceptron is up-sampled through the fourth convolutional layer, the fifth convolutional layer and the sixth convolutional layer to obtain the output result of the wafer defect detection model.
[0022] In the above embodiment, the decoder includes three convolutional layers and two multi-layer perceptrons to achieve upsampling of the target image and restore the target image to the input resolution.
[0023] In one embodiment, the first convolutional layer is connected to the sixth convolutional layer via a residual connection, the second convolutional layer is connected to the fifth convolutional layer via a residual connection, and the third convolutional layer is connected to the fourth convolutional layer via a residual connection;
[0024] The attention module is obtained by fusing the SimAM attention mechanism, the QKV attention mechanism and the double-layer KAN layer.
[0025] In the above embodiments, the attention module in the wafer defect detection model is obtained by fusing multiple attention mechanisms, which improves the calculation accuracy and speed of the wafer defect detection model and improves the accuracy of wafer defect detection.
[0026] In one embodiment, acquiring a target image of a wafer surface includes:
[0027] When preset conditions are met, video data of the wafer is acquired;
[0028] Performing frame extraction processing on the video data to obtain a first image;
[0029] The first image is subjected to resolution adjustment processing and converted into a red, green and blue three-channel format to obtain a target image.
[0030] In the above embodiment, the video data of the wafer can be acquired through the measuring platform, and the video data can be processed to further obtain a target image including a wafer surface image, thereby realizing defect detection processing on the wafer.
[0031] In a second aspect, the present application also provides a wafer defect detection device, comprising:
[0032] An acquisition module, used for acquiring a target image of a wafer surface;
[0033] An input module, used for inputting a target image into a wafer defect detection model to obtain an output result of the wafer defect detection model, wherein the wafer defect detection model includes an encoder and a decoder having a symmetrical relationship, and the encoder and the decoder are connected through an attention module;
[0034] The determination module is used to determine the detection result of the target image according to the output result, and the detection result includes the type of the wafer.
[0035] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the wafer defect detection method described in any one of the first aspects above is implemented.
[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the wafer defect detection method described in any one of the first aspects above is implemented.
[0037] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the wafer defect detection method described in any one of the first aspects above.
[0038] The above-mentioned wafer defect detection method, device, equipment, storage medium and program product first obtain a target image of the wafer surface, then input the target image into a wafer defect detection model to obtain the output result of the wafer defect detection model, wherein the wafer defect detection model includes an encoder and a decoder with a symmetrical relationship, and the encoder and the decoder are connected through an attention module. Finally, the detection result of the target image is determined according to the output result, and the detection result includes the type of wafer. In this way, by adding an attention module between the encoder and the decoder in the wafer defect detection model, the feature extraction capability of the wafer detection model is improved, thereby improving the detection accuracy of the target image, that is, improving the accuracy of wafer defect detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0040] Figure 1 is an application environment diagram of a wafer defect detection method in an embodiment;
[0041] Figure 2 1 is a schematic diagram of a process of a wafer defect detection method in an embodiment;
[0042] Figure 3 A schematic diagram of a measurement platform of a wafer defect detection method according to an embodiment;
[0043] Figure 4 is a schematic flow chart of a wafer defect detection method in another embodiment;
[0044] Figure 5 A schematic flow chart of the processing steps of a wafer defect detection model encoder in one embodiment;
[0045] Figure 6 A schematic diagram of the structure of a wafer defect detection model in another embodiment;
[0046] Figure 7 A schematic flow chart of processing steps of a wafer defect detection model decoder in one embodiment;
[0047] Figure 8 A schematic diagram of a process for obtaining a target image in another embodiment;
[0048] Fig. 9 is a schematic flow chart of a wafer defect detection method in another embodiment;
[0049] Fig.10 A schematic diagram of a deployment scheme of a wafer defect detection method in an embodiment;
[0050] Fig.11 is a structural block diagram of a wafer defect detection device in one embodiment;
[0051] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0053] The wafer defect detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the measuring station 102 communicates with the server 104 through the network, and the server 104 communicates with the display device 106 through the network. The data storage system can store the data that the server 104 needs to process and the historical data of wafer defect detection. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the measuring station 102 is used to place the wafer to be inspected, and obtain the surface image of the wafer by setting a camera. The server 104 can be an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers, or it can be a cloud server providing cloud computing services. The display device 106 can be a display, a display screen or a screen, etc., for displaying the wafer defect detection results.
[0054] In an exemplary embodiment, Figure 2 As shown, a wafer defect detection method is provided, which is applied to Figure 1 The server in the example is used to illustrate, including the following steps 201 to 203. Among them:
[0055] Step 201, acquiring a target image of the wafer surface.
[0056] Optionally, the target image is collected by the measuring table and then sent to the server. The step of the measuring table collecting the target image of the wafer surface may include, first, fixing the wafer sample to be measured on the measuring table, and calibrating the position of the wafer sample so that the camera can be aligned with the wafer sample, and then photographing the wafer through the camera of the measuring table. Optionally, it is necessary to obtain the image of the wafer surface by deep ultraviolet single-wavelength high-power laser illumination and optical bright field imaging with high resolution and large imaging field of view. For large wafer samples, the image can be scanned from top to bottom and from left to right in sequence, and the wafer sample can be divided into multiple pictures. For small wafers, the complete image can be directly formed into one picture. Finally, the resolution of the captured wafer sample image is adjusted. Optionally, the sample image size can be scaled to a resolution of 256*256 and then converted to a format to obtain the target image. Optionally, the scaled resolution can also be other resolutions, which is not limited in the embodiments of the present application. Among them, the data acquisition state of the wafer measuring table can be as follows: Figure 3 shown.
[0057] Step 202, inputting the target image into the wafer defect detection model to obtain the output result of the wafer defect detection model.
[0058] Among them, the wafer defect detection model includes an encoder and a decoder with a symmetrical relationship, and the encoder and the decoder are connected through an attention module. Optionally, the wafer defect detection model can be a deep learning image segmentation algorithm designed based on the KAN attention mechanism (KANAttention). The overall structure of the wafer defect detection model is a U-shaped structure, with an encoder on the left and a decoder on the right in the symmetrical structure, and an attention module between the encoder and the decoder. The encoder is used for feature extraction, and the bottom of the U-shaped structure is a deep feature. For deep features, the KAN attention mechanism can extract global fine features, thereby improving the feature extraction capability of the wafer defect detection model.
[0059] Optionally, the training process of the wafer defect detection model may include obtaining sample images of multiple wafer surfaces, annotating and preprocessing the sample images to obtain sample data, and inputting the sample data into the initial wafer defect detection model for training to obtain the wafer defect detection model. Among them, the structure of the initial wafer defect detection model is as described above. Optionally, when annotating the sample images, Labelme, a general annotation tool for deep learning, can be used for annotation, and all defects on the sample images of the wafer are circled in fine granularity in turn, and the sample images of wafers without defects do not need to be processed. By fine-grained annotation, the precision of annotation is improved, and the accuracy of wafer defect detection can be further improved.
[0060] Since the annotated results are composed of a large number of coils surrounding wafer defects, it is not convenient to use them directly for model training, so data preprocessing is required. When the sample image is preprocessed, the annotated results are processed into a black and white mask. First, the coils containing wafer defects in the annotation results of each sample image are filled with white pixel blocks, and the defect-free areas are all filled with black pixels. Then normalization is performed, that is, all defect positions are 255 pixels, all defect-free positions are 0 pixels, and there are no intermediate transition pixels. Finally, the processed mask image is saved as a single-channel image as the final annotation result. For wafers without defects, OpenCV is used to directly generate a single-channel image with all 0 pixels as the final annotation result. Optionally, the above annotation and data preprocessing steps can also be implemented by clipping using Photoshop software, and the annotation results obtained in this way are more refined. By improving the precision of the annotation results, the detection accuracy of the wafer defect detection model can be further improved.
[0061] Optionally, the training process can directly use existing Python libraries for training and reasoning. The training framework uses pytorch. Optionally, some training hyperparameters can be set as follows: input resolution is 256*256, optimizer is Adam, batch size is 8, number of iterations is 400, learning rate is 0.0001, number of categories is 1, and cosine annealing learning rate is used to improve the model learning effect.
[0062] Step 203: Determine the detection result of the target image according to the output result.
[0063] Among them, the detection results include the type of wafer. The types of wafers may include normal wafers, micro-defect wafers, and defective wafers. Optionally, in order to facilitate user viewing, the output results of the wafer defect detection model may be classified, not only outputting the segmentation mask, but also directly displaying wafers without white pixel masks as normal wafers, wafers with a total area of white pixel masks less than a preset defect threshold as micro-defect wafers, and other wafers with a total area of white pixel masks greater than a preset defect threshold as defective wafers. According to the detection results, it is convenient for R&D personnel to pay attention to defective wafers and improve the efficiency of wafer detection and R&D.
[0064] In the above embodiment, first, a target image of the wafer surface is acquired, and then the target image is input into a wafer defect detection model to obtain an output result of the wafer defect detection model, wherein the wafer defect detection model includes an encoder and a decoder having a symmetrical relationship, and the encoder and the decoder are connected through an attention module, and finally, the detection result of the target image is determined according to the output result, and the detection result includes the type of wafer. In this way, by adding an attention module between the encoder and the decoder in the wafer defect detection model, the feature extraction capability of the wafer detection model is improved, thereby improving the detection accuracy of the target image, that is, improving the accuracy of defect detection of the wafer.
[0065] In one embodiment, the above step 202 can be performed as follows: Figure 4 As shown, including:
[0066] Step 401: input a target image into an encoder to obtain an output result of the encoder.
[0067] After the wafer defect detection model is trained, the target image is input into the encoder of the wafer detection model. Optionally, the encoder may include multiple convolutional layers and multiple multi-layer perceptrons. After the convolutional layers complete the downsampling and feature extraction processing, the multi-layer perceptron is used to obtain the output result of the encoder. The encoder may be composed of a KAN module, and the KAN module is composed of a KAN layer.
[0068] Step 402: input the output result of the encoder into the attention module to obtain the output result of the attention module.
[0069] Optionally, the U-shaped structure of the wafer defect detection model has deep features at the bottom of the U-shaped structure. Since KAN can obtain finer features than MLP (Multi-Layer Perceptron) and can obtain global features compared to CNN (Convolutional Neural Networks), the deep features at the bottom of the U-shaped structure are extracted through the attention module, that is, the three-layer KAN layer structure to extract global fine features. However, due to the large amount of calculation of KAN, it will significantly affect the speed of the algorithm. Therefore, placing KAN at the bottom of the U-shaped structure, the deep layer with a small feature map and less module usage can significantly reduce the problem of slow algorithm speed.
[0070] Step 403, input the output result of the attention module into the decoder to obtain the output result of the wafer defect detection model.
[0071] Optionally, the decoder may include multiple convolutional layers and multiple multi-layer perceptrons, and the feature resolution is doubled through the multiple convolutional layers until the input resolution is restored. The multiple convolutional layers of the decoder are symmetrical with the multiple convolutional layers of the encoder.
[0072] In the above embodiment, the target image is passed through the encoder, attention module and decoder of the wafer defect detection model to obtain an output result. The classification result of the target image can be determined based on the output result, thereby realizing defect detection of the wafer.
[0073] In one embodiment, the encoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a first multi-layer perceptron, and a second multi-layer perceptron. The above step 401 is as follows: Figure 5 As shown, including:
[0074] Step 501: input a target image into an encoder, and perform downsampling and feature extraction processing on the target image through a first convolutional layer, a second convolutional layer, and a third convolutional layer.
[0075] Optionally, the first three shallow feature extraction layers of the encoder include three convolutional layers, which complete three downsampling processes and feature extraction processes.
[0076] Step 502: Input the output of the third convolutional layer to the first multi-layer perceptron for processing, and input the output of the first multi-layer perceptron to the second multi-layer perceptron for processing, to obtain a processing result of the encoder.
[0077] Optionally, the network structure diagram of the encoder is as follows Figure 6 As shown in Figure 2. Multilayer perceptron is a basic artificial neural network model, which has a multilayer structure consisting of multiple neurons. The encoder includes two multilayer perceptrons, each of which is connected to an attention module.
[0078] In the above embodiment, the encoder includes three convolutional layers and two multi-layer perceptrons to implement downsampling and feature extraction of the target image.
[0079] In an embodiment of the present application, the decoder includes a third multi-layer perceptron, a fourth multi-layer perceptron, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer. The above step 403 is as follows: Figure 7 As shown, it may include:
[0080] Step 701: Input the output of the attention module to the third multi-layer perceptron for processing, and input the output of the third multi-layer perceptron to the fourth multi-layer perceptron for processing to obtain the processing result of the fourth multi-layer perceptron.
[0081] Optional, please refer to the decoder network structure diagram Figure 6 ,The output of the attention module is processed by two layers of multi-layer perceptrons and then input into the convolutional layer.
[0082] Step 702, up-sample the processing result of the fourth multi-layer perceptron through the fourth convolutional layer, the fifth convolutional layer and the sixth convolutional layer to obtain the output result of the wafer defect detection model.
[0083] In the above embodiment, the decoder includes three convolutional layers and two multi-layer perceptrons to achieve upsampling of the target image and restore the target image to the input resolution.
[0084] Optionally, the first convolution layer is connected to the sixth convolution layer through a residual connection, the second convolution layer is connected to the fifth convolution layer through a residual connection, and the third convolution layer is connected to the fourth convolution layer through a residual connection; the attention module is obtained by fusion of SimAM attention mechanism, QKV attention mechanism and double-layer KAN layer.
[0085] Optionally, adding an attention module between the KAN of the encoder and decoder can improve the feature extraction ability of the model. The attention module is the PSK module, which is a fusion algorithm of the SimAM attention mechanism, the QKV attention mechanism and the double-layer KAN layer. Among them, the SimAM attention mechanism is a lightweight, parameter-free attention mechanism designed for convolutional neural networks, and has significant advantages in improving the performance of convolutional neural networks. The QKV attention mechanism is an attention mechanism used to process sequence data in deep learning. It captures the global dependencies in the input sequence by introducing three vectors: query, key and value. The double-layer KAN layer is a new neural network architecture based on the Kolmogorov-Arnold representation theorem. It improves accuracy and interpretability by using learnable activation functions on weights rather than nodes. The PSK module obtained by fusing the three algorithms has high detection accuracy and fast algorithm calculation speed. The Dice accuracy of the wafer defect detection model can be directly improved by about 0.9%. Please continue to refer to the complete structure diagram of the wafer defect detection model Figure 6 .
[0086] In the above embodiments, the attention module in the wafer defect detection model is obtained by fusing multiple attention mechanisms, which improves the calculation accuracy and speed of the wafer defect detection model and improves the accuracy of wafer defect detection.
[0087] In one embodiment, a target image of the wafer surface is acquired, such as Figure 8 As shown, including:
[0088] Step 801, obtaining video data of a wafer when a preset condition is met.
[0089] Optionally, the preset condition is that the wafer sample to be inspected has been placed in a suitable inspection position and calibrated, and the lighting and camera meet the above-mentioned wafer inspection conditions. The rotation speed of the measuring table is adjusted so that the camera can be controlled to capture video data of the wafer. After shooting, the video data is sent to the server.
[0090] Step 802: Perform frame extraction processing on the video data to obtain a first image.
[0091] The video is subjected to frame extraction processing according to a preset ratio to obtain a first image. The preset ratio can be determined according to the rotation speed of the measuring table to ensure that the first image can include the entire surface of the wafer. Optionally, when the wafer is large, the first image can include multiple images, and when the wafer is small, the first image can be one.
[0092] Step 803: perform resolution adjustment processing on the first image and convert it into a red, green and blue three-channel format to obtain a target image.
[0093] Optionally, in order to improve the detection accuracy of the wafer defect detection model, the resolution of the first image is adjusted. The resolution test will be performed before inputting the model. If it does not meet the requirements, further adjustment is required. After the resolution is adjusted, the first image is converted into data in a red, green and blue three-channel format to obtain a target image, which can be input into the trained wafer defect detection model for classification detection.
[0094] In the above embodiment, the video data of the wafer can be acquired through the measuring platform, and the video data can be processed to further obtain a target image including a wafer surface image, thereby realizing defect detection processing on the wafer.
[0095] In the embodiments of the present application, Fig. 9 As shown, a wafer defect detection method is provided, comprising:
[0096] Step 901, acquiring a target image of the wafer surface.
[0097] Step 902: input the target image into the encoder to obtain the output result of the encoder.
[0098] Step 903: input the output result of the encoder into the attention module to obtain the output result of the attention module.
[0099] Step 904, input the output result of the attention module into the decoder to obtain the output result of the wafer defect detection model.
[0100] Step 905: Determine the detection result of the target image according to the output result.
[0101] In an embodiment of the present application, the deployment scheme of the wafer defect detection method can be as follows: Fig.10 As shown, in order to facilitate R&D personnel to view wafer defects and historical wafer detection data, the wafer defect detection model is deployed through the server, and the wafer defect detection results are displayed on a large screen. In order to increase the calculation speed of the wafer defect detection model, libraries such as TensorRT and VLLM (Virtual Large Language Model) can be used to further accelerate. Generally, due to the large production volume of wafers, in order to prevent misdetection and missed detection of wafer defects, it is usually necessary to perform multiple defect detections on the wafer, which increases the computational workload of the algorithm. Optionally, the wafer defect detection model can be deployed on a large deep learning server, such as a multi-card multi-node server, to increase the calculation speed of the model.
[0102] Optionally, the camera acquires the target image of the wafer on the wafer production line, sends it to the server for video frame extraction, data format adjustment, and detection through the wafer defect detection model. After detection, the recognition result of the wafer defect detection model can be transmitted to the front-end large screen through WebSocket (a full-duplex communication protocol), and the detection data is saved in the back-end database. The front-end large screen can display real-time results and query historical records. Due to the fusion algorithm and data annotation method in the attention module of the wafer defect detection model of this application, the segmentation results of the wafer target image are extremely fine, and the Dice accuracy of the detection can reach about 93% or more. At the same time, by adding a database to save historical detection data and displaying the detection results in real time on a large screen, the flexibility and practicality of wafer defect detection are improved.
[0103] Normally, a wafer production line requires an absolutely dust-free environment, which places extremely high demands on the cleanliness and skills of the inspectors. The mass production of wafers also determines that it places great demand on manpower. In the above embodiment, the deployment scheme of the wafer defect detection method is used to realize automatic defect detection of wafers without the need for human participation. At the same time, the segmentation accuracy of the wafer target image is improved. By optimizing the wafer defect detection model algorithm, the detection accuracy and speed of the algorithm are improved, and the scope of application is wider and the detection effect is better.
[0104] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0105] Based on the same inventive concept, the embodiment of the present application also provides a wafer defect detection device for implementing the wafer defect detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more wafer defect detection device embodiments provided below can refer to the limitations of the wafer defect detection method above, and will not be repeated here.
[0106] In an exemplary embodiment, Fig.11 As shown, a wafer defect detection device 1100 is provided, including: an acquisition module 1101, an input module 1102 and a determination module 1103, wherein:
[0107] An acquisition module 1101 is used to acquire a target image of a wafer surface;
[0108] An input module 1102 is used to input a target image into a wafer defect detection model to obtain an output result of the wafer defect detection model, wherein the wafer defect detection model includes an encoder and a decoder having a symmetrical relationship, and the encoder and the decoder are connected through an attention module;
[0109] The determination module 1103 is used to determine the detection result of the target image according to the output result, and the detection result includes the type of the wafer.
[0110] In one embodiment, the input module 1102 is specifically used to input the target image into the encoder to obtain the output result of the encoder; input the output result of the encoder into the attention module to obtain the output result of the attention module; input the output result of the attention module into the decoder to obtain the output result of the wafer defect detection model.
[0111] In one embodiment, the encoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a first multi-layer perceptron and a second multi-layer perceptron. The input module 1102 is specifically used to input the target image into the encoder, perform downsampling and feature extraction on the target image through the first convolutional layer, the second convolutional layer and the third convolutional layer; input the output of the third convolutional layer into the first multi-layer perceptron for processing, and input the output of the first multi-layer perceptron into the second multi-layer perceptron for processing, to obtain the processing result of the encoder.
[0112] In one embodiment, the decoder includes a third multi-layer perceptron, a fourth multi-layer perceptron, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer. The input module 1102 is specifically used to input the output of the attention module into the third multi-layer perceptron for processing, and input the output of the third multi-layer perceptron into the fourth multi-layer perceptron for processing to obtain the processing result of the fourth multi-layer perceptron; the processing result of the fourth multi-layer perceptron is up-sampled through the fourth convolutional layer, the fifth convolutional layer and the sixth convolutional layer to obtain the output result of the wafer defect detection model.
[0113] In one of the embodiments, the first convolutional layer is connected to the sixth convolutional layer through a residual connection, the second convolutional layer is connected to the fifth convolutional layer through a residual connection, and the third convolutional layer is connected to the fourth convolutional layer through a residual connection; the attention module is obtained by fusion of SimAM attention mechanism, QKV attention mechanism and double-layer KAN layer.
[0114] In one embodiment, the acquisition module 1101 is specifically used to acquire video data of a wafer when preset conditions are met; perform frame extraction on the video data to obtain a first image; perform resolution adjustment on the first image and convert it into a red, green and blue three-channel format to obtain a target image.
[0115] Each module in the above-mentioned wafer defect detection device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.
[0116] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.12As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store target image data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a wafer defect detection method is implemented.
[0117] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0118] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: acquiring a target image of a wafer surface; inputting the target image into a wafer defect detection model to obtain an output result of the wafer defect detection model, wherein the wafer defect detection model includes an encoder and a decoder having a symmetrical relationship, and the encoder and the decoder are connected through an attention module; and determining a detection result of the target image based on the output result, wherein the detection result includes the type of wafer.
[0119] In one embodiment, when the processor executes the computer program, the following steps are also implemented: inputting the target image into the encoder to obtain the output result of the encoder; inputting the output result of the encoder into the attention module to obtain the output result of the attention module; inputting the output result of the attention module into the decoder to obtain the output result of the wafer defect detection model.
[0120] In one embodiment, the encoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a first multi-layer perceptron, and a second multi-layer perceptron. When the processor executes the computer program, the following steps are also implemented: a target image is input into the encoder, and the target image is downsampled and feature extracted through the first convolutional layer, the second convolutional layer, and the third convolutional layer; the output of the third convolutional layer is input into the first multi-layer perceptron for processing, and the output of the first multi-layer perceptron is input into the second multi-layer perceptron for processing, to obtain the processing result of the encoder.
[0121] In one embodiment, the decoder includes a third multi-layer perceptron, a fourth multi-layer perceptron, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer. When the processor executes the computer program, the following steps are also implemented: the output of the attention module is input into the third multi-layer perceptron for processing, and the output of the third multi-layer perceptron is input into the fourth multi-layer perceptron for processing to obtain the processing result of the fourth multi-layer perceptron; the processing result of the fourth multi-layer perceptron is up-sampled through the fourth convolutional layer, the fifth convolutional layer and the sixth convolutional layer to obtain the output result of the wafer defect detection model.
[0122] In one embodiment, the first convolutional layer is connected to the sixth convolutional layer through a residual connection, the second convolutional layer is connected to the fifth convolutional layer through a residual connection, and the third convolutional layer is connected to the fourth convolutional layer through a residual connection; the attention module is obtained by fusion of SimAM attention mechanism, QKV attention mechanism and double-layer KAN layer.
[0123] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when preset conditions are met, video data of the wafer is acquired; frame extraction is performed on the video data to obtain a first image; resolution adjustment is performed on the first image, and it is converted into a red, green and blue three-channel format to obtain a target image.
[0124] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: acquiring a target image of a wafer surface; inputting the target image into a wafer defect detection model to obtain an output result of the wafer defect detection model, wherein the wafer defect detection model includes an encoder and a decoder having a symmetrical relationship, and the encoder and the decoder are connected through an attention module; determining a detection result of the target image based on the output result, and the detection result includes the type of wafer.
[0125] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: inputting the target image into the encoder to obtain the output result of the encoder; inputting the output result of the encoder into the attention module to obtain the output result of the attention module; inputting the output result of the attention module into the decoder to obtain the output result of the wafer defect detection model.
[0126] In one embodiment, the encoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a first multi-layer perceptron, and a second multi-layer perceptron. When the computer program is executed by the processor, the following steps are also implemented: a target image is input into the encoder, and the target image is downsampled and feature extracted through the first convolutional layer, the second convolutional layer, and the third convolutional layer; the output of the third convolutional layer is input into the first multi-layer perceptron for processing, and the output of the first multi-layer perceptron is input into the second multi-layer perceptron for processing, to obtain the processing result of the encoder.
[0127] In one embodiment, the decoder includes a third multi-layer perceptron, a fourth multi-layer perceptron, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer. When the computer program is executed by the processor, the following steps are also implemented: the output of the attention module is input into the third multi-layer perceptron for processing, and the output of the third multi-layer perceptron is input into the fourth multi-layer perceptron for processing to obtain the processing result of the fourth multi-layer perceptron; the processing result of the fourth multi-layer perceptron is up-sampled through the fourth convolutional layer, the fifth convolutional layer and the sixth convolutional layer to obtain the output result of the wafer defect detection model.
[0128] In one embodiment, the first convolutional layer is connected to the sixth convolutional layer through a residual connection, the second convolutional layer is connected to the fifth convolutional layer through a residual connection, and the third convolutional layer is connected to the fourth convolutional layer through a residual connection; the attention module is obtained by fusion of SimAM attention mechanism, QKV attention mechanism and double-layer KAN layer.
[0129] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when preset conditions are met, video data of the wafer is acquired; frame extraction is performed on the video data to obtain a first image; resolution adjustment is performed on the first image, and it is converted into a red, green and blue three-channel format to obtain a target image.
[0130] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the following steps: acquiring a target image of a wafer surface; inputting the target image into a wafer defect detection model to obtain an output result of the wafer defect detection model, wherein the wafer defect detection model includes an encoder and a decoder having a symmetrical relationship, and the encoder and the decoder are connected through an attention module; determining a detection result of the target image based on the output result, wherein the detection result includes the type of the wafer.
[0131] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: inputting the target image into the encoder to obtain the output result of the encoder; inputting the output result of the encoder into the attention module to obtain the output result of the attention module; inputting the output result of the attention module into the decoder to obtain the output result of the wafer defect detection model.
[0132] In one embodiment, the encoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a first multi-layer perceptron, and a second multi-layer perceptron. When the computer program is executed by the processor, the following steps are also implemented: a target image is input into the encoder, and the target image is downsampled and feature extracted through the first convolutional layer, the second convolutional layer, and the third convolutional layer; the output of the third convolutional layer is input into the first multi-layer perceptron for processing, and the output of the first multi-layer perceptron is input into the second multi-layer perceptron for processing, to obtain the processing result of the encoder.
[0133] In one embodiment, the decoder includes a third multi-layer perceptron, a fourth multi-layer perceptron, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer. When the computer program is executed by the processor, the following steps are also implemented: the output of the attention module is input into the third multi-layer perceptron for processing, and the output of the third multi-layer perceptron is input into the fourth multi-layer perceptron for processing to obtain the processing result of the fourth multi-layer perceptron; the processing result of the fourth multi-layer perceptron is up-sampled through the fourth convolutional layer, the fifth convolutional layer and the sixth convolutional layer to obtain the output result of the wafer defect detection model.
[0134] In one embodiment, the first convolutional layer is connected to the sixth convolutional layer through a residual connection, the second convolutional layer is connected to the fifth convolutional layer through a residual connection, and the third convolutional layer is connected to the fourth convolutional layer through a residual connection; the attention module is obtained by fusion of SimAM attention mechanism, QKV attention mechanism and double-layer KAN layer.
[0135] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when preset conditions are met, video data of the wafer is acquired; frame extraction is performed on the video data to obtain a first image; resolution adjustment is performed on the first image, and it is converted into a red, green and blue three-channel format to obtain a target image.
[0136] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0137] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0138] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0139] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A wafer defect detection method, characterized in that: The method comprises: Acquire a target image of the wafer surface; Inputting the target image into a wafer defect detection model to obtain an output result of the wafer defect detection model, wherein the wafer defect detection model includes an encoder and a decoder having a symmetrical relationship, and the encoder and the decoder are connected through an attention module; A detection result of the target image is determined according to the output result, where the detection result includes a type of wafer.
2. The method according to claim 1, characterized in that: The step of inputting the target image into a wafer defect detection model to obtain an output result of the wafer defect detection model includes: Inputting the target image into the encoder to obtain an output result of the encoder; Inputting the output result of the encoder into the attention module to obtain the output result of the attention module; The output result of the attention module is input into the decoder to obtain the output result of the wafer defect detection model.
3. The method according to claim 2, characterized in that The encoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a first multi-layer perceptron and a second multi-layer perceptron, and inputting the target image into the encoder to obtain an output result of the encoder includes: Inputting the target image into the encoder, and performing downsampling processing and feature extraction processing on the target image through the first convolution layer, the second convolution layer, and the third convolution layer; The output of the third convolutional layer is input into the first multi-layer perceptron for processing, and the output of the first multi-layer perceptron is input into the second multi-layer perceptron for processing to obtain a processing result of the encoder.
4. The method according to claim 3, characterized in that The decoder includes a third multi-layer perceptron, a fourth multi-layer perceptron, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer, and the output result of the attention module is input into the decoder to obtain the output result of the wafer defect detection model, including: Inputting the output of the attention module into the third multi-layer perceptron for processing, and inputting the output of the third multi-layer perceptron into the fourth multi-layer perceptron for processing, to obtain a processing result of the fourth multi-layer perceptron; The processing result of the fourth multi-layer perceptron is up-sampled through the fourth convolutional layer, the fifth convolutional layer and the sixth convolutional layer to obtain the output result of the wafer defect detection model.
5. The method according to claim 4, characterized in that The first convolution layer is connected to the sixth convolution layer through a residual connection, the second convolution layer is connected to the fifth convolution layer through a residual connection, and the third convolution layer is connected to the fourth convolution layer through a residual connection; The attention module is obtained by fusing the SimAM attention mechanism, the QKV attention mechanism and the double-layer KAN layer.
6. The method according to claim 1, characterized in that The step of acquiring a target image of the wafer surface comprises: When a preset condition is met, obtaining video data of the wafer; Performing frame extraction processing on the video data to obtain a first image; The first image is subjected to resolution adjustment processing and converted into a red, green and blue three-channel format to obtain the target image.
7. A wafer defect detection device, characterized in that: The device comprises: An acquisition module, used for acquiring a target image of a wafer surface; An input module, used for inputting the target image into a wafer defect detection model to obtain an output result of the wafer defect detection model, wherein the wafer defect detection model includes an encoder and a decoder having a symmetrical relationship, and the encoder and the decoder are connected through an attention module; A determination module is used to determine a detection result of the target image according to the output result, wherein the detection result includes a type of wafer.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.