Deep learning-based wafer production process defect classification analysis method and equipment
By applying a deep neural network model based on deep learning in the wafer production process, the problems of high requirements for defect detection environment, poor generalization and insufficient real-time performance in the prior art are solved, and efficient, accurate and real-time defect classification is achieved.
Patent Information
- Application Number
- CN202510064799.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-15
- Publication Date
- 2025-04-25
AI Technical Summary
In the existing wafer production process, defect detection relies on manual or digital image processing, and there are problems such as high detection environment requirements, poor generalization and insufficient real-time performance.
Using a deep neural network model based on deep learning, a network structure containing convolution modules and C2f modules is constructed, and a variety of data enhancement methods are combined to achieve efficient classification of wafer defects.
It improves the accuracy and robustness of the model, reduces the calculation amount, realizes real-time detection, and meets the demand for fast and accurate detection in wafer production processes.
Smart Images
Figure CN119888360A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of semiconductor wafer production process, and in particular to an analysis method, device and storage medium for wafer production process defect classification based on deep learning. Background Art
[0002] Wafer production process refers to a series of process flows in the semiconductor manufacturing process to process silicon wafers (also called wafers) into chips. The process involves multiple steps, including wafer cleaning, chemical etching, photolithography, ion implantation, diffusion, deposition, etc. In the existing production process, although there are high-precision and high-quality equipment and materials, due to various reasons in a complex manufacturing environment, the wafers obtained after each process may have defects. For example, the wafer may have surface scratches, uneven film, contamination, crystal defects, electrochemical corrosion and other defects. Although they seem small, these defects will eventually have a great impact on the performance of the entire product.
[0003] Therefore, it is very necessary to control and inspect the quality of wafers in production. In addition to controlling and inspecting the raw materials, auxiliary materials, and process rules and technical conditions of each specific product, it is particularly important to conduct strict quality inspection on the finished products after processing.
[0004] Manual inspection of wafer production processes is generally carried out by observing the appearance and comparing it. For example, sample comparison can be used to determine surface scratches, surface particles, fine cracks, pores, pits and other defects. However, this usually requires professional inspection personnel, and the data volume is large and the classification accuracy requirements are high, so it is impossible to achieve full coverage.
[0005] With the development of digital image processing technology, the use of computer detection has gradually replaced manual detection. However, the detection method of digital image processing has high requirements for the detection environment and poor generalization, which is not conducive to large-scale promotion. Recently, with the application of deep learning technology, detection methods based on deep learning have begun to appear, various classification networks have emerged, and their reasoning accuracy has been continuously improved. However, it also brings some problems. The number of layers of the model continues to increase, and the corresponding parameters have also increased exponentially. At the same time, most models require high-computing hardware support, so they cannot meet the requirements of real-time classification. Summary of the invention
[0006] In view of the shortcomings of existing algorithms, with the support of hardware that meets the high computing power of the model, the present invention proposes a deep learning-based wafer production process defect classification analysis method and equipment to solve the problems that the current classification algorithm model has many parameters, which is not conducive to improving the model reasoning speed.
[0007] To achieve the above object, the present invention adopts the following technical solutions:
[0008] A method for analyzing wafer production process defect classification based on deep learning, comprising the following steps:
[0009] Step 1: Collect source images and preprocess the images to establish a benchmark data set; the preprocessing includes: flipping, rotating and randomly deleting a certain area in the image.
[0010] Step 2: Build a deep neural network model, input the preprocessed image into the deep neural network model, calculate the loss value through the cross entropy with L2 regularization term, and classify the image through the classifier;
[0011] Step 3: Input the image to be classified into the deep neural network model and output the inference result.
[0012] Furthermore, the backbone of the deep neural network model includes five convolution modules and four C2f modules. The connection order is convolution module, convolution module, C2f module, convolution module, C2f module, convolution module, C2f module, convolution module, C2f module, to complete the image feature extraction. The head part is a classifier module to complete the score classification of the image. The number of convolution modules is 1, and the number of C2f modules n is 1, 2, 2, 1.
[0013] Furthermore, the convolution module is Conv2d+BatchNorm2d+SiLU, and the C2f module first splits the features after being processed by the convolution module into two parts of features, one part of the features is retained without any processing, and the other part is processed by several BottleNecks; each BottleNeck will be divided into two channels, one is to pass the processed features to the next BottleNeck, and the other is retained for the subsequent concat connection, and finally all the features are merged after passing through n BottleNecks.
[0014] Where h is the image height, w is the image width, and d is the image channel. In the first layer, the input is h×w×d data, and after passing through the convolution module with a 3*3 convolution kernel, a stride of 2, and an output channel number of ch_out, the output is The second layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The third layer, the input is The data passes through the C2f module and the output is The fourth layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The fifth layer, the input is The data passes through the C2f module and the output is The sixth layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The seventh layer, the input is The data passes through the C2f module and the output is The eighth layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The ninth layer, the input is The data passes through the C2f module and the output is
[0015] Furthermore, the C2f module structure is as follows:
[0016] First, a convolution is performed on the input feature map to double the number of channels. Specifically, after the h×w×d data passes through a 1*1 convolution kernel, a stride of 1, and a padding of 0, the output channel number is ch_out=2×d, and the output is h×w×ch_out; after the output data is split, the number of channels becomes half of the original h×w×d, as follows: the first branch, the input is h×w×ch_out data, after Split, the output is two branches of h×w×d; the feature map of one branch is not processed, and the other branch is passed to n BottleNeck modules for feature fusion operation, 1 BottleNeck The specific modules are as follows: the input data of h×w×d is continuously passed through two 3×3 convolution kernels, the stride is 1, and the padding is 1 convolution modules, and two branches are obtained with the output data of h×w×d; the output feature maps of the n Bottleneck modules are concatenated with the original feature map, as follows: the n branches with the output of h×w×d are concatenated by Concat to obtain the output of h×w×0.5(n+2)d; finally, the concatenated feature map is convolved once to output the feature map of the target number of channels, as follows: the input data of h×w×0.5(n+2)d is passed through a 1×1 convolution kernel, the stride is 1, and the padding is 0 convolution module to obtain the output data of h×w×d.
[0017] Furthermore, the formula for calculating the loss value of the cross entropy with L2 regularization term is:
[0018]
[0019] Among them, nc is the number of categories, yi is the true value, pi is the predicted value, and α is the weight coefficient of the regularization term.
[0020] Step 3: Input the image to be classified into the deep neural network model and output the inference result.
[0021] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.
[0022] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.
[0023] It can be seen from the above technical scheme that the analysis method for wafer production process defect classification based on deep learning of the present invention uses a deep neural network to aggregate multi-scale information by splicing the outputs of different Bottleneck modules and the original feature map together; at the same time, the convolution operation in the C2f module can effectively compress the feature map, reduce the amount of calculation, greatly improve the model training and reasoning speed, and meet the requirements of real-time detection;
[0024] The wafer production process defect method based on deep learning uses a variety of data enhancement methods to effectively improve the accuracy and robustness of the model; BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a flowchart of the present invention;
[0026] Figure 2 is an operating principle diagram of an embodiment of the present invention;
[0027] Figure 3 It is a structural diagram of a C2F module according to an embodiment of the present invention. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.
[0029] like Figure 1 As shown, the analysis method for wafer production process defect classification based on deep learning described in this embodiment includes the following steps:
[0030] S1. Collect source images and preprocess the images to establish a benchmark data set; the preprocessing includes: flipping, rotating and randomly deleting a certain area in the image.
[0031] S2, build a deep neural network model, input the preprocessed image into the deep neural network model, calculate the loss value through the cross entropy with L2 regularization term, and classify the image through the classifier;
[0032] S3: Input the image to be classified into the deep neural network model and output the inference result.
[0033] The following combination Figure 2 Specific instructions:
[0034] The backbone of the deep neural network model includes five convolution modules and four C2f modules. The connection order is convolution module, convolution module, C2f module, convolution module, C2f module, convolution module, C2f module, convolution module, C2f module, to complete the image feature extraction. The head part is the classifier module to complete the image score classification. The number of convolution modules is 1, and the number of C2f modules n is 1, 2, 2, 1.
[0035] Specifically, the convolution module is Conv2d+BatchNorm2d+SiLU. The C2f module first splits the features processed by the convolution module into two parts. One part of the features is retained without any processing, and the other part is processed by several BottleNecks. Each BottleNeck will be divided into two channels, one is to pass the processed features to the next BottleNeck, and the other is retained for the subsequent concat connection. Finally, all the features are merged after passing through n BottleNecks.
[0036] Where h is the image height, w is the image width, and d is the image channel. In the first layer, the input is h×w×d data, and after passing through the convolution module with a 3*3 convolution kernel, a stride of 2, and an output channel number of ch_out, the output is The second layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The third layer, the input is The data passes through the C2f module and the output is The fourth layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The fifth layer, the input is The data passes through the C2f module and the output is The sixth layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The seventh layer, the input is The data passes through the C2f module and the output is The eighth layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The ninth layer, the input is The data passes through the C2f module and the output is
[0037] Among them, the C2f module structure is as follows:
[0038] First, a convolution is performed on the input feature map to double the number of channels. Specifically, after the h×w×d data passes through a 1*1 convolution kernel, a stride of 1, and a padding of 0, the output channel number is ch_out=2×d, and the output is h×w×ch_out; after the output data is split, the number of channels becomes half of the original h×w×d, as follows: the first branch, the input is h×w×ch_out data, after Split, the output is two branches of h×w×d; the feature map of one branch is not processed, and the other branch is passed to n BottleNeck modules for feature fusion operation, 1 BottleNeck The specific modules are as follows: the input data of h×w×d is continuously passed through two 3×3 convolution kernels, the stride is 1, and the padding is 1 convolution modules, and two branches are obtained with the output data of h×w×d; the output feature maps of the n Bottleneck modules are concatenated with the original feature map, as follows: the n branches with the output of h×w×d are concatenated by Concat to obtain the output of h×w×0.5(n+2)d; finally, the concatenated feature map is convolved once to output the feature map of the target number of channels, as follows: the input data of h×w×0.5(n+2)d is passed through a 1×1 convolution kernel, the stride is 1, and the padding is 0 convolution module to obtain the output data of h×w×d.
[0039] The formula for calculating the loss value of cross entropy with L2 regularization is:
[0040]
[0041] Among them, nc is the number of categories, yi is the true value, pi is the predicted value, and α is the weight coefficient of the regularization term.
[0042] The following examples illustrate:
[0043] In the deep learning network model construction stage, before extracting features, the pre-acquired image dataset of photoelectron microscope imaging is subjected to data preprocessing and data enhancement. The original image has an unfixed width and height, so the original image is adjusted to a constant width and height. In addition, in order to expand the original dataset and avoid overfitting, the present invention performs data expansion, including horizontal and vertical flipping, random cropping and rotation, random deletion of a certain area in the image, etc., and finally scales the image to a resolution of 480×480×3 as the input of the model.
[0044] The dataset is used to fit the deep learning model; the validation dataset is used to optimize the model hyperparameters; and the test dataset is used to obtain an unbiased estimate of the model performance. In this study, the training, validation, and test datasets account for 7 / 10, 2 / 10, and 1 / 10 of the total number of images.
[0045] Perform feature extraction on the image, and extract features in five sizes: 240×240, 120×120, 60×60, 30×30, and 15×15;
[0046] When the feature size remains unchanged, the C2F module is used, as follows: Figure 2 As shown:
[0047] First, the input feature map is convolved once to double the number of channels of the input. After the output data is split, the number of output channels is reduced to half of the original number. The feature map of one branch is not processed, and the other branch is passed to n BottleNeck modules for feature fusion operation.
[0048] In the BottleNeck module, the input h×w×d data is continuously passed through two 3×3 convolution kernels, a stride of 1, and a padding of 1 convolution module to obtain two branches with output h×w×d data; the output feature maps of the n Bottleneck modules are concat-joined with the original feature map to obtain an output of h×w×0.5(n+2)d; finally, the concatenated feature map is passed through a 1×1 convolution kernel, a stride of 1, and a padding of 0 convolution module to obtain an output of h×w×d data.
[0049] The loss value is calculated using the cross entropy with L2 regularization term. After the loss value is calculated, the model weight is updated using the back propagation algorithm. When the number of iterations reaches the maximum number of iterations or the model tends to fit, the training ends and the model weight file is saved.
[0050] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.
[0051] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.
[0052] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the deep learning-based analysis methods for wafer production process defect classification in the above-mentioned embodiments.
[0053] It is understandable that the system, device and storage medium provided in the embodiments of the present invention correspond to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the above methods.
[0054] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.
[0055] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0056] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0057] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for analyzing wafer production process defect classification based on deep learning, characterized in that: The following steps are included: S1, collect source images, preprocess the images, and establish a benchmark data set; Preprocessing includes: flipping, rotating, and randomly deleting a region in the image; S2, build a deep neural network model, input the preprocessed image into the deep neural network model, calculate the loss value through the cross entropy with L2 regularization term, and classify the image through the classifier; S3: Input the image to be classified into the deep neural network model and output the inference result.
2. The method for analyzing wafer production process defect classification based on deep learning according to claim 1, characterized in that: The backbone of the deep neural network model includes five convolution modules and four C2f modules. The connection order is convolution module 1, convolution module 2, C2f module 1, convolution module 3, C2f module 2, convolution module 4, C2f module 3, convolution module 5, and C2f module 4, which completes the image feature extraction. The head part of the deep neural network model is the classifier module, which completes the scoring classification of the image. The number of convolution modules is 1, and the number n of C2f modules is 1 for C2f module one, 2 for C2f module two, 3 for C2f module three, and 1 for C2f module four.
3. The method for analyzing wafer production process defect classification based on deep learning according to claim 2, characterized in that: The convolution module is Conv2d+BatchNorm2d+SiLU. The C2f module first splits the features processed by the convolution module into two parts. One part of the features is retained without any processing, and the other part is processed by several BottleNecks. Each BottleNeck will be divided into two channels, one is to pass the processed features to the next BottleNeck, and the other is retained for the subsequent concat connection. Finally, all the features are merged after passing through n BottleNecks. Among them, h is the image height, w is the image width, and d is the image channel; In the first layer, the input is h×w×d data, and after passing through the convolution module with a 3*3 convolution kernel, a stride of 2, and an output channel number of ch_out, the output is The second layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The third layer, the input is The data passes through the C2f module and the output is The fourth layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The fifth layer, the input is The data passes through the C2f module and the output is The sixth layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The seventh layer, the input is The data passes through the C2f module and the output is The eighth layer, the input is The data is processed by a 3*3 convolution kernel and a convolution module with a stride of 2, and the output is The ninth layer, the input is The data passes through the C2f module and the output is 4. The method for analyzing wafer production process defect classification based on deep learning according to claim 3 is characterized in that: The C2f module structure includes: First, perform a convolution on the input feature map to double the number of channels. After the output data is split, the number of channels becomes half of the original h×w×d, as follows: the first branch, the input is h×w×ch_out data, after split, the output is two branches of h×w×d; The feature map of one branch is not processed, and the feature map of the other branch is passed to n BottleNeck modules for feature fusion operation; Concatenate the output feature maps of n Bottleneck modules with the original feature map; Finally, a convolution is performed on the concatenated feature map to output a feature map with the target number of channels.
5. The method for analyzing wafer production process defect classification based on deep learning according to claim 4, characterized in that: Perform a convolution on the input feature map to double the number of channels. Specifically, the h×w×d data passes through a 1*1 convolution kernel with a stride of 1 and padding of 0. After the convolution module with an output channel number of ch_out=2×d, the output is h×w×ch_out.
6. The method for analyzing wafer production process defect classification based on deep learning according to claim 4, characterized in that: The feature map of one branch is not processed, and the other branch is passed to n BottleNeck modules for feature fusion operation. The specific operation of a BottleNeck module is as follows: the input h×w×d data passes through two 3×3 convolution kernels, with a stride of 1 and a padding of 1, and the two branches output h×w×d data.
7. The method for analyzing wafer production process defect classification based on deep learning according to claim 4, characterized in that: The output feature maps of n Bottleneck modules are concatenated with the original feature map, as follows: n branches with outputs of h×w×d are concatenated to obtain an output of h×w×0.5(n+2)d.
8. The method for analyzing wafer production process defect classification based on deep learning according to claim 4, characterized in that: Finally, a convolution is performed on the concatenated feature map to output the feature map of the target number of channels, as follows: input h×w×0.5(n+2)d data, pass through a convolution module with a 1×1 convolution kernel, a stride of 1, and padding of 0, and obtain h×w×d output data.
9. The method for analyzing wafer production process defect classification based on deep learning according to claim 1, characterized in that: The formula for calculating the loss value of cross entropy with L2 regularization is: Among them, nc is the number of categories, yi is the true value, pi is the predicted value, and α is the weight coefficient of the regularization term.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.