Wafer distribution diagram mixed defect detection model compression system and method

Abstract

The invention relates to a wafer distribution diagram mixed defect detection model compression system and method. The system comprises a data preprocessing module which constructs a grading defect labeling system through multi-label coding; the teacher model adopts a progressive lightweight depth separable convolutional network, and a linear bottleneck structure and multi-level feature extraction are integrated; the student model is based on a binary neural network architecture and comprises a parallel branch structure, a grouping dilated convolution unit and a feature fusion unit; the knowledge distillation module realizes knowledge migration through a multi-stage training process, including layered loss function optimization, dynamic weight balance adjustment and end-to-end fine adjustment, and the method comprises the steps of multi-scale normalized data preprocessing, lightweight teacher model training, binarization student model construction and staged knowledge distillation. And finally, outputting a high-compression-ratio defect detection model adaptive to the edge computing equipment. Experiments show that the model parameter quantity is reduced by more than 70% and the reasoning speed is increased by more than 2.5 times on the premise of keeping more than 85% of classification accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection and edge intelligent computing, in particular to a wafer map mixed defect detection model compression system and method. BACKGROUND

[0002] Wafer map defect detection is a key link of semiconductor manufacturing process monitoring and yield management, which provides data support for process anomaly positioning and quality tracing by analyzing the electrical parameters or defect distribution characteristics of each die on the wafer. With the evolution of process technology to nanoscale, wafer defects show mixed characteristics of multiple types coexisting and spatial distribution disorder. Researchers have used deep neural networks to enhance wafer MAP features, which can effectively identify wafer defect patterns caused by specific process steps.

[0003] However, the existing scheme has two significant defects: first, the existing model is not adapted to the edge computing scenario, and its computational load and memory occupation far exceed the hardware limits of embedded devices, making it difficult to meet the real-time requirements of online detection on the production line. Second, the existing model compression strategy mainly targets single-stage optimization, resulting in a sharp drop in classification model accuracy, especially for mixed defect identification.

[0004] Therefore, there is an urgent need for a lightweight defect detection method for edge computing that can not only ensure compatibility with embedded systems with ultra-low parameter quantity, but also maintain high-precision classification performance for mixed defects, ultimately meeting the low-power, low-latency, and high-generalization requirements of semiconductor production lines for defect detection tasks. SUMMARY

[0005] This section aims to outline some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the specification to avoid obscuring the purpose of this section, abstract and title, and such simplifications or omissions cannot be used to limit the scope of the present application.

[0006] Therefore, the purpose of the present application is to provide a wafer map mixed defect detection model compression system and method, which uses lightweight neural network architecture design and cross-scale knowledge distillation optimization to build a low-parameter two-stage compression model, achieving high-precision real-time classification of mixed type wafer defects and meeting the low-power, low-latency online detection requirements of semiconductor production lines for complex defect patterns.

[0007] To solve the above technical problems, according to one aspect of the present application, the present application provides the following technical scheme: A wafer map mixed defect detection model compression system, comprising: A data preprocessing module is configured to obtain a wafer defect atlas dataset and construct a hierarchical labeling system containing multiple defect patterns through multi-label encoding. The teacher model module adopts a progressive lightweight depth separable convolution network architecture, contains multiple feature extraction layers, each layer integrates a linear bottleneck structure and a depth separable convolution, the input preprocessing size is adapted to an edge computing device, and a global pooling is followed by a full connection layer. The student model module adopts a multi-scale feature extraction architecture based on a binary neural network, contains a parallel branch structure, a grouped convolution module and a feature fusion unit, and finally outputs a classification result through adaptive pooling. The knowledge distillation module is configured to perform a multi-stage training process, including independently training a teacher model, optimizing a student model using a hierarchical loss function, dynamically adjusting loss weight balance, and performing end-to-end fine-tuning.

[0008] As a preferred scheme of the wafer distribution map mixed defect detection model compression system, the depth separable convolution of the teacher model decomposes the standard convolution into a channel-by-channel convolution and a point-by-point convolution.

[0009] As a preferred scheme of the wafer distribution map mixed defect detection model compression system, the grouped convolution module of the student model expands the receptive field to more than 5*5 through a hollow convolution while keeping the channel dimension unchanged, and introduces a cross-group feature interaction mechanism.

[0010] As a preferred scheme of the wafer distribution map mixed defect detection model compression system, the hierarchical loss function of the knowledge distillation module contains a loss for low-level feature matching and a loss for high-level output distribution.

[0011] As a preferred scheme of the wafer distribution map mixed defect detection model compression system, the binary neural network architecture of the student model contains a binary convolution layer, and through the cooperative design of binary weight and dynamic scaling factor, a layer-by-layer channel expansion stacking structure is adopted under the condition of preserving the original weight gradient back propagation.

[0012] A wafer distribution map mixed defect detection model compression method, the steps are as follows: The wafer defect atlas dataset is preprocessed, and a hierarchical labeling system is constructed through multi-label encoding. A progressive lightweight depth separable convolution network is constructed and trained as a teacher model, and the teacher model contains multiple feature extraction layers, each layer integrates a linear bottleneck structure and a depth separable convolution. The student model based on the binary neural network comprises a parallel branch structure, a grouped convolution module and a feature fusion unit; The knowledge distillation is performed through a multi-stage training process, including independent training of a teacher model, optimization of a student model by using a hierarchical loss function, dynamic adjustment of loss weight balance and end-to-end fine tuning.

[0013] As a preferred scheme of the wafer distribution map mixed defect detection model compression method, a nonlinear learning rate scheduling strategy is used in the training process of the teacher model.

[0014] As a preferred scheme of the wafer distribution map mixed defect detection model compression method, in the multi-stage training process of the knowledge distillation, when the verification set accuracy fluctuation is less than 0.5% for 3 consecutive rounds, the KL divergence weight is adjusted by increasing by 0.1 step.

[0015] Compared with the prior art, the present application has the beneficial effects that: 1. Lightweight depth separable convolution network architecture: a lightweight residual module is constructed, a 1x1 upsizing convolution and a 3x3 depth separable convolution are cascaded, high-efficiency feature extraction is realized under the condition of single-channel input by gradually increasing the number of channels and combining residual connection and adaptive average pooling, and the model parameter quantity is controlled within 45K and is suitable for mobile terminal deployment.

[0016] 2. Efficient lightweight network architecture based on binary weight: a binary network comprising a binary convolution layer is constructed, through the cooperative design of binary weight and dynamic scaling factor, under the condition of preserving the original weight gradient back propagation, a stack structure of layer-by-layer channel expansion is adopted to realize model parameter quantity compression and calculation efficiency optimization.

[0017] 3. Lightweight model compression method based on knowledge distillation: through the construction of a teacher-student double model cooperative training framework, the learning process of the binary network (student model) is guided by the lightweight depth separable convolution network (teacher model). Under the premise of preserving the characteristics of binary weight, the low-precision small model reaches the classification accuracy comparable to the teacher model. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the present application will be described in detail below in combination with the drawings and detailed embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor. Among them: Figure 1 The flowchart of the wafer distribution map mixed defect detection model compression method of the present application; Figure 2 A teacher model light weight network design flow chart is provided for the present application. Figure 3 A student model binary network design flow chart is provided for the present application. DETAILED DESCRIPTION

[0019] In order to make the above objectives, characteristics and advantages of the present application more apparent and comprehensible, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0020] Secondly, the present application is described in detail in combination with the schematic diagram. In the detailed description of the embodiments of the present application, the cross-sectional view of the device structure will be partially enlarged without the general proportion for the convenience of description, and the schematic diagram is only an example, which should not limit the scope of protection of the present application herein. In addition, the three-dimensional spatial dimensions of length, width and depth should be included in the actual manufacture.

[0021] In order to make the objectives, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0022] The present application provides a wafer defect detection method based on light weight neural network design and combination of binary neural network and knowledge distillation model, aiming to solve the problems of large parameter quantity, slow inference speed and high deployment cost of existing detection models. Through the synergistic effect of light weight neural network architecture optimization, binary convolution layer design and knowledge distillation strategy, the parameter compression and precision maintenance of the defect classification model are realized, which meets the real-time demand of industrial online detection. The above wafer defect detection method based on light weight neural network design and combination of binary neural network and knowledge distillation model will be described in detail below in combination with Figures 1-3 and embodiment 1-2. Embodiment 1

[0023] As shown in Figure 1 A wafer distribution map mixed defect detection model compression method, comprising the following steps: Wafer defect map data preparation The public wafer defect map dataset MixedWM38 from the actual production line is adopted, which contains multiple typical mixed defect types. The original multi-label encoding is converted into 38 single-label index by adopting one-hot encoding conversion Light weight design of teacher model As shown in Figure 2As shown in the figure, a teacher model of a lightweight depth-wise separable convolutional network architecture is constructed, which adopts a three-level lightweight module, including 32, 64, and 128-channel feature extraction layers connected in sequence. Each module uses the ReLU6 activation function, and the compression ratio of the linear bottleneck structure is 1:1.5; the single-channel wafer atlas is preprocessed to a size of 52×52; the feature extraction layer: contains 3 inverted residual modules, each module integrates a linear bottleneck structure and a depth-wise separable convolution, with an expansion ratio of 1.5; the classifier uses global average pooling followed by a single-layer full connection, in which the depth-wise separable convolution is decomposed into channel-by-channel convolution and point-by-point convolution by decomposing the standard convolution, reducing the computational complexity to 1 / 5 of the traditional convolution.

[0024] Binarized network design of student model like Figure 3 As shown in the figure, a binary network structure is constructed, which includes two levels of binary convolution modules. Each module consists of a binary convolution layer, a batch normalization layer and a ReLU activation function, and finally outputs the classification result through adaptive pooling and a fully connected layer.

[0025] Knowledge distillation training process Knowledge transfer is achieved through a three-stage training process. First, the initial training of the teacher model is completed independently on the wafer defect dataset. Then, the teacher model parameters are frozen and the student model parameters are optimized synchronously through distillation loss, using a weighted combination of KL divergence loss and cross entropy loss, where the KL divergence weight is The initial value is set to 0.3. Finally, the loss balance is dynamically adjusted according to the accuracy of the validation set. When the accuracy of the validation set fluctuates less than 0.5% for three consecutive rounds, The value was adjusted in increments of 0.1. Finally, on the MixedWM38 dataset, compared to the traditional ResNet-18, the model parameters were reduced by 78% while maintaining 88% classification accuracy, and the inference speed was increased by 3.2 times.

[0026] Example 2 like Figure 1 As shown, a wafer distribution map hybrid defect detection model compression method includes the following steps: (1) Wafer defect map data preparation We used the publicly available MixedWM38 dataset of wafer defect images from actual production lines. We used one-hot encoding and normalized the original grayscale images to 64×64, constructing a hierarchical annotation system encompassing 38 defect pattern categories.

[0027] (2) Lightweight design of teacher model like Figure 2As shown in the figure, the teacher model of the lightweight deep separable convolution network architecture is constructed, and a four-stage progressive lightweight module is adopted: the input layer performs 64x64 size preprocessing on the single-channel wafer map, and maps to 16 channels through a 3x3 convolution kernel; the feature extraction layer includes three-stage channel expansion modules, the first stage is composed of two reverse residual modules, a linear bottleneck structure with an expansion ratio of 6 (compression ratio 1:6) is adopted to realize 16→32 channel conversion, the second stage integrates three reverse residual modules with an expansion ratio of 4 and a compression ratio of 1:4 to complete 32→64 channel feature extraction, and the third stage expands to 128 channels through four reverse residual modules with an expansion ratio of 3 and a compression ratio of 1:3, and all modules adopt a ReLU6 activation function.

[0028] (3) Binary network design of student model As shown in the figure, the binary network structure based on the XNOR-Net architecture is constructed, and a three-stage feature extraction module is adopted: the first stage binary module integrates three parallel binary convolution branches, each branch is followed by a batch normalization layer and a PReLU activation function, and the channel number is expanded to 16; the second stage adopts a group binary convolution, introduces cross-group interaction under the condition of maintaining 16 channel dimensions, and improves the receptive field through a hollow convolution; the third stage feature fusion module adopts a binary deep separable convolution, and outputs a 32-channel feature map; finally, the adaptive pooling and binary fully connected layer are used to realize defect classification. Figure 3

[0029] (4) Knowledge distillation training process The knowledge transfer is realized through a four-stage progressive training process. First, the initial training of the teacher model is independently completed on the MixedWM38 dataset, and the cosine annealing learning rate scheduling is adopted; then, the parameters of the teacher model are frozen, and the student model is optimized through a hierarchical distillation strategy, the low-level feature matching adopts the MSE loss, the high-level output adopts the temperature weighted KL divergence loss, and the KL divergence weight The initial value is 0.4, and then the loss balance is dynamically adjusted according to the validation set accuracy; finally, the attention module parameters of the teacher model are unfrozen, and the end-to-end fine-tuning is performed at 1 / 10 of the initial learning rate.

[0030] The application also provides a wafer distribution map mixed defect detection model compression system to realize the steps of the above-mentioned wafer distribution map mixed defect detection model compression method, which comprises a data preprocessing module, a teacher model module, a student model module and a knowledge distillation module.

[0031] The data preprocessing module is used to obtain wafer defect map data set, and a hierarchical annotation system containing multiple defect modes is constructed through multi-label coding; ​The teacher model module adopts a progressive lightweight deep separable convolution network architecture, contains multiple feature extraction layers, each layer integrates a linear bottleneck structure and a deep separable convolution, the input preprocessing size is adapted to the edge computing device, and the classifier adopts a global pooling followed by a fully connected layer; The student model module adopts a multi-scale feature extraction architecture based on a binary neural network, contains a parallel branch structure, a grouped convolution module and a feature fusion unit, and finally outputs a classification result through adaptive pooling; The knowledge distillation module is used to perform a multi-stage training process, including independent training of the teacher model, optimization of the student model using a hierarchical loss function, dynamic adjustment of the loss weight balance, and end-to-end fine-tuning.

[0032] Among them, the deep separable convolution of the teacher model decomposes the standard convolution into channel-by-channel convolution and point-by-point convolution, reducing the calculation to 1 / 3-1 / 10 of the traditional convolution, the grouped convolution module of the student model expands the receptive field to more than 5*5 while keeping the channel dimension unchanged, and introduces a cross-group feature interaction mechanism, the hierarchical loss function of the knowledge distillation module contains a loss for low-level feature matching and a loss for high-level output distribution, the binary neural network architecture of the student model contains a binary convolution layer, and through the cooperative design of binary weight and dynamic scaling factor, under the condition of preserving the original weight gradient back propagation, a stacked structure with layer-by-layer channel expansion is adopted.

[0033] Although the present application has been described above with reference to the implementation, various improvements can be made and equivalent replacements can be made to the components without departing from the scope of the present application. In particular, as long as there is no structural conflict, each feature in the disclosed embodiments of the present application can be combined with each other in any way, and the combinations are not exhaustively described in the specification only for the purpose of omitting the length and saving resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A wafer distribution map hybrid defect detection model compression system, characterized in that: include: The data preprocessing module is used to obtain the wafer defect map dataset and build a hierarchical annotation system containing multiple types of defect patterns through multi-label encoding; The teacher model module uses a progressively lightweight depthwise separable convolutional network architecture, consisting of multiple feature extraction layers. Each layer integrates a linear bottleneck structure and depthwise separable convolutions. The input is pre-processed to adapt the size to the edge computing device, and the classifier uses global pooling followed by a fully connected layer. The student model module adopts a multi-scale feature extraction architecture based on a binary neural network, including a parallel branch structure, a grouped convolution module, and a feature fusion unit, and finally outputs the classification results through adaptive pooling; A knowledge distillation module that performs a multi-stage training process, including independently training a teacher model, optimizing a student model using a hierarchical loss function, dynamically adjusting the loss weight balance, and performing end-to-end fine-tuning.

2. The wafer distribution map hybrid defect detection model compression system according to claim 1, characterized in that: The depthwise separable convolution of the teacher model decomposes the standard convolution into channel-wise convolution and point-wise convolution.

3. The wafer distribution map hybrid defect detection model compression system according to claim 1, characterized in that: The grouped convolution module of the student model expands the receptive field to more than 5×5 through hollow convolution while keeping the channel dimension unchanged, and introduces a cross-group feature interaction mechanism.

4. The wafer distribution map hybrid defect detection model compression system according to claim 1, characterized in that: The hierarchical loss function of the knowledge distillation module includes a loss for low-level feature matching and a loss for high-level output distribution.

5. The wafer distribution map hybrid defect detection model compression system according to claim 1, characterized in that: The student model's binary neural network architecture includes a binary convolution layer, and through the collaborative design of binary weights and dynamic scaling factors, a stacked structure with layer-by-layer channel expansion is adopted while retaining the original weight gradient back propagation.

6. A wafer distribution map hybrid defect detection model compression method, using the system according to any one of claims 1 to 5, characterized in that: Here are the steps: Preprocess the wafer defect map dataset and build a hierarchical annotation system through multi-label encoding; Build and train a progressively lightweight depthwise separable convolutional network as a teacher model. The teacher model consists of multiple levels of feature extraction layers, each integrating a linear bottleneck structure and depthwise separable convolution. Constructing a student model based on a binary neural network, wherein the student model comprises a parallel branch structure, a group convolution module and a feature fusion unit; Knowledge distillation is performed through a multi-stage training process, including independently training a teacher model, optimizing the student model with a hierarchical loss function, dynamically adjusting the loss weight balance, and performing end-to-end fine-tuning.

7. The wafer distribution map hybrid defect detection model compression method according to claim 1, characterized in that: A nonlinear learning rate scheduling strategy is adopted during the training process of the teacher model.

8. The wafer distribution map hybrid defect detection model compression method according to claim 1, characterized in that: In the multi-stage training process of the knowledge distillation, when the accuracy of the validation set fluctuates by less than 0.5% for three consecutive rounds, the KL divergence weight is adjusted in increments of 0.1.

Images