KIA Net network model and its image, high-precision wafer defect detection and segmentation method
Through the KIA Net network model, combined with discrete wavelet transformation and hybrid attention mechanism, the problems of insufficient detection accuracy and poor robustness in the existing wafer defect detection technology are solved, and efficient and accurate defect detection and segmentation are achieved, and the complexity is adapted to The change of wafer defect image optimizes the use of computing resources.
Patent Information
- Application Number
- CN202411245121.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-06
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-09-06
AI Technical Summary
The existing wafer defect detection technology has problems such as insufficient detection accuracy, poor robustness, low efficiency, and inability to effectively segment complex and mixed-type defects. The existing methods are sensitive to noise and have large calculations, making it difficult to meet the needs of high-speed production.
Using the KIA Net network model, combining discrete wavelet transformation, spatial pyramid pooling module, KAN network and hybrid attention mechanism, a multi-task learning framework is built for image detection classification and segmentation, improving detection accuracy and robustness, and optimizing computing efficiency , and enhance the interpretability of the model.
Significantly improve the accuracy, robustness and adaptability of image detection, improve detection efficiency, realize high-precision defect detection and segmentation, overcome the shortcomings of traditional methods in large-scale applications, and adapt to complex and variable and often noise-containing The image recognition of wafer defects optimizes the use of computing resources.
Smart Images

Figure CN118762013B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image detection technology, in particular to the field of high-precision image detection technology. Background Art
[0002] With the rapid development of electronic technology, the global semiconductor industry continues to grow. As a key component in the manufacture of integrated circuits, the quality of wafers is directly related to the performance and stability of products. During the wafer production process, various defects often occur due to a variety of complex factors, and high-precision detection technology is urgently needed to ensure product quality. At present, wafer defect detection is shifting from traditional manual to machine learning and deep learning technology, aiming to improve the accuracy and efficiency of detection. Traditional manual wafer defect detection methods are time-consuming, labor-intensive and inefficient, while wafer defect detection based on machine vision and deep learning technology is a method that uses computer vision technology and deep learning algorithms to automatically detect and identify wafer defects. Through image acquisition, image processing and pattern recognition technologies, it can realize automatic detection and analysis of wafer defects, thereby improving detection efficiency and accuracy.
[0003] Although existing image processing and recognition algorithms have made some achievements in defect classification and segmentation, they are still insufficient in accuracy when dealing with complex and mixed types of defects, and research on high-precision defect detection models under the multi-task learning framework is still weak. For example, for the detection of wafer products, there is a wafer defect detection method and device with Chinese patent application number 202410501637.2, which proposes a wafer defect detection method based on polarized light imaging and multi-stage filtering, and uses deep learning and complex image processing technology to significantly improve weak defect detection capabilities and improve detection accuracy. However, this wafer defect detection method is too complex, has high hardware requirements, and relies on high-quality training data. It also has limited real-time performance and is difficult to meet high-speed production requirements. At the same time, the stability of the imaging system and parameter control also need to be highly precise, which increases the difficulty of application. Furthermore, Chinese Patent Application No. 202111097347.9 proposes a wafer defect detection method, which proposes an automated wafer defect detection method that integrates image acquisition and multi-step image processing technology. It can efficiently and accurately identify wafer surface defects and background anomalies, significantly improving detection efficiency and accuracy. However, the proposed detection method is limited to specific imaging conditions, the threshold setting is complex and requires fine adjustment, the amount of calculation is large and sensitive to noise, and the stability of background processing is affected by multiple factors, affecting versatility and ease of operation. In addition, Chinese Patent Application No. 202211545534.3 proposes an unsupervised wafer defect detection method based on deep learning, which proposes an unsupervised wafer defect detection method based on deep learning. By constructing the Gaussian distribution of normal samples and calculating the Mahalanobis distance between the image to be tested and the normal sample, high-precision defect detection without annotation is achieved, reducing the annotation cost and improving the detection efficiency and stability. However, this method has high requirements on the accuracy of normal sample construction and feature extraction, and the amount of calculation is large under high-resolution images, which may affect the detection speed. In addition, the adaptability to noise and complex backgrounds still needs further verification and optimization. In addition, Chinese patent application number 202410316719.X is a wafer defect detection method and equipment based on a neural network model. It proposes a wafer defect detection method based on a neural network, which automatically classifies known and unknown defects, manually confirms new categories and then feeds back to retrain the model to improve detection accuracy. However, although it is innovative in wafer defect detection, model retraining requires manual labeling, unknown category processing relies on manual labor, new category recognition efficiency is low, and model implementation details are vague, lacking targeted optimization. These factors limit its full degree of automation and detection effect.
[0004] In summary, it can be seen that the existing technology has shortcomings such as single detection disease type, poor robustness of detection algorithm, low efficiency, and inability to segment wafer defective areas. Summary of the invention
[0005] The first object of the present invention is to provide a KIA Net network model, which provides a new image detection network model. The second object of the present invention is to provide an image detection defect method of the above-mentioned KIA Net network model. The third object of the present invention is to provide a high-precision wafer defect detection and segmentation method of the KIA Net network model. The method of the present invention can significantly improve the accuracy, robustness and adaptability of image detection, while optimizing computational efficiency and enhancing the interpretability of the model.
[0006] To achieve the above-mentioned purpose, the technical solution of the present invention is: a KIA Net network model, including a classification network branch for image detection classification to obtain a classification type, the classification network branch includes a network input, a discrete wavelet transform module, a residual density module combining residual network learning and dense network connection, a fusion feature attention module combining a KAN network and a hybrid attention mechanism, a spatial pyramid pooling module and a classification type output, the network inputs an image to be detected and is processed by a discrete wavelet transform module, a residual density module combining residual network learning and dense network connection, a fusion feature attention module combining a KAN network and a hybrid attention mechanism, and a spatial pyramid pooling module, and then outputs the classification result of the image to be detected in the classification type.
[0007] The discrete wavelet transform module is provided with multiple levels, and the discrete wavelet transform module of the first level transforms the image to be detected into multiple sub-band feature image data of different scale features, and processes them in multiple branches. The feature image data is respectively processed by the discrete wavelet transform module, residual density module and / or fusion feature attention module of other levels in the branches, and then processed by the spatial pyramid pooling module.
[0008] The KIA Net network model also includes a segmentation network branch, and the classification network branch includes a deep pyramid pooling convolutional neural network with an encoder feature extraction part and a decoder part, and a segmentation result output; the network inputs an image to be detected and processes it through a deep pyramid pooling convolutional neural network, and then outputs a segmentation result of the image to be detected in a classification type.
[0009] The feature extraction part of the encoder includes a plurality of convolution blocks including a pyramid pooling convolution layer and a separable pyramid pooling convolution layer, and the decoder part includes a deep pyramid pooling transposed convolution block, and the connection of the convolution blocks includes a gradient connection and / or a skip connection.
[0010] A KIA Net network model image defect detection method is constructed with a KIA Net network model and a defect image data set. The KIA Net network model includes a classification network branch. The defect image data set is a defect image containing multiple defect types of the detected object. The defect images are labeled with defect type labels by an image annotation tool, and randomly sampled in proportion to make a training set, a test set and a validation set. The defect image data set is used to detect the KIA Net network model. Net network model is used for task training, and the task training includes classification training; the defect classification processing method of the classification network branch is as follows: S1, the network inputs the image to be detected, S2, the image is subjected to feature extraction through the discrete wavelet transform module to capture the detail information on the image and obtain feature image data, S3, the feature of the feature image data is fused with the residual density module to enhance the model's ability to express features, and a fusion feature attention module combining the KAN network and the hybrid attention mechanism is introduced to further enhance the recognition ability of key features through the attention mechanism, S4, the features processed by S3 are pooled through spatial pyramid pooling to enhance the model's adaptability to changes in different scales of the detection image, S5, the features after pooling are compared with the defect type labels of the defect image data set to identify the defect type for classification, and the defect type results are output.
[0011] The classification network branch is a multi-level setting with multiple discrete wavelet transform modules that can analyze at different resolutions. In S2, the first-level discrete wavelet transform module captures the feature extraction of detail information at different scales of the image, transforms sub-band feature image data of multiple different scale features, and processes them in multiple branches. In the branch, the sub-band feature image data is processed by S2, S3 and S4 of discrete wavelet transform modules at other levels.
[0012] The KIA Net network model also includes a segmentation network branch. The defect images in the defect image data set are also annotated with defect segmentation labels through an image annotation tool, and the task training includes segmentation training; the segmentation processing method of the segmentation network branch is as follows: the S1 network inputs the image to be detected, S11, extracts features from the image through a deep pyramid pooling convolutional neural network to segment the graphic, and compares the image with the defect segmentation labels of the defect image data set to optimize the segmentation accuracy, and S12, outputs the defect type result.
[0013] In S11, in the deep pyramid pooling convolutional neural network, features of different scales are extracted through multiple convolution blocks including a pyramid pooling convolution layer with batch normalization and a dropout layer, a separable pyramid pooling convolution layer, and a deep pyramid pooling transposed convolution block, and the connections of the convolution blocks include gradient connections and / or skip connections.
[0014] After the image to be detected is detected by the KIA Net network model, the label of the defect type is added to the defect image dataset, the KIA Net network model is trained for the task, and comparative tests and ablation experiments are carried out to adjust and optimize the model parameters.
[0015] By adopting the above technical scheme, the beneficial effects of the present invention are as follows: the KIA Net network model and image defect detection method of the above technical scheme, in response to current challenges such as defect diversity and image noise, proposes an efficient detection and segmentation method based on deep learning, and improves the accuracy, efficiency, interpretability and robustness of image defect detection by integrating multiple advanced technologies such as discrete wavelet transform, spatial pyramid pooling module, and fusion feature attention module combining KAN network and hybrid attention mechanism, thereby promoting the development of the semiconductor manufacturing field and achieving high-precision and high-efficiency defect detection and segmentation. Specifically, the introduction of discrete wavelet transform enables the model to effectively extract multi-scale information and adapt to changes in image scale, while the attention mechanism helps the model focus on key features, further improving the detection performance in complex scenarios; in addition, the present invention is also committed to optimizing the model structure and reducing the computational burden so that it can run efficiently under conventional computing resources; at the same time, by analyzing the contribution of each component of the model, the interpretability of the model is improved to make the decision-making process more transparent; finally, by realizing a multi-task learning framework, the present invention not only improves the classification accuracy of defect detection, but also can use a convolutional neural network based on deep pyramid pooling to efficiently segment images, broadens the application scope of the model, and fully meets the needs of industrial-grade high-precision image defect detection.
[0016] The application of the present invention can solve the problems of high precision and complexity of defect detection in the semiconductor wafer manufacturing process. By combining technologies such as discrete wavelet transform, the model's recognition ability for complex, changeable and often noisy wafer defect images is enhanced, the robustness of image processing is enhanced, and the detection efficiency is improved while ensuring high accuracy. It overcomes the shortcomings of traditional methods in large-scale applications, significantly improves the accuracy, robustness and adaptability of wafer defect detection, and at the same time optimizes computing efficiency and enhances the interpretability of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 The invention discloses a schematic diagram of a model structure of a KIA Net network model.
[0018] Figure 2 It is a schematic diagram of the model structure of a fusion feature attention module in a KIA Net network model involved in the present invention. DETAILED DESCRIPTION
[0019] In order to further explain the technical solution of the present invention, the present invention is described in detail below through specific embodiments.
[0020] The KIA Net network model disclosed in this embodiment is an embodiment listed by taking the defect detection in the semiconductor wafer manufacturing process as an example. The embodiment can be adjusted and changed accordingly in other products and image detection applications. Figure 1 As shown in the figure, the KIA Net (KAN Integrated Attention Network) network model of this embodiment includes a classification network branch for image detection classification to obtain a classification type and a segmentation network branch for image defect segmentation. It can be seen from the figure that the two branches are two separate parts. If segmentation is not required, the branch can be omitted. Figure 1 The KIA Net network model framework structure of this embodiment is described.
[0021] The classification network branch includes network input, discrete wavelet transform module (DWT), residual density module (DRB) combining residual network learning and dense network connection, fused feature attention module (IFA) based on KAN network (Kolmogorov-Arnold networks), spatial pyramid pooling module (SPP) and classification type output. The network inputs the image to be detected and processes it through the discrete wavelet transform module, the residual density module combining residual network learning and dense network connection, the fused feature attention module combining KAN network and hybrid attention mechanism, and the spatial pyramid pooling module, and then outputs the classification result of the image to be detected in the classification type. Figure 1 As shown in , the specific discrete wavelet transform module is arranged with multiple levels, and the discrete wavelet transform module of the first level transforms the image to be detected into multiple sub-band feature image data of different scale features, and processes them in multiple branches. The feature image data is respectively processed by the discrete wavelet transform modules, residual density modules and / or fusion feature attention modules of other levels in the branches, and then processed by the spatial pyramid pooling module.
[0022] The classification network branch includes a deep pyramid pooling convolutional neural network with an encoder feature extraction part and a decoder part, and a segmentation result output; the network inputs the image to be detected and processes it through the deep pyramid pooling convolutional neural network, and outputs the segmentation result of the image to be detected in the classification type. Figure 1 As shown in the figure, the specific feature extraction part of the encoder includes multiple convolution blocks including pyramid pooling convolution layers and separable pyramid pooling convolution layers, the decoder part includes deep pyramid pooling transposed convolution blocks, and the connections of the convolution blocks include gradient connections and skip connections.
[0023] After constructing the above-mentioned KIA Net network model, it is also necessary to prepare a defect image dataset, which is a defect image dataset that collects defect images containing multiple defect types of the detected object, annotates the defect images with defect types and defect segmentation labels through image annotation tools, and randomly samples them in proportion (such as the experimental ratio of 7:2:1 in this embodiment) to make training sets, test sets and verification sets. The defect image dataset is used to perform task training on the KIA Net network model, and the task training includes classification training and segmentation training (if segmentation processing is not required, the corresponding part of the pre-preparation and method process is removed). The image defect detection method of the KIA Net network model of the above-mentioned framework structure is disclosed below.
[0024] The defect classification processing method process of the classification network branch is:
[0025] S1, the network inputs the image to be detected, where the input can perform preprocessing operations such as denoising and normalization on the image;
[0026] S2, the image is subjected to feature extraction through a discrete wavelet transform module to capture detailed information on the image and obtain feature image data;
[0027] S3, the features of the feature image data are fused with the residual density module to enhance the model's ability to express features, and a fusion feature attention module that combines the KAN network and the hybrid attention mechanism is introduced to further improve the recognition ability of key features through the attention mechanism;
[0028] S4: The features processed by S3 are pooled through spatial pyramid pooling to enhance the adaptability of the model to the detection of different scale changes of the image;
[0029] S5. Compare the pooled features with the defect type labels of the defect image dataset to identify the defect type for classification, and output the defect type results.
[0030] In this embodiment, Figure 1 As shown, the classification network branch is a multi-level setting with multiple discrete wavelet transform modules that can analyze at different resolutions. In S2, the first-level discrete wavelet transform module captures the feature extraction of detail information at different scales of the image, transforms sub-band feature image data of multiple different scale features, and processes them in multiple branches. In the branch, the sub-band feature image data is processed by S2, S3 and S4 of discrete wavelet transform modules at other levels.
[0031] The segmentation processing method process of the segmentation network branch is:
[0032] This embodiment performs multiple tasks simultaneously, where the segmentation network branch and the classification network branch are the same network input end, and the S1 network inputs the image to be detected;
[0033] S11, extracting features from the image through a deep pyramid pooling convolutional neural network to segment the graphics, and comparing with the defect segmentation labels of the defect image data set to optimize the segmentation accuracy; in this step, the deep pyramid pooling convolutional neural network uses a pyramid pooling convolutional layer with batch normalization and a dropout layer, a separable pyramid pooling convolutional layer, and a multi-convolutional block of a deep pyramid pooling transposed convolutional block to extract features of different scales, and the connection of the convolutional blocks includes a gradient connection and / or a skip connection;
[0034] S12. Output defect type results.
[0035] In order to enable the image detection defect method of the KIA Net network model to continuously learn, improve and optimize, in the above method, after the network model completes the task, the label of the defect type of the image to be detected can be added to the defect image dataset, so as to train the KIA Net network model for the task, conduct comparative tests and ablation experiments, so as to continuously learn, adjust, improve and optimize the model parameters.
[0036] The KIA Net network model constructed by the present invention is built on the basis of multiple advanced technologies and methods. The overall architecture of KIA Net includes a classification network branch and a segmentation network branch, which are optimized for classification and segmentation tasks respectively. The classification network branch is a key component of the KIA Net network model. Its design integrates multiple cutting-edge algorithms and technologies, aiming to achieve high-precision classification of product defects (such as wafer products in the following embodiments). The network combines discrete wavelet transform, residual density module, spatial pyramid pooling module, and fusion feature attention module combined with KAN network and hybrid attention mechanism, so as to achieve detection and classification of product defects and accurate segmentation of defective areas.
[0037] The following describes in detail a specific method for applying the above KIA Net network model to wafer defect detection and segmentation tasks. A high-precision wafer defect detection and segmentation method using a KIA Net network model, such as Figure 1 and Figure 2 As shown, the classification and segmentation processing method process is as follows:
[0038] S1, the network inputs the wafer image to be detected and performs preprocessing operations such as denoising and normalization on the wafer image, enters S2 in the classification network branch, and enters S11 in the segmentation network branch;
[0039] S2, the wafer image is extracted by the first-level two-dimensional discrete wavelet transform module to capture the feature information of the details at different scales, and the sub-band feature image data of four different scale features are transformed and processed in three branches, such as Figure 1 The four sub-band feature image data are named AA sub-band, AD sub-band, DA sub-band and discard DD sub-band, AA is low frequency, and the others are high frequency. Discard DD is the discarded sub-band feature image data. The other sub-bands are also extracted image features through the second-level and third-level discrete wavelet transform modules in different branches.
[0040] S3, the feature information of different scales extracted at different levels of S2 is input into the residual density module, and these features are fused with the residual density module to enhance the model's ability to express features. A fusion feature attention module combining the KAN network and the hybrid attention mechanism is introduced to further improve the recognition ability of key features through the attention mechanism;
[0041] S4: The features processed by S3 are pooled through the spatial pyramid pooling module, which effectively captures the local and global features of the image at different scales and enhances the adaptability of the model to detecting changes in different scales of the image;
[0042] S5, comparing the pooled features with the defect type labels of the defect image dataset to identify the defect type for classification, and outputting the defect type result;
[0043] S11, extract features from the image through a deep pyramid pooling convolutional neural network to segment the image, and compare it with the defect segmentation labels of the defect image dataset to optimize the segmentation accuracy, S12, output the defect type result. Figure 1 As shown, in S11, the deep pyramid pooling convolutional neural network performs feature extraction of different scales through a 3*3 pyramid pooling convolutional layer with batch normalization and a dropout layer, a 3*3 separable pyramid pooling convolutional layer with batch normalization, and a deep pyramid pooling transposed convolutional block, and the connection of the convolutional blocks includes gradient connections and / or skip connections.
[0044] The following describes in detail the modules of the KIA Net network model framework structure and the overall structure of the framework, as well as the working process effect of the task processing method of the technical solution of the present invention.
[0045] Discrete wavelet transform (two-dimensional discrete wavelet transform 2D-DWT in the embodiment), as an initial processing step, is used to extract multi-scale information from the detection image. It can capture local details and global features in the image, can capture richer features, and maintain the transferability and learnability between features, such as Figure 1 As shown in the figure, the discrete wavelet transform generates image data of 4 sub-bands. These rich multi-scale information provides a solid foundation for subsequent feature extraction and classification. DWT enhances the model's adaptability to image scale changes and ensures good detection results on wafer images of different scales.
[0046] Residual density module: This module combines the advantages of residual learning and dense connections. By constructing densely connected residual blocks, it not only promotes the full extraction of features, but also effectively alleviates the common gradient vanishing problem in deep networks. Residual connections enable feature maps to be directly transmitted across layers, while dense connections enhance the information flow between layers, improving the model's representation ability and learning efficiency.
[0047] The fusion feature attention module is one of the innovative points of the KIA Net network model proposed in this invention. It combines the KAN network and the hybrid attention mechanism. This module can adaptively adjust the importance of different features and focus on more critical features according to task requirements, thereby improving the classification accuracy and robustness of the model. The application of this hybrid attention mechanism enables the KIA Net network model to more accurately identify various defects in wafer images.
[0048] The integration of these advanced algorithms and technologies enables the classification branch of the KIA Net network model to demonstrate excellent performance in product (wafer) defect detection. Through experimental verification, the KIA Net network model has achieved an accuracy rate of up to 99.03% on a data set containing multiple defect types, and has shown strong robustness even on a data set containing noise interference. In addition, the KIA Net network model also uses a multi-task learning framework to achieve integrated processing of defect detection and segmentation, further enhancing its value in practical applications.
[0049] In the segmentation branch of the KIA Net network model, the network adopts an encoder-decoder structure, and gradually refines the image through deep pyramid pooling convolution blocks and transposed convolution blocks. Through the combination of deep pyramid pooling convolution blocks and jump connections, it demonstrates its excellent technical characteristics and significant advantages and effects. Specifically, it is achieved by pooling feature maps at multiple scales in the deep pyramid pooling convolution block. These scales usually include pooling windows of different sizes. Each pooling operation will reduce the feature map of the image to different resolutions to capture semantic information from global to local. Then, these features of different scales are integrated to form a multi-scale feature representation, which enables the network to recognize various image features from fine-grained to coarse-grained. This method effectively extracts multi-scale semantic information from the image. This multi-level processing mechanism enables the network to capture various features from fine-grained to coarse-grained, thereby enhancing the network's ability to recognize complex and diverse defect patterns and significantly improving the accuracy and robustness of segmentation.
[0050] In a deep network, the gradient propagation layer by layer may easily lead to the gradient vanishing problem, which will affect the training effect and performance of the network. Therefore, the present invention uses a skip connection in the segmentation branch of the KIA Net network model, such as Figure 1 In the paper, a skip connection is added between the deep pyramid convolution block and the deep pyramid transposed convolution block. The skip connection allows the information of the encoder part to be directly passed to the decoder part, thereby providing more contextual information and semantic information. The skip connection effectively maintains the continuity and integrity of the information by directly passing the low-level features to the high-level ones, while accelerating the convergence speed and improving the stability of training. This is very helpful for the deep pyramid transposed convolution block to more accurately restore the details of the original image, especially when restoring features such as boundaries and subtle defects. In addition, the skip connection can make the gradient propagate better in the network and alleviate the problem of gradient disappearance, thereby improving the convergence speed during model training and thus improving the performance of the model. It also promotes feature reuse, allowing the network to learn the essential characteristics of defects more deeply.
[0051] As attached Figure 1An embodiment of a high-precision wafer defect detection and segmentation method of the KIA Net network model disclosed above, in the classification network branch of the model framework, different image features are extracted through multiple levels of DWT. In the network input stage, the two-dimensional discrete wavelet transform is first used to extract image features in three branches. In the four sub-bands from the DWT, AA still retains a large amount of information from the input image. Based on this, multiple levels of DWT can be applied to further decompose the features. As the number of two-dimensional discrete wavelet layers increases, the scale of the wavelet function increases, thereby compressing its spectrum in the frequency domain, making the bandwidth of the corresponding filter narrower, allowing the DWT to focus on more detailed information, which is called multi-resolution analysis. One advantage of doing this is that it allows the DWT to extract features from multiple angles. In this way, at a preliminary level, the DWT reveals a summary of the features, and at a deeper level, the DWT decomposes it into finer details. Here, three levels of DWT are used: the three-level DWT shows the finest details, the two-level DWT focuses on less specific details, and the first-level DWT shows the general information in the image. As Figure 1 In the first branch, the detailed information of the AA subband is deeply mined through the three-level DWT-Layer, and then the three frequency subbands of AA, AD, and DA are input into the wavelet domain learner. The learner consists of a density residual block and a hybrid attention module based on the KAN network channel attention and residual spatial attention. On this basis, a spatial pyramid pooling operation is performed. This process adjusts the weights of different channels and spatial dimensions respectively, so that the network can pay more attention to the key features in the image, thereby improving the performance and accuracy of the model; then the second branch is processed by the second and first level DWT-Layer respectively, and then passed to the wavelet domain learner, and finally spatial pyramid pooling is performed; then the third branch is processed by three first level DWT-Layers respectively, and then input into the wavelet domain learner, and finally spatial pyramid pooling is performed. The spatial pyramid pooling module effectively captures the local and global features of the image at different scales through multi-scale feature extraction, enhances the network's ability to understand multi-scale information in the image, and the features after multiple feature fusions and attention mechanisms are input into the fully connected layer to output the corresponding defect classification results. In the KIA Net network test of this embodiment, eight basic types of defects can be identified, and 38 defect types can be inferred based on the combination conditions.
[0052] In the segmentation network branch of the model framework of this embodiment, the encoder extracts features using multiple convolution blocks to extract multi-scale local details. The PCB includes a 3×3 pyramid pooling convolution layer with batch normalization and exit layer and a 3×3 separable pyramid pooling convolution layer with batch normalization. Compared with the convolution layer, the separable convolution layer has two more steps, namely, depth convolution and point-by-point convolution, which reduces the number of parameters and the amount of calculation. That is, the use of separable convolution in the segmentation branch of the KIA Net network can reduce the demand for memory bandwidth and calculation, and also improve the representation efficiency. In addition, after the last convolution block, the network uses pyramid pooling to extract multi-scale features of the defective wafer image. This method can further extract the overall distribution and subtle details in the wafer defect image, which is of great significance for the accurate classification and segmentation of semiconductor wafer defects. Compared with the traditional global average pooling method (GAP), it can better extract multi-scale features in the wafer defect segmentation task, enhance the robustness of the model, and adapt to input images of different sizes. It has certain advantages in wafer defect segmentation, and the segmentation branch finally generates a binary segmentation mask. In addition, the decoder part contains a deep pyramid pooling transposed convolution block, which is used to generate an output mask by upsampling the feature representation. The TPCB contains a transposed convolution layer, which uses learnable parameters to upsample the input feature map to the desired output feature map size. It interprets the original input data by filling in detail information and restores image details during the upsampling process. It accepts the information of the deep pyramid pooling convolution block through skip connections to enrich the semantic information of the input image, which reduces the error rate of mask generation.
[0053] In summary, the present invention provides an innovative network architecture. Its specific implementation method, unique application of multi-scale feature extraction and attention mechanism, as well as the high-precision recognition, defect detection algorithm and image segmentation capabilities achieved thereby, not only achieve significant advantages in defect classification, but also show excellent performance in defect segmentation. It achieved a high accuracy rate of 99.03% in the experimental wafer defect classification task, which is far higher than other methods, especially when processing noisy data sets, its performance is still outstanding.
[0054] In addition, the KIA Net network model innovatively adopts a multi-task learning framework, in which wafer images are segmented through a deep pyramid pooling convolutional neural network. This design enables the KIA Net network model to not only accurately identify the type of wafer defects, but also accurately segment the specific location of the defects. Experimental results show that the intersection over union (IoU) index of KIA Net in the defect segmentation task has reached an astonishing 0.9999, almost achieving pixel-level accurate segmentation, which is unprecedented in the field of wafer defect detection.
[0055] In addition, the KIA Net network model not only demonstrates high accuracy and strong robustness in wafer defect classification, but also achieves high accuracy in defect segmentation, providing all-round, high-precision technical support for wafer defect detection. These advantages make KIA Net have huge application potential and broad market prospects in the semiconductor manufacturing industry, which can significantly improve the quality and production efficiency of semiconductor products and promote the sustainable development of the semiconductor industry.
[0056] The above embodiments and drawings do not limit the product form and style of the present invention. Any appropriate changes or modifications made by ordinary technicians in the relevant technical field should be deemed to be within the patent scope of the present invention.
[0057] First, in the classification branch, the wafer image is input into the network for preprocessing. In this process, the image features are mainly decomposed into multiple sub-bands through two-dimensional discrete wavelet transform (DWT). DWT divides the image information into low-frequency (AA sub-band) and high-frequency parts (AD, DA, DD sub-bands), and extracts image features through different levels of DWT layers in different branches. In the feature extraction stage, the multi-scale information extracted by the DWT layer is input into the density residual block (Dense-ResidualBlock). This module combines the advantages of dense connection blocks (Dense Block) and residual blocks (Residual Block), can capture richer features, and maintain the transitivity and learnability between features. In order to more effectively select and aggregate valuable information, the network introduces a hybrid attention module (IFA). This module combines the channel attention mechanism based on the KAN network and the residual space attention mechanism, and adjusts the weights of different channels and spatial dimensions respectively, so that the network can pay more attention to the key features in the image, thereby improving the performance and accuracy of the model. Next, the features processed by the attention mechanism are input into the spatial pyramid pooling module (SPP). This module effectively captures the local and global features of the image at different scales through multi-scale feature extraction, and enhances the network's ability to understand multi-scale information in the image. The features after multiple feature fusions and attention mechanisms are input into the fully connected layer to output the corresponding defect classification results. The KIANet network can identify eight basic types of defects and infer 38 defect types based on the combination.
[0058] In the segmentation branch, the network adopts an encoder-decoder structure, gradually refines the image through deep pyramid pooling convolution blocks and transposed convolution blocks, and retains more semantic information through jump connections to generate the final binary segmentation mask to complete the wafer defect segmentation task. Finally, the network outputs the defect classification results and the corresponding segmentation mask for further wafer defect analysis and processing.
Claims
1. A KIA Net network model, characterized in that: The invention comprises a classification network branch for image detection classification to obtain a classification type, wherein the classification network branch comprises a network input, a discrete wavelet transform module, a residual density module combining residual network learning and dense network connection, a fusion feature attention module combining a KAN network and a hybrid attention mechanism, a spatial pyramid pooling module, and a classification type output. The network inputs an image to be detected, and after being processed by the discrete wavelet transform module, the residual density module combining residual network learning and dense network connection, the fusion feature attention module combining a KAN network and a hybrid attention mechanism, and the spatial pyramid pooling module, the classification result of the image to be detected is output in the classification type. The discrete wavelet transform module is provided with multiple levels, wherein the discrete wavelet transform module of the first level transforms the image to be detected into multiple sub-band feature image data of different scale features, and processes the sub-band feature image data in multiple branches, wherein the feature image data is respectively processed by the discrete wavelet transform module, the residual density module and / or the fusion feature attention module of other levels in the branches, and then processed by the spatial pyramid pooling module; The KIA Net network model also includes a segmentation network branch, and the classification network branch includes a deep pyramid pooling convolutional neural network with an encoder feature extraction part and a decoder part, and a segmentation result output; the network inputs the image to be detected and processes it through the deep pyramid pooling convolutional neural network, and outputs the segmentation result of the image to be detected in the classification type; The feature extraction part of the encoder includes a plurality of convolution blocks including a pyramid pooling convolution layer and a separable pyramid pooling convolution layer, and the decoder part includes a deep pyramid pooling transposed convolution block, and the connection of the convolution blocks includes a gradient connection and / or a skip connection.
2. A KIA Net network model image defect detection method, characterized in that: A KIA Net network model and a defect image data set are constructed. The KIA Net network model includes a classification network branch. The defect image data set is a collection of defect images containing multiple defect types of detected objects. The defect images are labeled with defect type labels using an image annotation tool, and randomly sampled in proportion to make a training set, a test set, and a validation set. The defect image data set is used to perform task training on the KIA Net network model, and the task training includes classification training. The defect classification processing method of the classification network branch is as follows: S1, the network inputs the image to be detected, S2, the image is feature extracted through a discrete wavelet transform module to capture the detail information on the image and obtain feature image data, S3, the feature of the feature image data is fused with the residual density module to enhance the model's ability to express features, and a fusion feature attention module combining the KAN network and the hybrid attention mechanism is introduced to further improve the recognition ability of key features through the attention mechanism, S4, the features processed by S3 are pooled through spatial pyramid pooling to enhance the model's adaptability to changes in different scales of the detection image, S5, the features after pooling are compared with the defect type labels of the defect image data set to identify the defect type for classification, and the defect type results are output; The classification network branch is a multi-level setting with multiple discrete wavelet transform modules that can analyze at different resolutions. In S2, the discrete wavelet transform module of the first level extracts the features of the detail information captured at different scales of the image, transforms the sub-band feature image data of multiple different scale features, and processes them in multiple branches. In the branches, the sub-band feature image data is processed by S2, S3 and S4 of the discrete wavelet transform modules of other levels. The KIA Net network model also includes a segmentation network branch. The defect images in the defect image data set are also annotated with defect segmentation labels by an image annotation tool. The task training includes segmentation training. The segmentation processing method of the segmentation network branch is as follows: S1, the network inputs the image to be detected, S11, extracts features of the image through a deep pyramid pooling convolutional neural network to segment the image, and compares the defect segmentation labels of the defect image data set to optimize the segmentation accuracy, S12, outputs the defect type result; In S11, in the deep pyramid pooling convolutional neural network, features of different scales are extracted through multiple convolution blocks including a pyramid pooling convolution layer with batch normalization and a dropout layer, a separable pyramid pooling convolution layer, and a deep pyramid pooling transposed convolution block, and the connections of the convolution blocks include gradient connections and / or skip connections.
3. The image defect detection method of a KIA Net network model as claimed in claim 2, characterized in that: After the image to be detected is detected by the KIA Net network model, the label of the defect type is added to the defect image dataset, the KIANet network model is trained for the task, and comparative tests and ablation experiments are carried out to adjust and optimize the model parameters.
4. A high-precision wafer defect detection and segmentation method based on a KIA Net network model, characterized in that: A KIANet network model and a defect image data set are constructed. The KIA Net network model includes a classification network branch and a segmentation network branch. The defect image data set is obtained by collecting defect images containing multiple defect types of the detected objects, annotating the defect images with defect types and defect segmentation labels through image annotation tools, and randomly sampling them in proportion to make training sets, test sets and validation sets. The defect image data set is used to perform task training on the KIA Net network model, and the task training includes classification training and segmentation training; S1, the network inputs the wafer image to be detected and preprocesses the wafer image, enters S2 in the classification network branch, and enters S11 in the segmentation network branch; S2. The wafer image is extracted from the details of the image at different scales through the first-level two-dimensional discrete wavelet transform module, and the sub-band feature image data of four different scale features are transformed and processed in three branches. The sub-band feature image data are named AA sub-band, AD sub-band, DA sub-band and discard DD sub-band, AA is low frequency, and the others are high frequency, where discard DD is the discarded sub-band feature image data, and the other sub-bands are also extracted image features through the second-level and third-level discrete wavelet transform modules in different branches; S3, the feature information of different scales extracted at different levels of S2 is input into the residual density module, and these features are fused with the residual density module to enhance the model's ability to express features. A fusion feature attention module combining the KAN network and the hybrid attention mechanism is introduced to further improve the recognition ability of key features through the attention mechanism; S4: The features processed by S3 are pooled through the spatial pyramid pooling module, which effectively captures the local and global features of the image at different scales and enhances the adaptability of the model to detecting changes in different scales of the image; S5, comparing the pooled features with the defect type labels of the defect image dataset to identify the defect type for classification, and outputting the defect type result; S11, extracting features from the image through a deep pyramid pooling convolutional neural network to segment the image, and comparing it with the defect segmentation labels of the defect image dataset to optimize the segmentation accuracy, S12, outputting the defect type result; In S11, the deep pyramid pooling convolutional neural network performs feature extraction at different scales through multiple convolution blocks including a 3*3 pyramid pooling convolution layer with batch normalization and a dropout layer, a 3*3 separable pyramid pooling convolution layer with batch normalization, and a deep pyramid pooling transposed convolution block, and the connections of the convolution blocks include gradient connections and / or skip connections.
