Wafer defect real-time detection method and system based on multi-scale feature fusion

Abstract

The invention discloses a wafer defect real-time detection method and system based on multi-scale feature fusion, and belongs to the field of wafer defect detection, and the method comprises the steps: carrying out the preliminary coding of each scale feature through employing a convolutional neural network for a generated feature set, and obtaining a feature vector group containing the local and global information of a defect; if the saliency of the high-frequency component in the feature vector group exceeds a preset threshold value, performing weighted enhancement on the high-frequency feature through an attention mechanism to obtain an enhanced feature set; according to the enhanced feature set, constructing an index structure based on Hash coding, and generating a defect feature index table capable of being quickly retrieved by mapping feature vectors to a low-dimensional space; updating the index table according to the matching result, combining the newly detected defect features with the time sequence prediction result, and generating an expanded defect feature index table; and for the expanded index table, an incremental clustering algorithm is adopted to classify defect features, and defect type distribution updated in real time is obtained.

Description

Technical Field

[0001] The present invention belongs to the field of wafer defect detection, and in particular relates to a real-time wafer defect detection method and system based on multi-scale feature fusion. Background Art

[0002] Wafer manufacturing is the core link of the semiconductor industry. Its quality control directly determines the performance and reliability of chips, and defect detection, as a key technology to ensure wafer quality, is of irreplaceable importance. As chip manufacturing processes continue to shrink, the types and complexity of wafer surface defects have increased significantly, and the demand for real-time detection technology has become more urgent. Traditional detection methods mostly rely on single-scale image analysis or simple feature extraction, which makes it difficult to cope with the diversity and complexity of micron or even nanometer-level defects. These methods often face problems of low computational efficiency or insufficient detection accuracy when processing high-resolution images, especially in dynamic production environments, making it difficult to achieve fast and accurate defect identification. In detection methods based on multi-scale feature fusion, the core challenge lies in how to effectively integrate feature information at different scales to achieve rapid retrieval and accurate matching of defects.

[0003] Single-scale features are difficult to fully characterize the morphology and texture of defects, while the fusion process of multi-scale features easily introduces increased computational complexity, resulting in limited real-time performance. In addition, the dynamic evolution of defects during the production process increases the difficulty of detection, and traditional static feature analysis cannot effectively track the changes in defects over time. These technical difficulties make it difficult for the system to simultaneously meet the requirements of efficient retrieval, accurate matching, and dynamic tracking. Therefore, how to design an efficient multi-scale feature index structure and combine it with the temporal evolution model of defects to achieve real-time retrieval and dynamic tracking of wafer defects has become a key issue in this research. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a real-time wafer defect detection method based on multi-scale feature fusion, comprising:

[0005] Acquire multi-scale features from high-resolution wafer images, extract texture and morphological information at different resolutions using a pyramid decomposition algorithm, and generate a feature set containing both low-frequency and high-frequency components.

[0006] For the generated feature set, a convolutional neural network is used to preliminarily encode the features of each scale to obtain a feature vector group containing local and global information of the defect;

[0007] If the significance of the high-frequency components in the feature vector group exceeds a preset threshold, the high-frequency features are weighted and enhanced through the attention mechanism to obtain an enhanced feature set;

[0008] Construct an index structure based on hash coding according to the enhanced feature set, and generate a defect feature index table that can be quickly retrieved by mapping the feature vector to a low-dimensional space;

[0009] Obtain wafer images with continuous time series from the production line, extract the enhanced feature set corresponding to each image, calculate the Euclidean distance between the feature sets, and obtain the change trend of defect features over time;

[0010] Perform time series prediction on the defect features through the Kalman filtering algorithm to generate a predicted feature set containing future defect states;

[0011] According to the predicted feature set and the index table, use the nearest neighbor search algorithm for feature matching to determine whether there is a defect pattern that matches the predicted feature set, and output the matching result;

[0012] Update the index table through the matching result, merge the newly detected defect features with the time series prediction results, and generate an extended defect feature index table;

[0013] For the extended index table, use the incremental clustering algorithm to classify the defect features to obtain the real-time updated defect type distribution.

[0014] Preferably, the process of generating the feature set including low-frequency and high-frequency components includes:

[0015] Obtain initial image data from high-resolution wafer images, and use a preprocessing algorithm to remove noise to obtain the first image;

[0016] Perform multi-scale decomposition on the first image through the pyramid decomposition algorithm to generate an image set with different resolutions;

[0017] Extract high-frequency components from the image set with different resolutions, use the wavelet transform algorithm to obtain texture information, and determine the texture feature set;

[0018] Extract low-frequency components from the image set with different resolutions, use the filtering algorithm to obtain morphological information, and determine the morphological feature set;

[0019] If the dimensions of the texture feature set and the morphological feature set are inconsistent, perform dimension alignment on the feature sets to obtain a unified feature set;

[0020] For the unified feature set, perform dimensionality reduction processing through the principal component analysis algorithm to obtain a compressed feature set;

[0021] Construct a multi-scale feature vector through the compressed feature set to generate the final feature set.

[0022] Preferably, the process of obtaining the feature vector group including local and global information of the defect includes:

[0023] Encode the multi-scale features of the feature set using a convolutional neural network to obtain a preliminary feature vector;

[0024] If the local information of the preliminary feature vector is complete, fuse the global information through network layer structure adjustment to obtain an optimized feature vector;

[0025] Based on the optimized feature vector, judge the significance of the local information of the defect and determine the significant feature subset;

[0026] For the significant feature subset, adjust the encoding parameters to generate a feature vector group containing local and global information of the defect.

[0027] Preferably, the process of obtaining the enhanced feature set includes:

[0028] If the significance of the high-frequency components in the feature vector group exceeds a preset threshold, obtain the high-frequency components from the input data through Fourier transform to obtain a high-frequency feature set;

[0029] Perform weighted processing on the high-frequency feature set through an attention mechanism to obtain an enhanced feature set;

[0030] Extract features from the enhanced feature set using a convolutional neural network to obtain an optimized feature set;

[0031] If the significance of the optimized feature set is lower than the preset threshold, expand it through data augmentation technology to obtain an enhanced feature set;

[0032] Group the enhanced feature set through a clustering algorithm to obtain a feature grouping set;

[0033] According to the feature grouping set, use a linear regression model to rank the feature importance to obtain a ranked feature set;

[0034] Through the ranked feature set, obtain the final feature optimization result and determine the enhanced feature set.

[0035] Preferably, the process of generating a defect feature index table that can be quickly retrieved includes:

[0036] Generate an enhanced feature set from the input data through feature extraction to obtain a set of feature vectors;

[0037] Process the set of feature vectors using a hash coding method to generate low-dimensional hash codes;

[0038] Construct an index structure based on the low-dimensional hash codes to generate a defect feature index table;[[ID=4|6]]

[0039] If a retrieval request arrives, query the matching low-dimensional hash codes through the index table to determine the candidate defect feature set;

[0040] For the candidate defect feature set, the cosine similarity is used to calculate the matching degree with the query feature, and a sorted defect feature list is obtained;

[0041] Based on the sorted defect feature list, a defect feature index table that can be quickly retrieved is generated.

[0042] Preferably, the process of obtaining the change trend of defect features over time includes:

[0043] Continuous sequential wafer images are obtained from the production line, and a first image sequence is obtained by using a preset image acquisition device;

[0044] For the first image sequence, a convolutional neural network is used to extract enhanced features, and a feature set corresponding to each image is obtained;

[0045] According to the feature set, the Euclidean distance between adjacent feature sets is calculated to obtain a distance sequence;

[0046] If a certain value in the distance sequence exceeds a preset threshold, it is judged as a defect feature, and a defect feature sequence is obtained;

[0047] According to the defect feature sequence, linear regression analysis is used to analyze the time series, and the change trend of defect features over time is obtained;

[0048] For the change trend, the trend slope is obtained, and it is judged whether the slope is positive, negative or zero to obtain the trend direction;

[0049] Through the trend direction, combined with the time series, the evolution mode of the defect feature is determined, and the change trend of the defect feature over time is obtained.

[0050] Preferably, the process of outputting the matching result includes:

[0051] The input prediction feature set is obtained, and a feature vector set is generated by using a feature extraction method;

[0052] The feature vector set is standardized through data preprocessing to obtain a standardized feature set;

[0053] If the standardized feature set meets the preset feature integrity condition, according to the defect pattern library in the preset index table, the nearest neighbor search algorithm is used to calculate the distance between the standardized feature set and each pattern in the defect pattern library, and a distance calculation result is obtained;

[0054] According to the distance calculation result, the pattern recognition rule is used to screen out the defect pattern with the smallest distance to obtain a preliminary matching pattern;

[0055] If the distance of the preliminary matching pattern is less than a preset threshold, the preliminary matching pattern is subjected to consistency verification through a result verification mechanism to determine the final matching pattern;

[0056] For the final matching pattern, extract the corresponding defect description information from the defect pattern library to generate the output matching result;

[0057] Through the output of the matching result, record the mapping relationship between the standardized feature set and the final matching pattern, and update the preset index table.

[0058] On the other hand, the present invention also provides a real-time wafer defect detection system based on multi-scale feature fusion, including:

[0059] A multi-scale feature extraction module, which is used to obtain multi-scale features from high-resolution wafer images, extract texture and morphological information at different resolutions through the pyramid decomposition algorithm, and generate a feature set containing low-frequency and high-frequency components;

[0060] A feature encoding module, which is used to perform preliminary encoding on each scale feature of the generated feature set by using a convolutional neural network to obtain a feature vector group containing local and global defect information;

[0061] A high-frequency feature enhancement module, which is used to, if the significance of the high-frequency components in the feature vector group exceeds a preset threshold, perform weighted enhancement on the high-frequency features through an attention mechanism to obtain an enhanced feature set;

[0062] An index construction module, which is used to construct an index structure based on hash coding according to the enhanced feature set, generate a defect feature index table that can be quickly retrieved by mapping the feature vectors to a low-dimensional space;

[0063] A time-series feature analysis module, which is used to obtain wafer images with continuous time series from the production line, extract the enhanced feature sets corresponding to each image, calculate the Euclidean distance between the feature sets, and obtain the change trend of the defect features over time;

[0064] A time-series prediction module, which is used to, if the distance difference in the change trend exceeds a preset dynamic threshold, perform time-series prediction on the defect features through the Kalman filter algorithm to generate a prediction feature set containing future defect states;

[0065] A feature matching module, which is used to perform feature matching by using the nearest neighbor search algorithm according to the prediction feature set and the index table, determine whether there is a defect pattern that matches the prediction feature set, and output the matching result;

[0066] An index update module, which is used to update the index table through the matching result, merge the newly detected defect features with the time-series prediction results, and generate an extended defect feature index table;

[0067] A defect classification module, which is used to classify the defect features by using an incremental clustering algorithm for the extended index table to obtain a real-time updated defect type distribution.

[0068] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor. When the processor executes the computing program, the method is implemented.

[0069] On the other hand, the present invention also provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the method is implemented.

[0070] Compared with the prior art, the present invention has the following advantages and technical effects:

[0071] The present invention discloses a wafer defect detection and prediction method. Through multi-scale feature extraction and convolutional neural network encoding, a feature vector group containing local and global defect information is obtained. Attention enhancement is performed on high-frequency features to highlight micron and nanometer-level defects. An index structure based on hash coding is constructed to achieve fast retrieval. By analyzing the feature change trend of consecutive sequential wafer images, the Kalman filter algorithm is used for defect state prediction. Nearest neighbor search is used for feature matching to judge the predicted defect pattern. The index table is dynamically updated and incremental clustering is performed to obtain the real-time defect type distribution. The present invention can effectively detect and predict wafer micro-defects, improve the accuracy and efficiency of defect identification, and provide important support for wafer manufacturing quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0073] Figure 1 is a schematic flowchart of the method according to an embodiment of the present invention;

[0074] Figure 2 is a schematic structural diagram of the system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0075] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will refer to the drawings and combine the embodiments to detail this application.

[0076] It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0077] Embodiment 1

[0078] AsFigure 1-2 As shown in Figure 1-2 , in this embodiment, a real-time wafer defect detection method and system based on multi-scale feature fusion are provided, including:

[0079] S101. Obtain multi-scale features from a high-resolution wafer image, extract texture and morphological information at different resolutions through a pyramid decomposition algorithm, and generate a feature set containing low-frequency and high-frequency components.

[0080] Obtain initial image data from a high-resolution wafer image, remove noise using a preprocessing algorithm to obtain a first image. Perform multi-scale decomposition on the first image through a pyramid decomposition algorithm to generate an image set with different resolutions. Extract high-frequency components from the image set with different resolutions, use a wavelet transform algorithm to obtain texture information, and determine a texture feature set. Extract low-frequency components from the image set with different resolutions, use a filtering algorithm to obtain morphological information, and determine a morphological feature set. If the dimensions of the texture feature set and the morphological feature set are inconsistent, align the dimensions of the feature set to obtain a unified feature set. For the unified feature set, perform dimensionality reduction processing using a principal component analysis algorithm to obtain a compressed feature set. Through the compressed feature set, construct a multi-scale feature vector to generate a final feature set.

[0081] Exemplarily, in the analysis of a high-resolution wafer image, first perform multi-scale decomposition using a Gaussian pyramid algorithm. Downsample by continuously applying a Gaussian kernel with σ = 1.6, and the resolution of each layer is reduced to 1 / 2 of the previous layer. A total of 5 layers of pyramids are constructed. For an original image of 2048×2048 pixels, the resolution of the second layer becomes 1024×1024, the third layer is 512×512, and so on. On each layer of the pyramid, use the LBP (Local Binary Pattern) algorithm to extract texture features, set a circular neighborhood with a radius of 3 pixels, and sample 8 points to generate a 256-dimensional feature vector. At the same time, apply the Sobel operator to calculate the gradient magnitude, and use a 3×3 convolution kernel to obtain morphological features. For high-frequency components, through Laplacian pyramid reconstruction, upsample the Nth layer image by 2 times and subtract it from the (N - 1)th layer to obtain a high-frequency residual map containing edges and defects. The low-frequency components are directly taken from the wavelet transform result of the top layer 512×512 image of the Gaussian pyramid. Use the db4 wavelet basis for 3-level decomposition, and extract the mean and variance of the LL3 subband as global features. Finally, merge the LBP features at each scale, the gradient magnitude statistic, the gray-level co-occurrence matrix (GLCM) features (including contrast, correlation, energy, and a displacement of d = 1 pixel is used during calculation) of the high-frequency residual map, and the wavelet low-frequency features into a 1536-dimensional feature vector to generate a feature set containing low-frequency and high-frequency components.

[0082] S102. For the generated feature set, use a convolutional neural network to perform preliminary encoding on the features at each scale to obtain a feature vector group containing local and global defect information.

[0083] Through data preprocessing, an initial feature set is obtained from the input image data. A convolutional neural network is used to encode the multi-scale features of the initial feature set to obtain a preliminary feature vector. If the local information of the preliminary feature vector is complete, the global information is fused through network layer structure adjustment to obtain an optimized feature vector. Based on the optimized feature vector, the significance of the local information of the defect is judged to determine the significant feature subset. For the significant feature subset, encoding parameter adjustment is adopted to generate a feature vector group containing local and global information of the defect. Through the evaluation of the feature extraction efficiency of the feature vector group, it is judged whether the preset threshold is met. If not, the encoding parameter adjustment step is returned to regenerate the feature vector group. The final feature vector group is obtained and output for subsequent defect detection processes.

[0084] Exemplarily, in the stage of generating the feature set, first, multi-scale sliding windows are used to extract local features of the image. For example, for a defect image of 512×512 pixels, three window sizes of 16×16, 32×32, and 64×64 are set, and the strides are 8, 16, and 32 pixels respectively. The texture feature vectors of each window are calculated through the Difference of Gaussian operator, and finally a combined feature set containing 128-dimensional HOG features, 64-dimensional LBP features, and 256-dimensional SIFT features is generated. Then, a three-layer convolutional neural network is constructed for feature encoding. In the first layer, a 5×5 convolutional kernel is used in combination with the ReLU activation function, the number of output channels is 64, and the stride is 2 to downsample the input feature map; in the second layer, a 3×3 dilated convolution is used to expand the receptive field with a dilation rate of 2, and the number of output channels is increased to 128; in the third layer, the features of each scale are compressed into a fixed-length 256-dimensional feature vector through global average pooling. In the feature fusion stage, the feature vectors of the three scales are combined through weighted concatenation, where the weight of the 16×16 scale is set to 0.3, the 32×32 scale is 0.5, and the 64×64 scale is 0.2 to form a final 768-dimensional mixed feature vector. To verify the effectiveness of the features, the t-SNE algorithm is used to reduce the high-dimensional features to a 2D space for visualization. When the Euclidean distance between samples of the same type of defect is less than 0.15, it indicates that the features have good intra-class aggregation. In the subsequent classifier, this feature vector group can enable the ResNet18 model to achieve an accuracy of 98.7% on the steel plate defect data set, which is 6.2 percentage points higher than that of a single-scale feature.

[0085] S103. If the significance of the high-frequency components in the feature vector group exceeds the preset threshold, the high-frequency features are weighted and enhanced through an attention mechanism to obtain an enhanced feature set.

[0086] If the significance of the high-frequency components in the feature vector group exceeds a preset threshold, the high-frequency components are obtained from the input data through Fourier transform to obtain a high-frequency feature set. The high-frequency feature set is weighted through an attention mechanism to obtain an enhanced feature set. A convolutional neural network is used to extract features from the enhanced feature set to obtain an optimized feature set. If the significance of the optimized feature set is lower than the preset threshold, it is augmented through data augmentation techniques to obtain an enhanced feature set. The enhanced feature set is grouped through a clustering algorithm to obtain a feature grouping set. Based on the feature grouping set, a linear regression model is used to rank the feature importance to obtain a ranked feature set. Through the ranked feature set, the final feature optimization result is obtained to determine the output feature set.

[0087] Exemplarily, in the feature vector group, first, the signal is transformed from the time domain to the frequency domain through the fast Fourier transform (FFT), and the amplitudes of each frequency component are calculated. Assuming the input signal is a sequence of length 1024, after FFT, 512 frequency components are obtained, where the high-frequency components are located at the 256th to 512th frequency points. By setting the preset threshold to 0.8, the significance of the high-frequency components is judged. If the amplitude of a certain high-frequency component exceeds 0.8, it is marked as a significant high-frequency component. Next, an attention mechanism is used to weight and enhance these significant high-frequency features.

[0088] S104. According to the enhanced feature set, an index structure based on hash coding is constructed. By mapping the feature vectors to a low-dimensional space, a defect feature index table that can be quickly retrieved is generated.

[0089] Reinforced feature sets are generated from the input data through feature extraction to obtain a set of feature vectors. The set of feature vectors is processed using a hash coding method to generate low-dimensional hash codes. An index structure is constructed based on the low-dimensional hash codes to generate a defect feature index table. If a retrieval request arrives, the matching low-dimensional hash codes are queried through the index table to determine a candidate set of defect features. For the candidate set of defect features, the cosine similarity is used to calculate the matching degree with the query feature to obtain a ranked list of defect features. Through the ranked list of defect features, the most similar defect feature is obtained to judge the final retrieval result. According to the final retrieval result, the corresponding defect feature information is output.

[0090] Exemplarily, when constructing an index structure based on hash coding, first, the local sensitive hashing (LSH) algorithm is used to perform dimensionality reduction processing on the enhanced feature set. Specifically, 4 hash functions are generated using the random projection method, and each function maps a 128-dimensional defect feature vector to an 8-bit binary code to form a compact hash key.

[0091] For example, for the feature vector [0.23, -0.45,..., 1.2], a hash code such as "10110010" is generated by calculating its dot product with a random matrix and binarizing it. Then, a multi-probe hash table is used to store these codes, with the bucket width set to 0.3. When querying the feature "10110100", the system retrieves all buckets with a Hamming distance less than or equal to 2 and returns the set of similar features. To optimize the retrieval efficiency, a Bloom filter is used to preprocess the query request, with a false positive rate set to 0.05, and 3 hash functions are used to quickly filter out irrelevant buckets. In the feature matching stage, the exact similarity of the retrieved candidate set is calculated, with a cosine similarity threshold of 0.85, and only features above this value are retained. When updating the index, the hash function parameters are adjusted through incremental learning. When a new batch of features arrives, the random projection matrix is updated with a learning rate of 0.01 to ensure that the index structure dynamically adapts to changes in the feature distribution. The entire process is accelerated through parallel computing. The features are partitioned into 8 computing nodes, with each node processing 16-dimensional sub-features, and the results are finally merged through a reduction operation, controlling the retrieval time of millions of features within 50 milliseconds.

[0092] S105. Obtain continuous sequential wafer images from the production line, extract the enhanced feature sets corresponding to each image, and calculate the Euclidean distance between the feature sets to obtain the change trend of the defect features over time.

[0093] Obtain continuous sequential wafer images from the production line using a preset image acquisition device to obtain a first image sequence. For the first image sequence, use a convolutional neural network to extract enhanced features to obtain the feature sets corresponding to each image. According to the feature sets, calculate the Euclidean distance between adjacent feature sets, with the formula d(p, q) = √(∑(pi - qi)²), where p and q are feature vectors, and pi and qi are vector components, to obtain a distance sequence. If a certain value in the distance sequence exceeds a preset threshold, it is determined as a defect feature to obtain a defect feature sequence. According to the defect feature sequence, use linear regression to analyze the time series to obtain the change trend of the defect features over time. For the change trend, obtain the trend slope, and determine whether the slope is positive, negative, or zero to obtain the trend direction. Through the trend direction, combined with the time series, determine the evolution pattern of the defect features to obtain an evolution feature set.

[0094] Exemplarily, on a wafer production line, continuous sequential images are collected by a high-speed industrial camera at a rate of 30 frames per second. The resolution of each frame of the image is 2048×2048 pixels. The adaptive histogram equalization algorithm is used for preprocessing, and the enhanced contrast parameter is set to α = 0.5. In the feature extraction stage, an improved ResNet50 network is used, and an SE attention module is added after the third convolutional layer to output a 512-dimensional feature vector. The spatial pyramid pooling layer is set with 4 scales (1×1, 2×2, 3×3, 6×6). For two adjacent frame feature vectors Ft and Ft+1, when calculating the normalized Euclidean distance, a dynamic weight matrix W = diag(0.2, 0.3, 0.5) is used to weight different feature channels, and the distance threshold is set to 1.25. When it is detected that the distance of 5 consecutive frames exceeds the threshold, the defect analysis process is triggered. A Savitzky-Golay filter with a sliding window size of 15 is used to smooth the distance curve, and the second-order polynomial fitting parameters are a = 0.03, b = -0.12, c = 1.45. In the trend analysis stage, the abnormality is judged by calculating the cross situation of the moving average (window width = 20) and the standard deviation band (±2σ). When the distance value breaks through the upper track and lasts for more than 3 sampling periods, it is determined as a progressive defect feature. For sudden defects, a CUSUM control chart is used for modeling, the reference value K = 0.8, the decision interval H = 5.2, and an alarm is issued when the cumulative sum exceeds H. All feature data are stored in a time series database with a sampling interval of 100 ms. An LSTM network is used to predict the feature change trend in the next 5 seconds. The number of hidden layer units is set to 128, the Dropout rate is 0.3, the Adam optimizer is used during training, the initial learning rate is 0.001, and the batch size is 64.

[0095] S106. If the distance difference in the change trend exceeds the preset dynamic threshold, the Kalman filtering algorithm is used to perform time series prediction on the defect feature to generate a prediction feature set including future defect states.

[0096] If the distance difference in the change trend exceeds the preset dynamic threshold, the defect feature is obtained from the input data through a feature extraction algorithm to obtain an initial defect feature set. The Kalman filtering algorithm is used to perform time series prediction on the initial defect feature set to generate a prediction feature set including future defect states. If the probability distribution of the defect states in the prediction feature set exceeds the preset range, a statistical analysis method is used to cluster the prediction feature set to determine the defect state classification. According to the defect state classification, a feature template matching the current classification is obtained from the historical data to obtain a standard feature template. By comparing the similarity between the prediction feature set and the standard feature template, the potential risk level of the defect state is judged. If the potential risk level is higher than the preset threshold, a weighted fusion method is used to optimize the prediction feature set to generate an optimized feature set. The time series distribution of future defect states is generated through the optimized feature set to obtain the final prediction result.

[0097] Exemplarily, in the scenario of industrial equipment defect monitoring, the preset dynamic threshold is 0.15 mm. When the distance differences (L2 norm between the current frame and the previous frame) of three consecutive frames of vibration signals reach 0.18 mm, 0.21 mm, and 0.25 mm, the prediction mechanism is triggered. The Kalman filter algorithm is used to establish a state space model, where the state variable X includes the defect length, width, and depth (initial value [2.1, 0.8, 0.3] mm), the observation matrix H is set as the identity matrix, the process noise covariance Q = diag(0.01, 0.01, 0.01), and the measurement noise covariance R = diag(0.05, 0.05, 0.05). The prior state estimate is calculated through the time update step, and then the latest observation value Zk = [2.3, 0.9, 0.4] mm is fused through the measurement update step. Finally, the posterior state estimate is obtained. After iteratively performing 10 predictions, a defect feature sequence for the next 5 minutes is generated. The prediction results show that the depth dimension increases most significantly, reaching 0.52 ± 0.03 mm at the 5th minute, exceeding the safety threshold of 0.5 mm, and triggering the warning mechanism. During the whole process, a sliding window mechanism is used to maintain 30 sets of historical data. When the root mean square value of the prediction error within the window continuously exceeds 0.08 mm, the Q matrix parameter is automatically adjusted to diag(0.015, 0.015, 0.015) to enhance the system sensitivity.

[0098] S107. According to the prediction feature set and the index table, the nearest neighbor search algorithm is used for feature matching to determine whether there is a defect pattern that matches the prediction feature set, and the matching result is output.

[0099] Obtain the input prediction feature set, and use the feature extraction method to generate a feature vector set. The feature vector set is standardized through data preprocessing to obtain a standardized feature set. If the standardized feature set meets the preset feature integrity condition, then according to the defect pattern library in the preset index table, the nearest neighbor search algorithm is used to calculate the distances between the standardized feature set and each pattern in the defect pattern library, and the distance calculation results are obtained. According to the distance calculation results, the pattern recognition rule is used to screen out the defect pattern with the smallest distance to obtain the preliminary matching pattern. If the distance of the preliminary matching pattern is less than the preset threshold, then the result verification mechanism is used to perform consistency verification on the preliminary matching pattern to determine the final matching pattern. For the final matching pattern, the corresponding defect description information is extracted from the defect pattern library to generate the matching result for output. Through the output of the matching result, the mapping relationship between the standardized feature set and the final matching pattern is recorded, and the preset index table is updated.

[0100] Exemplarily, during the feature matching process, it is first necessary to construct a feature index table containing known defect patterns. For example, the index table contains defect patterns A, B, and C, corresponding to feature vectors [0.8, 0.2, 0.5], [0.6, 0.4, 0.7], and [0.9, 0.1, 0.3] respectively. Suppose the input predicted feature set is [0.85, 0.15, 0.4]. The nearest neighbor search algorithm is used for matching. The Euclidean distances between the predicted feature set and each feature vector in the index table are calculated, and the distances to A, B, and C are 0.07, 0.36, and 0.12 respectively. By comparison, the distance between the predicted feature set and defect pattern A is the smallest. Therefore, it is determined that there is a defect pattern A that matches the predicted feature set. To further verify the accuracy of the matching result, the cosine similarity algorithm can be introduced. The cosine similarity between the predicted feature set and A is calculated to be 0.98, indicating that their directions in the feature space are highly consistent, further confirming the reliability of the matching result. Through this dual verification mechanism combining distance and similarity, the accuracy and robustness of feature matching can be effectively improved.

[0101] S108. Update the index table according to the matching result, merge the newly detected defect features with the time series prediction result, and generate an extended defect feature index table.

[0102] Obtain the newly recognized defect feature data from the detection device, use the feature extraction algorithm to determine the category and location of the defect features, and obtain the feature description set. By comparing the feature description set with the pre-established index table, if the category and location in the feature description set do not match the records in the index table, perform a table update operation to generate an updated index table. Obtain the updated index table, use the time series prediction model to analyze the evolution trend of the defect features, and obtain the predicted feature set. For the predicted feature set and the feature description set, use the data fusion algorithm to integrate their attribute information to generate a fused feature set. Extract extended features from the fused feature set, use the index generation algorithm to update the index table, and obtain an extended defect feature index table. If the number of features in the extended defect feature index table exceeds the preset threshold, use the feature screening algorithm to streamline redundant features to obtain an optimized index table. Through the optimized index table, use the data storage technology to save the final defect feature index to obtain the persistent storage result.

[0103] Exemplarily, during the defect detection process, first, the acquired product surface image is analyzed through an image processing algorithm, and a convolutional neural network (CNN) is used to extract defect features such as cracks and scratches. Suppose 5 new defects are detected, and their feature vectors are [0.85, 0.12, 0.34], [0.92, 0.08, 0.45], [0.78, 0.15, 0.29], [0.88, 0.10, 0.37], [0.81, 0.13, 0.31] respectively. Then, a time series prediction model (such as LSTM) is used to predict the defect development trend. Suppose the prediction results are that the defect areas will increase by 10%, 15%, 8%, 12%, and 9% respectively. These prediction results are combined with the detected defect features to generate an extended defect feature index table.

[0104] For example, the feature vector of the first defect is updated to [0.85, 0.12, 0.34, 0.10], where 0.10 represents the predicted area increase ratio. In this way, the extended index table not only contains the current defect features but also their future development trends, providing more comprehensive data support for subsequent quality control. Finally, a clustering algorithm (such as K-means) is used to group the extended defect features. Suppose they are divided into 3 categories, representing minor, medium, and severe defects respectively, so as to take corresponding treatment measures for different categories of defects.

[0105] S109. For the extended index table, an incremental clustering algorithm is used to classify the defect features to obtain a real-time updated defect type distribution.

[0106] Obtain defect feature data from the extended index table, extract key features through preprocessing to obtain a feature vector set. Use an incremental clustering algorithm to classify the feature vector set to determine the initial defect type distribution. If the feature vector set changes, the clustering model is dynamically adjusted through data stream processing to update the defect type distribution. According to the updated defect type distribution, obtain the center point features of the clustering model to get the clustering result. For the clustering result, reorganize the extended index table through an index management mechanism to obtain an optimized index structure. Extract defect features from the optimized index structure, and loop to execute incremental clustering to update the defect type distribution in real time. If the change trend of the type distribution is stable, verify the consistency of the clustering model through feature extraction to determine the final defect type distribution.

[0107] Exemplarily, in the extended index table, an incremental clustering algorithm is used to classify defect features. First, the clustering centers are initialized by the K-means algorithm. Assume the initial clustering centers are C1(0.2, 0.3), C2(0.5, 0.6), and C3(0.8, 0.1). When the new defect data D1(0.25, 0.35) is input, calculate its Euclidean distance from each clustering center. It is found that the distance from D1 to C1 is the smallest, so D1 is classified into the cluster where C1 is located. Subsequently, update the coordinates of C1 to (0.225, 0.325) to reflect the addition of the new data. Then, when D2(0.55, 0.65) is input, calculate its distance from each clustering center in the same way. It is found that the distance from D2 to C2 is the smallest, so D2 is classified into the cluster where C2 is located, and update the coordinates of C2 to (0.525, 0.625). In this way, the system can update the defect type distribution in real time to ensure the accuracy of the clustering result. During the analysis process, the silhouette coefficient is used to evaluate the clustering effect. Assume the current silhouette coefficient is 0.65, indicating that the clustering effect is good. In addition, the system also detects outliers through the DBSCAN algorithm. If D3(0.9, 0.95) is found to be an outlier, it will be marked separately for further analysis. Through the above method, the system can efficiently classify defect features and update the defect type distribution in real time.

[0108] On the other hand, this embodiment also provides a real-time wafer defect detection system based on multi-scale feature fusion, including:

[0109] A multi-scale feature extraction module, which is used to obtain multi-scale features from high-resolution wafer images, extract texture and morphological information at different resolutions through the pyramid decomposition algorithm, and generate a feature set containing low-frequency and high-frequency components;

[0110] A feature encoding module, which is used to perform preliminary encoding on the features of each scale in the generated feature set by using a convolutional neural network to obtain a feature vector group containing local and global defect information;

[0111] A high-frequency feature enhancement module, which is used to, if the significance of the high-frequency components in the feature vector group exceeds a preset threshold, perform weighted enhancement on the high-frequency features through an attention mechanism to obtain an enhanced feature set;

[0112] An index construction module, which is used to construct an index structure based on hash coding according to the enhanced feature set, generate a defect feature index table that can be quickly retrieved by mapping the feature vectors to a low-dimensional space;

[0113] A time-series feature analysis module, which is used to obtain wafer images with continuous time series from the production line, extract the enhanced feature sets corresponding to each image, calculate the Euclidean distance between the feature sets, and obtain the change trend of defect features over time;

[0114] A timing prediction module, which is used to perform timing prediction on defect features through the Kalman filtering algorithm and generate a prediction feature set containing future defect states if the distance difference in the change trend exceeds a preset dynamic threshold;

[0115] A feature matching module, which is used to perform feature matching according to the prediction feature set and the index table by using the nearest neighbor search algorithm, determine whether there is a defect pattern that matches the prediction feature set, and output a matching result;

[0116] An index update module, which is used to update the index table through the matching result, merge the newly detected defect features with the timing prediction result, and generate an extended defect feature index table;

[0117] A defect classification module, which is used to classify defect features by using an incremental clustering algorithm for the extended index table to obtain a real-time updated defect type distribution.

[0118] On the other hand, this embodiment also provides an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor. When the processor executes the computing program, the method is implemented.

[0119] On the other hand, this embodiment also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and is characterized in that when the computer program is executed by a processor, the method is implemented.

[0120] The above is only a preferred specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A real-time wafer defect detection method based on multi-scale feature fusion, characterized in that Including: Obtain multi-scale features from high-resolution wafer images, extract texture and morphological information at different resolutions through the pyramid decomposition algorithm, and generate a feature set containing low-frequency and high-frequency components; For the generated feature set, use a convolutional neural network to preliminarily encode the features at each scale to obtain a feature vector group containing local and global defect information; If the significance of the high-frequency components in the feature vector group exceeds a preset threshold, use an attention mechanism to weight and enhance the high-frequency features to obtain an enhanced feature set; According to the enhanced feature set, construct an index structure based on hash coding, and generate a defect feature index table that can be quickly retrieved by mapping the feature vectors to a low-dimensional space; Obtain wafer images with continuous time series from the production line, extract the enhanced feature sets corresponding to each image, calculate the Euclidean distance between the feature sets, and obtain the change trend of defect features over time; Perform temporal prediction on the defect features through the Kalman filter algorithm to generate a predicted feature set containing future defect states; According to the predicted feature set and the index table, use the nearest neighbor search algorithm for feature matching to determine whether there is a defect pattern that matches the predicted feature set, and output the matching result; Update the index table through the matching result, merge the newly detected defect features with the temporal prediction results, and generate an extended defect feature index table; For the extended index table, use an incremental clustering algorithm to classify the defect features to obtain a real-time updated defect type distribution.

2. The method according to claim 1, wherein The process of generating the feature set containing low-frequency and high-frequency components includes: Obtain initial image data from high-resolution wafer images, and use a preprocessing algorithm to remove noise to obtain the first image; Perform multi-scale decomposition on the first image through the pyramid decomposition algorithm to generate an image set with different resolutions; Extract high-frequency components from the image sets with different resolutions, use the wavelet transform algorithm to obtain texture information, and determine the texture feature set; Extract low-frequency components from the image sets with different resolutions, use a filtering algorithm to obtain morphological information, and determine the morphological feature set; If the dimensions of the texture feature set and the morphological feature set are inconsistent, align the dimensions of the feature sets to obtain a unified feature set; For the unified feature set, perform dimensionality reduction processing using the principal component analysis algorithm to obtain a compressed feature set; Through the compressed feature set, construct multi-scale feature vectors to generate the final feature set.

3. The method according to claim 1, wherein The process of obtaining the feature vector group containing local and global defect information includes: Use a convolutional neural network to encode the multi-scale features of the feature set to obtain preliminary feature vectors; If the local information of the preliminary feature vectors is complete, fuse the global information through network layer structure adjustment to obtain optimized feature vectors; According to the optimized feature vectors, judge the significance of the local defect information and determine the significant feature subset; For the significant feature subset, adjust the coding parameters to generate a feature vector group containing local and global defect information.

4. The method according to claim 1, wherein The process of obtaining the enhanced feature set includes: If the significance of the high-frequency components in the feature vector group exceeds a preset threshold, obtain the high-frequency components from the input data through Fourier transform to obtain the high-frequency feature set; The high-frequency feature set is weighted through an attention mechanism to obtain an enhanced feature set; A convolutional neural network is used to extract features from the enhanced feature set to obtain an optimized feature set; If the significance of the optimized feature set is lower than a preset threshold, it is augmented through data augmentation techniques to obtain an augmented feature set; The augmented feature set is grouped through a clustering algorithm to obtain a feature grouping set; According to the feature grouping set, a linear regression model is used to rank the feature importance to obtain a ranked feature set; Through the ranked feature set, the final feature optimization result is obtained to determine the enhanced feature set.

5. The method according to claim 1, wherein The process of generating a defect feature index table that can be quickly retrieved includes: An enhanced feature set is generated from input data through feature extraction to obtain a set of feature vectors; The set of feature vectors is processed using a hash coding method to generate low-dimensional hash codes; An index structure is constructed based on the low-dimensional hash codes to generate a defect feature index table; If a retrieval request arrives, the matching low-dimensional hash codes are queried through the index table to determine a candidate defect feature set; For the candidate defect feature set, the cosine similarity is used to calculate the matching degree with the query feature to obtain a ranked list of defect features; Based on the ranked list of defect features, a defect feature index table that can be quickly retrieved is generated.

6. The method according to claim 1, wherein The process of obtaining the change trend of defect features over time includes: Continuous sequential wafer images are obtained from the production line using a preset image acquisition device to obtain a first image sequence; For the first image sequence, a convolutional neural network is used to extract enhanced features to obtain a feature set corresponding to each image; According to the feature sets, the Euclidean distance between adjacent feature sets is calculated to obtain a distance sequence; If a certain value in the distance sequence exceeds a preset threshold, it is determined as a defect feature to obtain a defect feature sequence; According to the defect feature sequence, linear regression is used to analyze the time series to obtain the change trend of defect features over time; For the change trend, the trend slope is obtained, and it is judged whether the slope is positive, negative or zero to obtain the trend direction; Through the trend direction, combined with the time series, the evolution pattern of defect features is determined to obtain the change trend of defect features over time.

7. The method according to claim 1, characterized in that, The process of outputting the matching result includes: An input predicted feature set is obtained, and a set of feature vectors is generated using a feature extraction method; The set of feature vectors is standardized through data preprocessing to obtain a standardized feature set; If the standardized feature set meets the preset feature integrity condition, according to the defect pattern library in the preset index table, the nearest neighbor search algorithm is used to calculate the distance between the standardized feature set and each pattern in the defect pattern library to obtain a distance calculation result; According to the distance calculation result, the pattern recognition rule is used to screen out the defect pattern with the smallest distance to obtain a preliminary matching pattern; If the distance of the preliminary matching pattern is less than a preset threshold, the preliminary matching pattern is verified for consistency through a result verification mechanism to determine the final matching pattern; For the final matching pattern, the corresponding defect description information is extracted from the defect pattern library to generate an output matching result; Through the output of the matching result, the mapping relationship between the standardized feature set and the final matching pattern is recorded, and the preset index table is updated.

8. A real-time wafer defect detection system based on multi-scale feature fusion, characterized in that, Including: A multi-scale feature extraction module, which is used to obtain multi-scale features from high-resolution wafer images, extract texture and morphological information at different resolutions through a pyramid decomposition algorithm, and generate a feature set containing low-frequency and high-frequency components; A feature encoding module, which is used to preliminarily encode each scale feature of the generated feature set by using a convolutional neural network to obtain a feature vector group containing local and global defect information; A high-frequency feature enhancement module, which is used to, if the significance of the high-frequency components in the feature vector group exceeds a preset threshold, perform weighted enhancement on the high-frequency features through an attention mechanism to obtain an enhanced feature set; An index construction module, which is used to construct an index structure based on hash coding according to the enhanced feature set, and generate a defect feature index table that can be quickly retrieved by mapping the feature vectors to a low-dimensional space; A time-series feature analysis module, which is used to obtain wafer images with continuous time series from a production line, extract the enhanced feature sets corresponding to each image, calculate the Euclidean distance between the feature sets, and obtain the change trend of defect features over time; A time-series prediction module, which is used to, if the distance difference in the change trend exceeds a preset dynamic threshold, perform time-series prediction on the defect features through a Kalman filtering algorithm to generate a prediction feature set containing future defect states; A feature matching module, which is used to perform feature matching by using a nearest neighbor search algorithm according to the prediction feature set and the index table, judge whether there is a defect pattern that matches the prediction feature set, and output a matching result; An index update module, which is used to update the index table according to the matching result, merge the newly detected defect features with the time-series prediction results, and generate an extended defect feature index table; A defect classification module, which is used to classify the defect features by using an incremental clustering algorithm for the extended index table to obtain a real-time updated defect type distribution.

9. An electronic device, comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, characterized in that, When the processor executes the computing program, the method described in any one of claims 1-7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the method described in any one of claims 1-7 is implemented.

Images