Hierarchical clustering-based unsupervised silicon wafer surface defect detection method and device

Abstract

The invention discloses an unsupervised silicon wafer surface defect detection method and device based on hierarchical clustering, relates to the field of image processing and computer vision, and utilizes an unsupervised clustering technology to detect silicon wafer surface defects. The method comprises the following steps: extracting multi-scale features from a plurality of normal silicon wafer images through a pre-trained convolutional neural network; integrating the multi-scale features of the plurality of pictures by adopting a hierarchical fusion strategy; constructing a feature memory library by using a clustering algorithm, and generating a clustering center; generating patch features for the test image through a consistent fusion process; through comparative supervision, normal patches are close to a clustering center, and abnormal patches are pushed away; calculating the distance between the patch and the clustering center, and judging abnormity; and mapping the position of the abnormal patch, and positioning scratches, cracks, pollution or recesses. According to the method, through an unsupervised mode, a large amount of defect labeling data is not needed, the data preparation cost is effectively reduced, and the accuracy and robustness of silicon wafer surface defect detection are remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision, specifically to an unsupervised method and apparatus for detecting surface defects on silicon wafers based on hierarchical clustering. This method can efficiently detect and locate surface defects on silicon wafers, improving the accuracy and robustness of defect detection in semiconductor manufacturing. Background Technology

[0002] Silicon wafer surface defect detection is a key technology in semiconductor manufacturing and quality control, widely used in the production of integrated circuits, solar cells, and microelectronic devices. Its main purpose is to identify anomalies such as surface scratches, cracks, contamination, or dents by analyzing silicon wafer images, providing a basis for product quality assessment and process optimization. In integrated circuit manufacturing, even minute defects can lead to chip performance degradation or failure, requiring detection technologies with high precision and sensitivity. In solar cell production, surface defects affect photoelectric conversion efficiency, necessitating automated inspection to ensure product consistency. In the field of microelectronic devices, defect detection supports the development of high-reliability components, contributing to technological advancement. In conclusion, silicon wafer surface defect detection technology is of great significance for improving the efficiency of the semiconductor industry, reducing production costs, and ensuring product quality.

[0003] However, silicon wafer surface defect detection faces multiple challenges. First, defect samples are scarce, limiting the performance of traditional supervised learning methods due to a lack of labeled data. Second, silicon wafer images are susceptible to changes in lighting, reflection, or microscope viewing angle; these non-defect factors can cause false anomalies, increasing the risk of false detections. Furthermore, defect morphologies are diverse, ranging from minute scratches to large-area contamination, making it difficult for traditional methods to effectively integrate multi-scale features, thus limiting detection accuracy. In complex scenes, background textures or particle noise further interfere with detection, reducing robustness. Current methods mostly employ uniform feature processing, lacking differentiated strategies for features at different scales, failing to fully exploit the intrinsic properties of features, and hindering breakthroughs in detection technology.

[0004] Therefore, to overcome the aforementioned challenges, this invention proposes an unsupervised silicon wafer surface defect detection method based on hierarchical clustering. The method aims to construct a normal feature memory library and optimize feature distribution through unsupervised clustering and contrastive supervision techniques to accurately locate defects. This method requires no large amount of labeled data, effectively suppresses false anomaly interference, and improves detection accuracy and robustness, providing an efficient and reliable quality control solution for semiconductor manufacturing. Summary of the Invention

[0005] To address the aforementioned technical problems, the present invention aims to propose an unsupervised silicon wafer surface defect detection method and apparatus based on hierarchical clustering, in order to solve the challenges of scarce defect samples, non-defect interference, insufficient multi-scale feature fusion, and insufficient robustness in complex scenarios.

[0006] In a first aspect, the present invention provides an unsupervised method for detecting surface defects on silicon wafers based on hierarchical clustering, comprising the following steps:

[0007] S1 extracts multi-scale features from normal silicon wafer reference images through a pre-trained convolutional neural network, and integrates the features using a hierarchical progressive fusion strategy to generate a representative normal reference feature representation.

[0008] S2, use clustering algorithms to optimize normal reference feature identifiers, construct a normal silicon wafer feature memory library, and generate compact cluster centers;

[0009] S3, performs the same multi-scale feature extraction and fusion process on a single silicon wafer image to finally obtain patch features at each spatial location of the image;

[0010] S4. By using a comparative supervision method, the features of normal patches are brought closer to the cluster centers of the memory bank to generate a defect scoring map.

[0011] S5 maps the spatial location of abnormal patches, generates a defect map, and locates anomalies such as scratches, cracks, contamination, or dents on the silicon wafer surface.

[0012] Secondly, the present invention provides an unsupervised silicon wafer surface defect detection device based on hierarchical clustering, comprising:

[0013] The multi-scale feature extraction module is configured to extract multi-scale features from normal silicon wafers and test images using a pre-trained convolutional neural network to support subsequent defect analysis.

[0014] The incremental update module is configured to dynamically adjust the feature representation and optimize the stability of normal silicon wafer features across multiple scales through a incremental update process.

[0015] The hierarchical fusion module is configured to integrate multi-scale features to generate a unified feature representation, thereby improving the comprehensiveness of detection.

[0016] The contrast-supervised learning module is configured to bring the features of normal patches closer to the cluster center of the memory bank through contrast optimization, while pushing the features of abnormal patches further away, to generate an initial defect score.

[0017] The defect location module is configured to map the spatial location of abnormal patches, accurately locating scratches, cracks, contamination, or dents on the silicon wafer surface.

[0018] Thirdly, the present invention provides an apparatus comprising one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to perform the method as described in the first aspect.

[0019] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] (1) This invention uses an unsupervised hierarchical clustering method, which solves the problem of scarce defective samples, eliminates the need for a large amount of labeled data, and significantly reduces data preparation costs.

[0022] (2) By progressively updating the feature distribution, the features of multiple normal samples are absorbed, effectively suppressing non-defect interference such as illumination and reflection in a single normal sample, and improving detection robustness.

[0023] (3) By utilizing multi-scale feature fusion and patch space mapping, we can accurately capture various defects such as scratches and cracks, and improve detection accuracy.

[0024] (4) The overall approach is highly efficient and applicable to complex scenarios, providing a reliable quality control solution for semiconductor manufacturing. Attached Figure Description

[0025] Figure 1 This is a flowchart of an unsupervised silicon wafer surface defect detection method based on hierarchical clustering. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0027] Example 1:

[0028] An unsupervised method and apparatus for detecting surface defects on silicon wafers based on hierarchical clustering, the method comprising the following steps:

[0029] Step 101: Input Image Preprocessing and Feature Extraction. First, image samples from multiple normal silicon wafers are acquired and input into a pre-trained convolutional neural network. Feature maps at different scales are extracted through multiple layers of the network to capture information about the image at different levels, such as texture, shape, and edges. Each image is converted into a set of feature representations containing multi-scale information.

[0030] Step 102: Hierarchical Progressive Feature Fusion. To enhance the representational power of features, feature maps from multiple samples at the same scale are integrated into representative features for that scale using a fusion strategy. Subsequently, a bottom-up approach from low to high levels is adopted to progressively fuse representative features from all scales, generating a unified and stable fused feature map to represent the overall feature structure of a normal image.

[0031] Step 103: Memory Construction. All spatial locations in the fused feature map are expanded into several patch-level feature vectors and aggregated into a complete feature set. This set is analyzed using clustering methods to extract several representative cluster centers. These cluster centers, as typical features of normal images, are stored to construct a memory, used to represent the distribution of normal samples in the feature space.

[0032] Step 104: Image Processing. Perform the same multi-scale feature extraction and fusion operations as for normal samples on the silicon wafer image to be detected to obtain the fused feature map of the image, and further extract the patch feature vectors at its spatial location.

[0033] Step 105: Feature Comparison and Anomaly Score Generation. The similarity of each patch feature in the image to be detected is compared with the cluster centers in the memory database. By comparing distance relationships, it is determined whether the feature is close to the normal memory centers; if the deviation is large, it is considered an anomaly region. Based on the degree of deviation, a corresponding anomaly score is generated for each spatial location, forming a defect scoring map.

[0034] Step 106: Defect Map Generation and Location Mapping. Based on the high-scoring areas in the anomaly scoring map, the corresponding locations in the image space are deduced, thus generating a visualized defect map. This map can intuitively display the locations of potential defects such as scratches, cracks, contamination, and dents in the image, achieving automated defect localization.

[0035] Example 2:

[0036] To further illustrate the unsupervised silicon wafer surface defect detection method proposed in this invention, the following combines mathematical expressions and... Figure 1 The steps described in Example 1 will be explained in detail.

[0037] Step 201: Multi-scale feature extraction

[0038] In the method of this invention, to ensure the capture of multi-level information on the silicon wafer surface, from microscopic texture to macroscopic structure, image features need to be extracted from multiple levels. Given N normal silicon wafer images X i , where i∈{1,2,…,N}, are input into a pre-trained deep convolutional network WRN50-2 to extract multi-scale feature maps at L different depths.

[0039] To enhance the stability and consistency of feature representation, the original feature maps extracted from each layer are spatially averaged to compress redundant region information. Then, a lightweight convolutional module is used for structural refinement and dimensionality unification, ultimately forming a normalized feature map of each image at layer l. The calculation formula is as follows:

[0040]

[0041] Where AvgPool(·) represents average pooling, and Conv(·) represents standard convolution. This represents the feature extraction function of the l-th layer of the WRN50-2 network.

[0042] Step 202: Construction of the central feature map

[0043] To obtain a stable feature distribution representation for each layer across all normal samples, this invention employs a recursive feature mean aggregation strategy to progressively fuse the feature maps extracted from each layer. Specifically, for each layer l, starting from the first image sample, new image features are sequentially incorporated into the mean update to construct the central representation of that layer across all samples. The update formula is as follows:

[0044]

[0045] The initial setting is as follows: The final calculated center feature map This represents a stable feature representation of all training samples at this level, used for subsequent hierarchical fusion. This step not only effectively mitigates local perturbations caused by non-defect factors such as illumination and reflection in individual images, but also improves the compactness and robustness within the feature space.

[0046] Step 203: Hierarchical fusion to generate normal feature map C L

[0047] To fully integrate multi-scale semantic information and eliminate distribution differences between different levels, this invention designs a bottom-up hierarchical feature fusion mechanism. The fusion process starts from the bottom layer and integrates the central feature maps of higher layers layer by layer. A unified fusion feature map is formed.

[0048] First, initialize the lowest-level features:

[0049]

[0050] Then, the fusion operation is performed layer by layer in sequence:

[0051]

[0052] The fusion function ψ(·) consists of an upsampling module (at uniform resolution), feature concatenation, and convolutional units, exhibiting parameter sharing to ensure consistency between the training and inference phases. The final output is a fused feature map. As a centralized representation of the normal image feature space.

[0053] Step 204: To construct a compact representation of the normal feature distribution that can be used for comparison, this step will fuse the feature maps: Memory construction C L Unfold along the spatial dimension to obtain patch feature representations at all image locations:

[0054]

[0055] The above set covers the local features of all spatial regions. Then, the K-means clustering method is used to compress and model it, forming a representative set of class centers. The clustering objective function is as follows:

[0056]

[0057] in, γ c This is a hyperparameter for controlling the clustering granularity. The clustering results constitute a compact representation of the normal image feature distribution, which is the memory.

[0058] Step 205: Test Image Feature Extraction and Fusion

[0059] For the silicon wafer image X to be detected, the present invention uses the same steps as the memory bank construction (steps 201 to 203) to obtain the fused patch feature set P. L And flatten it into a sequence of patch vectors:

[0060]

[0061] Each patch in this set represents a semantic embedding feature at a spatial location in the image under test.

[0062] Step 206: Comparative Supervised Learning Mechanism

[0063] In unsupervised defect detection, traditional supervised classification methods are difficult to apply directly due to the lack of labeled samples. Therefore, this invention proposes a hybrid boundary contrastive learning mechanism to optimize the topological structure of the feature embedding space, making patch features in normal regions more tightly clustered, while abnormal regions are moved away from the normal distribution center, thus forming discriminative abnormal boundaries.

[0064] This mechanism revolves around the idea of ​​"positive sample attraction" and "negative sample repulsion." Specifically, it targets the patch features p in each image to be tested. ′ i, from memory Retrieve several feature centers with the highest similarity to the positive sample and construct a positive sample set. Furthermore, a certain number of negative samples are selected from distant but still confusing centers to construct a negative sample set.

[0065] Positive sample construction logic: Positive sample set Includes p′ i The nearest K pos There are several memory centers. These centers typically originate from normal regions, therefore the patch to be tested should be positioned close to them to form a compact normal class distribution. The loss function for this part is designed as follows:

[0066]

[0067] Where, c′ j,+ is the memory center of the j-th positive sample; d(·,·) represents the Euclidean distance function; r is the hypersphere radius, a learnable parameter; v is the normalization factor, equal to the total number of patches; max(·,0) represents the hinge loss structure, which incurs loss only when the distance exceeds the radius. The core function of this loss term is to construct an "attraction constraint": a patch feature is only penalized when its distance from the positive sample center exceeds the allowable radius r, thereby prompting the patch embeddings in the normal region to remain sufficiently compact.

[0068] Negative sample construction logic: negative sample set Then it includes p ′ i is at a distance from K in the middle interval neg There are several memory centers. These "pseudo-similar but fundamentally different" centers constitute potential confusion boundaries. To prevent aberrant patches from approaching normal centers, this invention designs a separation loss term as follows:

[0069]

[0070] Where, c′ j,― Let represent the cluster center of the j-th negative sample; α is the interval control hyperparameter; this loss term constructs a "repulsion constraint": if an abnormal patch is too close to the center of a negative sample, it may be misclassified as a normal region, at which point a positive gradient is generated, pushing it away from the boundary.

[0071] Total Loss Target

[0072] The final contrastive loss function is a weighted combination of the two items mentioned above:

[0073]

[0074] This process simultaneously enables the aggregation of patches within the "hypersphere constructed by the trusted normal clustering center" and the formation of isolation between the "hyperspheres of the critical confusion center", thereby effectively constructing implicit classification boundaries in the embedding space in unsupervised scenarios.

[0075] Step 207: Anomaly Score Generation and Heatmap Output

[0076] Finally, during the testing phase, for each spatial location (x, y), the patch feature p can be used to determine the optimal location. ′ Calculate the anomaly score based on the nearest distance between (x,y) and the center of the memory bank:

[0077]

[0078] The above rating chart A score Composed of scores for all pixels, it reflects the degree to which each region deviates from normal feature patterns. A higher score indicates that the region is more likely to be an anomaly.

[0079] By interpolating and normalizing the anomaly scoring map, a final visual heatmap can be generated, which intuitively presents the location of defects such as scratches, cracks, contamination, or dents, enabling automated location.

[0080] Example 3:

[0081] A parallel feature processing device for unsupervised defect detection on silicon wafer surfaces includes a processor, a memory, and an image input interface. The memory stores program instructions executable by the processor, which, when executing these instructions, implements the following functional modules:

[0082] To address potential structural defects in the input image, this device first extracts multi-scale features from the image using a feature extraction module. This module receives the input silicon wafer image X. i A pre-trained convolutional neural network is used to extract feature maps of different scales from multiple layers. After pooling and normalization, a set of multi-scale semantic representations is output. Used to construct cross-scale information fusion structures.

[0083]

[0084] The extracted feature maps retain multi-scale semantic information of the silicon wafer surface, from local texture to global structure, which facilitates accurate modeling of defects in the future.

[0085] Subsequently, the device enters the feature fusion processing stage. The fusion module is responsible for aggregating standardized features from different levels using a bottom-up strategy to generate a fused feature map C. L This feature map contains rich local texture and global structural information. The fusion process initializes with the sample mean and completes semantic completion at each level through a unified structure. The aggregation process can be simplified as follows:

[0086]

[0087] After obtaining the fused feature map, the device enters the memory construction phase. This phase will... LExpand into a patch feature set, and generate a compact set of normal sample feature centers through unsupervised clustering (such as K-means). That is, the memory bank:

[0088]

[0089] The established It represents the distribution center of normal structures in the feature space in the training samples, and is used for subsequent comparison and supervision.

[0090] Next, the device enters the supervised feature matching and comparison phase. This module is used to perform supervised regulation of the embedding space. In subsequent processes, the images X to be trained and tested undergo the same extraction and fusion process to obtain the fused feature map P. L The module uses a memory. For reference, the corresponding positive and negative sample centers are retrieved, and the features are aggregated and separated. The loss function during the training phase is:

[0091]

[0092] It includes a positive sample aggregation and pull mechanism and a negative sample separation and repulsion mechanism to enhance the structural constraints of the embedding space.

[0093] Repeat the above steps to gradually optimize the model. In each iteration, the model adjusts its parameters based on the loss function value, gradually reducing the difference between the predicted results and the true labels. During training, the model is also evaluated using a validation set to ensure its generalization ability. After training is complete, the model is finally evaluated using a test set to measure its performance in real-world applications.

[0094] Finally, during the inference phase, this module also performs feature matching with the memory for anomaly score generation. The device then enters the defect output module. This module receives the fused features P from the test image. L and connect it with the memory bank center Distances between pixels are compared, and an anomaly score map is generated based on the minimum Euclidean distance. Each pixel or patch position (x, y) corresponds to a score value, representing the degree of deviation of its features from the normal template. A deviation exceeding a set hypersphere radius r is considered an anomaly. The scoring function is expressed as:

[0095]

[0096] Rating Chart A score After upsampling and interpolation, the output is visualized to form an intuitive defect heatmap, indicating the location and intensity of potential abnormal areas such as cracks, scratches, and contamination on the silicon wafer surface.

[0097] Unless otherwise specified, the model numbers of the various devices in this embodiment of the invention are not limited, and any device that can perform the above functions is acceptable.

Claims

1. An unsupervised silicon wafer surface defect detection method based on hierarchical clustering, characterized in that, Includes the following steps: S1 extracts multi-scale features from multiple normal silicon wafer reference images through a pre-trained convolutional neural network, and integrates features of different scales using a hierarchical progressive fusion strategy to generate a representative normal reference feature representation, fully capturing the semantic information of the silicon wafer surface from local texture to global structure. S2 utilizes a clustering algorithm to optimize normal reference features. By analyzing the distribution characteristics in the feature space, a normal silicon wafer feature memory library is constructed, generating compact and representative cluster centers to represent typical patterns of normal silicon wafer features. S3 performs the same multi-scale feature extraction and fusion operation as the normal reference image on the single silicon wafer image to be detected, and obtains the patch-level feature representation of the image at each spatial location, preserving the integrity of local and global information; S4, based on the contrastive supervised learning method, calculates the distance relationship between patch features and cluster centers in the memory, so that normal patch features are closer to the corresponding cluster centers, while abnormal patch features are pushed away, generating a defect scoring map that reflects the degree of abnormality. S5 maps the position of abnormal patches in the image space based on the scores of the abnormal patches in the defect scoring map, and generates a visualized defect map for accurately locating abnormal areas such as scratches, cracks, contamination or dents on the silicon wafer surface.

2. The unsupervised silicon wafer surface defect detection method based on hierarchical clustering according to claim 1, characterized in that, Using a pre-trained deep convolutional neural network WRN50-2, feature maps of multiple levels are extracted from multiple normal silicon wafer reference images to capture low-level texture information, mid-level edge information, and high-level semantic information. Spatial average pooling is performed on each level of feature map to compress redundant information in the spatial dimension and reduce interference from non-defect factors such as illumination or reflection. A lightweight convolutional module is then used to refine the structure and unify the dimensions of the pooled feature maps to generate normalized feature maps. F l =Conv(Pool(WRN50_2 l (I))) By employing a bottom-up, hierarchical, progressive fusion strategy, feature maps from different levels are integrated layer by layer to generate a unified normal reference feature representation. Ensure semantic consistency and stability of features across multiple scales.

3. The unsupervised silicon wafer surface defect detection method based on hierarchical clustering according to claim 1, characterized in that, The fused feature map generated in step S1 is expanded along the spatial dimension to obtain a set of patch feature vectors for all spatial locations, covering local and global feature information of the silicon wafer surface. The K-means clustering algorithm is used to perform unsupervised clustering analysis on the patch feature vector set. By minimizing the distance from feature points to cluster centers, a set of compact and representative cluster centers is generated. The objective function is: Where C′={c′1,…,c′ K }, K represents the initial set of cluster centers, and K is the total number of cluster centers. The generated cluster centers are stored as a normal silicon wafer feature memory for subsequent comparison of features and anomaly detection of the image to be detected.

4. The unsupervised silicon wafer surface defect detection method based on hierarchical clustering according to claim 1, characterized in that, The same multi-scale feature extraction process as in step S1 is performed on the single silicon wafer image to be inspected, and multi-level feature maps are generated using the WRN50-2 network. The same hierarchical progressive fusion strategy as the normal reference image is adopted to integrate the multi-level feature maps into a unified fusion feature map. The fusion feature map is expanded along the spatial dimension to generate a set of patch feature vectors of the image to be inspected at each spatial location, which are used for subsequent anomaly scoring and defect localization.

5. The unsupervised silicon wafer surface defect detection method based on hierarchical clustering according to claim 1, characterized in that, For each patch feature vector in the image to be detected, several cluster centers with the closest Euclidean distance to it are retrieved from the normal silicon wafer feature memory to form a positive sample set, representing the feature pattern of the normal region; several cluster centers with distances between them and the patch features in the middle range are selected from the memory to form a negative sample set, representing the feature pattern of potential confusion; the feature embedding space is optimized through a positive sample attraction mechanism and a negative sample repulsion mechanism, so that normal patch features are close to the positive sample cluster centers and abnormal patch features are far away from the negative sample cluster centers. The loss function is: Where, p′ i Let c′ be the feature vector of the i-th patch. j,+ and c′ j,- These are the cluster centers for positive and negative samples, respectively, where r is the hypersphere radius, α is the margin control parameter, and K is the cluster center for positive and negative samples. p and K n The numbers of positive and negative samples are respectively; based on the distance relationship between the patch features and the cluster center, the anomaly score of each spatial location is calculated to generate a defect score map reflecting the degree of anomaly on the silicon wafer surface.

6. The unsupervised silicon wafer surface defect detection method based on hierarchical clustering according to claim 1, characterized in that, Based on the defect scoring map generated in step S4, calculate the minimum Euclidean distance between each patch feature and the cluster center of the memory bank to obtain the anomaly score: The anomaly scoring map is upsampled and normalized, and the scoring values ​​are mapped to the image space to generate a visualized defect heatmap. By analyzing high-scoring areas in the heatmap, abnormal areas such as scratches, cracks, contamination, or dents on the silicon wafer surface can be accurately located, and intuitive defect location information can be output.

7. An unsupervised silicon wafer surface defect detection device based on hierarchical clustering, characterized in that, This device achieves unsupervised defect detection by integrating multiple functional modules, and is suitable for silicon wafer surface quality control in semiconductor manufacturing. These modules include: a multi-scale feature extraction module, a progressive update module, a hierarchical fusion module, a contrastive supervised learning module, and a defect localization module.

8. An unsupervised silicon wafer surface defect detection device based on hierarchical clustering, characterized in that, It includes one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to initiate the method steps and apparatus as described in any one of claims 1 to 7, for achieving efficient and robust detection of silicon wafer surface defects.

Images