Wafer defect recognition method and device based on feature fusion, equipment and medium

Abstract

The application relates to the fields of artificial intelligence technology and semiconductor detection technology and discloses a wafer defect recognition method based on feature fusion, which comprises the following steps: collecting a first type image and a second type image of a wafer detection area and performing alignment processing; performing feature extraction on the first type image and the second type image to generate first type features and second type features; fusing the first type features and the second type features to generate fusion features; performing filtering processing on the fusion features to screen target fusion features meeting a confidence threshold condition; and determining defect positions and corresponding process layer information based on the target fusion features. Through the fusion of features of different types of images, the wafer surface defect characterization capability is enhanced, and the recognition accuracy of the detection system is improved; through the screening of the target fusion features meeting the confidence threshold, the false detection and the missed detection are reduced, and the detection reliability is improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence technology and semiconductor inspection technology, and in particular to a method, apparatus, device and storage medium for wafer defect identification based on feature fusion. Background Technology

[0002] With the continuous evolution of semiconductor manufacturing processes, the requirements for the accuracy and real-time performance of wafer surface defect detection are increasing. However, existing detection technologies mainly rely on single imaging modes, such as optical inspection (OI), scanning electron microscopy (SEM), or transmission electron microscopy (TEM), which often have limitations when dealing with complex defect morphologies. For example, optical inspection is limited by resolution and has insufficient ability to detect nanoscale defects, while electron microscopy, although having higher resolution, has a limited detection range, making it difficult to meet the needs of high-throughput production environments.

[0003] Wafer manufacturing involves multiple processes (such as photolithography, etching, and deposition), and defects can occur between different process layers. Existing detection methods struggle to effectively distinguish the specific process layer where a defect is located. Especially at the boundaries of multilayer structures, defects may have similar material properties to those of the layers above and below, making it difficult for detection systems to accurately determine the source of defects and affecting the accuracy of process optimization and fault tracing.

[0004] Although existing methods have attempted to combine multimodal data for joint detection, current multimodal data fusion methods are relatively simple, mainly relying on image stitching or separate analysis, failing to fully utilize the complementary information of different modalities. This data processing approach limits the accuracy of defect identification, especially when identifying complex defects, and may increase the risk of false positives and false negatives due to incomplete information or insufficient feature representation.

[0005] Furthermore, high-end semiconductor manufacturing places higher demands on the real-time performance of inspection systems. However, existing high-precision inspection methods are computationally intensive and slow in processing, making it difficult to balance inspection accuracy and production efficiency. In actual production lines, inspection systems often face insufficient data processing capabilities, leading to delays in defect identification, affecting real-time adjustments to the production process, and consequently increasing manufacturing costs and product defect rates. Summary of the Invention

[0006] The main objective of this invention is to provide a wafer defect identification method, apparatus, device, and storage medium based on feature fusion, aiming to solve the technical problems in the existing wafer defect detection process, such as the single multimodal data fusion method, insufficient defect layer identification capability, resulting in low identification accuracy of complex defects, high false detection and false negative rates, and difficulty in accurately determining the process layer where the defect is located.

[0007] To achieve the above objectives, the present invention provides a wafer defect identification method based on feature fusion, comprising:

[0008] Acquire a first type of image and a second type of image of the wafer inspection area, and perform alignment processing on the first type of image and the second type of image;

[0009] Feature extraction is performed on the first type of image and the second type of image respectively to generate first type features and second type features;

[0010] The first type of feature and the second type of feature are combined to generate a fused feature;

[0011] The fused features are filtered, and target fused features that meet the confidence threshold conditions are selected.

[0012] Based on the target fusion features, the defect location and corresponding process layer information are determined.

[0013] Furthermore, to achieve the above objectives, the present invention provides a wafer defect identification device based on feature fusion, comprising:

[0014] The image acquisition and alignment module is used to acquire a first type of image and a second type of image of the wafer inspection area, and to perform alignment processing on the first type of image and the second type of image;

[0015] The feature extraction module is used to extract features from the first type of image and the second type of image respectively, and generate first type features and second type features;

[0016] The feature fusion module is used to fuse the first type of feature and the second type of feature to generate a fused feature;

[0017] The feature selection and filtering module is used to perform filtering processing on the fused features and select target fused features that meet the confidence threshold conditions.

[0018] The defect location and process layer identification module is used to determine the defect location and corresponding process layer information based on the target fusion features.

[0019] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a feature fusion-based wafer defect identification program stored in the memory and executable on the processor, wherein when the feature fusion-based wafer defect identification program is executed by the processor, it implements the steps of the feature fusion-based wafer defect identification method as described above.

[0020] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a wafer defect identification program based on feature fusion, wherein the wafer defect identification program based on feature fusion, when executed by a processor, implements the steps of the wafer defect identification method based on feature fusion as described above.

[0021] Beneficial Effects: This invention relates to the fields of artificial intelligence and semiconductor inspection technology, and discloses a wafer defect identification method based on feature fusion. The method includes: acquiring a first type of image and a second type of image of the wafer inspection area and performing alignment processing; extracting features from the first type of image and the second type of image to generate first type features and second type features; fusing the first type features and the second type features to generate fused features; performing filtering processing on the fused features to select target fused features that meet a confidence threshold condition; and determining the defect location and corresponding process layer information based on the target fused features. This invention enhances the characterization ability of wafer surface defects and improves the identification accuracy of the inspection system by fusing features from different types of images; reduces false detections and missed detections and improves inspection reliability by selecting target fused features that meet a confidence threshold; accurately determines the process layer where the defect is located through hierarchical identification based on target fused features, achieving precise defect localization; and, combined with multimodal feature analysis, enables the inspection system to adapt to different defect types, improving quality control capabilities in the wafer manufacturing process. Attached Figure Description

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0023] Figure 1 This is a schematic diagram of an application environment for a wafer defect identification method based on feature fusion in one embodiment of the present invention;

[0024] Figure 2 This is a flowchart illustrating an embodiment of the wafer defect identification method based on feature fusion according to the present invention.

[0025] Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the wafer defect identification device based on feature fusion of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0027] Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0028] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0029] The wafer defect identification method based on feature fusion provided in this invention can be applied to, for example... Figure 1 In this application environment, the user terminal communicates with the server via a network. The server can acquire first-type and second-type images of the wafer inspection area through the user terminal and perform alignment processing; extract features from the first-type and second-type images to generate first-type features and second-type features; fuse the first-type and second-type features to generate fused features; perform filtering processing on the fused features to select target fused features that meet the confidence threshold conditions; and determine the defect location and corresponding process layer information based on the target fused features. This invention enhances the characterization ability of wafer surface defects and improves the recognition accuracy of the inspection system by fusing features from different types of images; reduces false detections and missed detections and improves detection reliability by selecting target fused features that meet the confidence threshold; accurately determines the process layer where the defect is located through hierarchical recognition based on the target fused features, achieving precise defect localization; and, combined with multimodal feature analysis, enables the inspection system to adapt to different defect types, improving the quality control capability in the wafer manufacturing process. The user terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0030] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the wafer defect identification method based on feature fusion provided by the present invention. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0031] like Figure 2 As shown, the wafer defect identification method based on feature fusion proposed in this invention includes the following steps:

[0032] S10, acquire a first type image and a second type image of the wafer inspection area, and perform alignment processing on the first type image and the second type image;

[0033] In this embodiment, image acquisition of the wafer inspection area is the foundation for multimodal defect identification. During the wafer manufacturing process, defects such as particulate contamination, microcracks, and interlayer fractures may exist on the surface. The detection of these defects usually requires the coordinated use of multiple imaging modes. The first type of image and the second type of image typically refer to two different types of electron microscopy imaging data, such as transmission electron microscopy (TEM) and scanning transmission electron microscopy (STEM).

[0034] The wafer surface contains multiple functional areas, and it is usually necessary to select key areas for inspection, such as critical pattern areas, solder joint areas, or high-stress areas. Inspection points can be precisely selected by pre-setting inspection coordinates or by combining an autofocus system.

[0035] Transmission electron microscopy (TEM) is used to acquire transmission images to observe the internal structure of the wafer, including defects in the deposited and etched layers. Scanning transmission electron microscopy (STEM) is used to acquire scanning transmission images to capture surface morphology features, and annular dark-field imaging is used to enhance the contrast of specific defects.

[0036] Parameters such as the electron beam accelerating voltage, beam current intensity, and working distance need to be optimized based on material properties and defect types. In transmission mode, sample thickness and incident angle need to be adjusted to ensure optimal imaging results. In scanning transmission mode, detector angle and imaging magnification need to be optimized to enhance contrast and tonal information.

[0037] A high-precision motion platform, combined with a computer control system, precisely adjusts the sample position to ensure the electron beam irradiates the target area, preventing detection area shift. Autofocus and image enhancement algorithms are integrated to reduce focal length drift caused by sample surface inhomogeneity, improving image clarity. In multi-exposure mode, beam intensity and exposure time are adjusted to balance high signal-to-noise ratio and detection efficiency.

[0038] Because the electron beam path and detector position may differ in different imaging modes, differences in the image coordinate system may occur, necessitating spatial error compensation. A feature point matching algorithm is used to calculate the offset between images based on wafer surface markers or key structural points.

[0039] Affine transformations, perspective transformations, or non-rigid transformations are used to adjust images from different modalities to the same coordinate system. Combined with camera calibration parameters, distortions introduced by different detectors are corrected to ensure accurate spatial correspondence.

[0040] TEM typically provides high-resolution details, while STEM provides overall topographic information. Alignment is required at different resolutions to avoid contrast failure due to scale differences. A pyramid matching algorithm is employed to match feature points at multiple scales, progressively adjusting the alignment accuracy of the two images.

[0041] Image feature matching alignment: Extract corner features (such as SIFT, ORB, Harris), calculate key point offset, and achieve accurate alignment.

[0042] Alignment based on physical model: Based on the optical path parameters of the electro-optical system, an image distortion correction model is established, and deformation error is eliminated through mathematical transformation.

[0043] Alignment based on deep learning: Train a neural network model to automatically learn the correspondence between different imaging modes and achieve end-to-end alignment processing.

[0044] Multimodal image acquisition and alignment enhance the collaborative capabilities between different detection modes, enabling more accurate defect identification. The use of a high-precision motion platform and electron beam imaging optimization improves image spatial resolution and contrast. Feature point matching and geometric transformation eliminate spatial errors between different modal data, enhancing defect comparability and identification accuracy, and enabling the detection system to perform more accurate defect layering analysis.

[0045] S20, feature extraction is performed on the first type image and the second type image respectively to generate first type features and second type features;

[0046] In this embodiment, the aligned first-type and second-type images contain defect information at different levels, requiring separate feature extraction for subsequent feature fusion and defect analysis. Transmission electron microscopy (TEM) is primarily used to acquire the internal structural features of wafer materials, while scanning transmission electron microscopy (STEM) is mainly used to extract surface morphology and hierarchical features. In the preceding data acquisition, the wafer inspection area has been located using a high-precision motion platform, and the imaging system parameters have been optimized to achieve high contrast and resolution in the acquired images. Based on this high-quality image data, an independent feature extraction network needs to be constructed to extract key defect information from different modalities and ensure that subsequent feature fusion fully utilizes the complementarity of the two types of images.

[0047] To extract internal structural information and defect texture features from the first type of image, a transmission feature extraction network was constructed. This network employs an improved convolutional neural network (CNN), using residual connection modules to enhance the stability of deep feature extraction and improve the ability to identify low-contrast defects. Simultaneously, dilated convolution modules are introduced to expand the receptive field, thereby capturing defect distribution information over a larger area and ensuring that feature extraction is not affected by the loss of information in local areas. Furthermore, to accommodate defect detection at different scales, a feature pyramid network (FPN) is used, enabling the effective extraction of both small and large defects and improving the expressive power of features at different levels. In the feature extraction process for the second type of image, a scanning transmission feature extraction network was constructed. Multi-scale convolution modules enhance the ability to capture defect morphology information, while a contrast enhancement module is combined to make weak-contrast defect regions more prominently represented, improving the model's ability to distinguish between different types of defects. For defects with complex hierarchical structures, structural tensor analysis is employed to maintain the integrity of defect edges and optimize the stability of feature representation, ensuring that the complementary information of these two types of features can be fully utilized in the subsequent feature fusion stage.

[0048] When constructing a deep learning-based feature extraction network, various optimization strategies can be employed. For example, an attention mechanism can be incorporated into the transmission feature extraction network to enable the model to automatically focus on defect regions and reduce the interference of background noise on feature extraction. Simultaneously, a multi-scale fusion module can be introduced into the scanning transmission feature extraction network to enhance the model's adaptability to defects of different scales. During training, supervised learning is performed using high-confidence defect data, and self-supervised learning methods are used to enhance the model's generalization ability to new defect types. Furthermore, feature enhancement strategies can be combined, such as random rotation, scale transformation, and contrast adjustment, to expand the training data and improve the model's robustness.

[0049] In one specific implementation, a deep residual network (ResNet) is used as the base model in the convolutional neural network structure, and the feature representation capability of the model is improved by stacking multiple residual blocks. During feature extraction, the first few convolutional layers are used to extract basic texture information, while the subsequent deep network is used to learn more advanced feature patterns. During training, a class-balanced loss function is used to address the problem of uneven sample distribution between different defect categories and improve the recognition ability of small sample categories.

[0050] In another specific implementation, for the multi-scale defect detection problem, a method combining dilated convolution with a pyramid feature extraction network (FPN) is used to ensure that the model can effectively extract defect information at different scales. By setting receptive fields of different scales, both small and large defects can be represented in a unified feature space, and are not affected by the input image resolution.

[0051] By constructing independent transmission feature extraction networks and scanning transmission feature extraction networks, accurate extraction of defect information from both types of images was achieved. Introducing multi-scale feature extraction methods improved the model's adaptability to defects of different sizes. Furthermore, combining optimization strategies such as residual connectivity and structural tensor analysis enhanced the detection accuracy of low-contrast defects and defects with complex hierarchical structures. Ultimately, this enabled the system to more accurately acquire information on wafer surface and internal defects, providing more stable input data for subsequent feature fusion and defect classification.

[0052] S30, merge the first type of feature and the second type of feature to generate a fused feature;

[0053] In this embodiment, after feature extraction, the first type of features and the second type of features respectively contain defect information from transmission electron microscopy (TEM) and scanning transmission electron microscopy (STEM) images. Since the two types of features originate from different sources and are complementary in their information representation, feature fusion is necessary to ensure that the final defect detection model can fully utilize the advantages of both types of data, improving recognition accuracy and stability. In the preceding steps, these two types of features have been extracted using different feature extraction networks, ensuring consistency in feature numerical distribution and scale to avoid information loss or interference due to data format mismatch during the fusion process.

[0054] The first step in the fusion process is to align the spatial relationships and channel dimensions of different features, ensuring that the first and second types of features can be computed within the same feature space. Since they may have different scales, orientations, or numerical ranges, feature standardization is necessary to maintain data consistency. Simultaneously, to avoid information redundancy or feature loss from simple concatenation, the two types of features are initially fused using a channel concatenation method to form initial fused features. To further optimize feature representation, a channel attention mechanism is used to calculate the importance weights of the first and second types of features, and an adaptive weighting method is used to adjust the contribution of different channels. Addressing the issues of feature complementarity and redundant information, a distance metric module is used to calculate the channel similarity matrix between the first and second types of features, ensuring that the most discriminative feature channels are selected during the fusion process. Furthermore, element-wise multiplication is used to weight and fuse the first and second types of features, preserving effective information from both types of features, and nonlinear transformations are used to optimize the final feature representation, improving the model's ability to identify complex defects.

[0055] Feature fusion can be achieved through combinations of different approaches, including channel concatenation, adaptive weight calculation, and nonlinear transformation optimization. Channel concatenation-based fusion is suitable for scenarios with high feature space consistency, offering low computational complexity. It allows for direct feature combination via channel stacking and further optimizes feature relationships through subsequent convolutional operations. For situations with significant information differences between features of different modalities, attention-based fusion can be employed. This involves calculating feature weights using channel attention and dynamically adjusting the contribution of different channels during feature fusion to improve the model's ability to focus on key features. Furthermore, in scenarios with limited data or constantly emerging new defect types, self-supervised learning methods can be combined. Contrastive learning or variational autoencoders (VAEs) can optimize the feature fusion process, enabling the model to maintain good generalization ability even with limited labeled data. For computationally constrained devices, such as embedded systems or real-time detection tasks, lightweight network structures can be used, employing depthwise separable convolution to reduce computation while maintaining the effectiveness of feature fusion.

[0056] Example Explanation: Fusion Detection of Low-Contrast Defects During wafer inspection, some defect areas exhibit low contrast, making it difficult to accurately extract features using a single imaging mode. By fusing the deep structural information from TEM images with the high-contrast morphological features from STEM images, the visibility of low-contrast defects can be effectively enhanced, improving detection accuracy.

[0057] By fusing first-type and second-type features, the model's adaptability to multimodal data is improved, enabling features from different imaging modes to complement each other and enhancing defect detection accuracy. Channel stitching and adaptive weight calculation allow the model to dynamically adjust the contribution of different modalities, avoiding information loss or interference from redundant features in the final classification results. A nonlinear activation optimization strategy further enhances the expressive power of the fused features, enabling the defect detection system to handle different types of defects more stably and improving quality control capabilities in the wafer manufacturing process.

[0058] S40, perform filtering on the fusion features and select target fusion features that meet the confidence threshold conditions;

[0059] In this embodiment, after fusing the first type of features and the second type of features, the fused features contain key information from both modalities, but may still contain redundant features, noise, or low-confidence samples. To improve the accuracy of defect identification, the fused features need to be filtered to remove low-confidence features and retain only the target fused features that meet the confidence threshold, ensuring the reliability of subsequent defect localization and hierarchical identification. The preceding steps have completed feature extraction and fusion, and optimized the feature representation through attention mechanisms and nonlinear transformations. Based on this, further feature filtering is needed so that the final model only utilizes high-confidence, valid data.

[0060] A deep learning model is used to classify the fused features, and the predicted probability distribution for each defect category is calculated. The predicted probability reflects the model's confidence in each category. A maximum probability method or a Bayesian uncertainty estimation method is used to calculate the confidence score for each fused feature, serving as an indicator of feature reliability.

[0061] The confidence scoring mechanism quantifies the reliability of fused features, ensuring that only data with high confidence is used for defect analysis. Confidence scoring can be based on uncertainty measures such as maximum probability, information entropy, or Markov Monte Carlo (MCMC) sampling estimation to ensure that the scoring results accurately reflect feature quality.

[0062] A confidence threshold is set to filter the fused features, removing those below the threshold to reduce the impact of invalid data on model decisions. An adaptive threshold adjustment mechanism is employed, allowing the threshold to be dynamically optimized according to different process requirements to adapt to the detection requirements of different production environments.

[0063] After screening, fusion features with confidence scores above a set threshold are retained as the final target fusion features. These target fusion features are used for subsequent defect localization and process-level analysis, ensuring that only high-confidence data are used for the final decision.

[0064] Feature filtering and selection can be achieved in various ways. Deep learning-based methods can calculate confidence scores using predicted probability distributions and then select features by setting static or adaptive thresholds. This approach is suitable for applications with high data quality requirements and can dynamically adjust thresholds to adapt to different process requirements. Furthermore, adversarial sample detection can be combined to filter out anomalous samples and remove low-confidence data by judging whether features have significant uncertainty. If computational resources are limited, rule-based confidence selection methods can be used, setting fixed thresholds to select features and improving computational efficiency. For online detection systems, time series analysis can also be combined to statistically analyze historical detection data and dynamically adjust confidence thresholds to optimize real-time detection performance.

[0065] By calculating the confidence score of fused features and setting a threshold to filter target fused features, the reliability of defect identification is improved. An adaptive threshold adjustment mechanism is introduced, enabling the system to dynamically optimize the screening criteria according to process requirements, ensuring stable detection performance under different production environments. Combined with a deep learning-based prediction probability calculation method, high-confidence features can be screened more accurately, improving the accuracy of final defect localization and hierarchical identification.

[0066] S50, determine the defect location and corresponding process layer information based on the target fusion features.

[0067] In this embodiment, after screening the target fusion features, these high-confidence features are used for defect localization and process layer analysis. The target fusion features contain spatial, morphological, and hierarchical information about the defects, thus enabling precise determination of their physical location on the wafer and further analysis of their corresponding process layers. The preceding steps have ensured the quality of the target fusion features; the core of this current step is to utilize these features for spatial mapping, process layer matching, and defect localization to form a complete defect information description.

[0068] The pre-stored process layer template contains structural feature parameters of each process layer on the wafer, providing typical features for each layer. Through feature similarity calculation, the target fused features are matched with the process layer template to determine the process layer where the defect resides. Using a feature space mapping method, the target fused features are projected onto the process layer feature space, a similarity score is calculated, and the process layer with the highest score is selected as the defect's assigned layer.

[0069] The target fusion features contain the location coordinates of defects in image space, but these coordinates need to be converted into physical coordinates of the wafer inspection area for tracking and adjustment in the production management system. A coordinate transformation model is used to map the image coordinates to the actual coordinate system of the wafer inspection area. Through distortion correction and scale normalization, it is ensured that the transformed coordinates accurately reflect the location of the defects on the wafer.

[0070] After process layer matching and coordinate transformation, the defect location coordinates are bound to the corresponding process layer number to form a complete defect information description. A data table relating defect location to process layer is generated, recording the specific location of the defect, the corresponding process layer number, and other related information. This information is stored using a standardized data format to ensure compatibility across different systems and its use in production management and quality control.

[0071] Defect location and process layer determination can be achieved using template matching, feature mapping, or machine learning methods. Template matching relies on pre-stored process layer templates and is suitable for scenarios with known process flows, enabling rapid matching of defects with process layers. Feature mapping projects target fused features into the process layer feature space to calculate a similarity score and determine the best-matching process layer, suitable for situations where there are subtle differences between process layers. For complex defect localization tasks, machine learning methods can be combined, using labeled defect data to train a classification model to predict the process layer to which the defect belongs, and a regression model can be used to optimize the accuracy of defect coordinate transformation. Furthermore, in terms of data storage and system integration, standardized interface protocols, such as JSON or XML formats, can be used to enable efficient parsing and application of defect information by automated production management systems (MES) and quality control systems (QMS).

[0072] Example: During semiconductor manufacturing, various defects may occur on the wafer surface, such as particle contamination, scratches, cracks, and interlayer fractures. These defects may originate from processes such as photolithography, etching, and thin film deposition, affecting chip yield. Therefore, manufacturers need a high-precision, real-time, layer-specific defect detection system to accurately identify the location, type, and process layer affiliation of defects, and transmit the detection results to the Manufacturing Execution System (MES) to adjust process parameters and improve product yield.

[0073] After photolithography and etching processes, the wafer enters the defect inspection process. The automated inspection equipment uses two imaging modes—transmission electron microscopy (TEM) and scanning transmission electron microscopy (STEM)—to scan the wafer surface and internal layers.

[0074] Transmission mode (TEM): Acquires structural information inside the wafer material and detects tiny particles and cracks.

[0075] Scanning Transmission Mode (STEM): Acquires layer information on the wafer surface and extracts morphological features of different process layers.

[0076] To ensure image consistency, the system uses a high-precision motion platform to position the wafer and perform image alignment and distortion correction, so that subsequent analysis is based on a unified coordinate system.

[0077] Transmission imaging (TEM): An improved CNN model is used, combining residual connection modules and dilated convolution modules, to extract information on particle defects and cracks inside the material.

[0078] Scanning Transmission Imaging (STEM): Utilizing multi-scale convolution modules and contrast enhancement modules, surface morphology features are extracted to distinguish defects in different process layers.

[0079] To improve the accuracy of defect detection, the system employs a feature fusion algorithm: first, transmission features and scanning transmission features are concatenated along the channel dimension to generate initial fused features. An attention mechanism is then used to calculate the weight of each feature, improving the model's ability to distinguish defect types. Finally, a distance metric module is used to analyze the similarity between different channels, eliminating redundant features and generating the final fused features.

[0080] The predicted probability distribution of each fusion feature is calculated using a defect classification model to obtain the confidence level for each defect category. A threshold is set, for example, samples with a confidence level below 95% are filtered out to ensure the reliability of subsequent detection results. Only target fusion features with high confidence are retained for the next step of analysis.

[0081] The target fusion features are matched with the process layer template to determine the process layer number where the defect is located. Using the physical coordinate transformation model of the wafer inspection area, the defect detection coordinates are converted into actual physical coordinates (e.g., millimeter-level precision). A table relating the defect location to the process layer is generated, recording the defect's spatial location, process layer affiliation, and confidence level information.

[0082] A color-coded method is used to mark defect information on the wafer surface image: photolithography layer defects are marked with a blue outline; etched layer defects are marked with a red outline; and deposited layer defects are marked with a green outline.

[0083] The generated layered inspection result map can be viewed in real time through the production management system (MES), and supports interactive zooming, filtering, and classification of defect information.

[0084] By matching target fusion features with process layer templates, the specific process layer of the defect is determined. Combined with coordinate transformation technology, the physical coordinates of the defect within the wafer inspection area are precisely located. The process layer feature mapping method improves the accuracy of defect layering and ensures that defect information can be standardized for storage and application, providing reliable data support for subsequent production management and quality control.

[0085] This invention relates to the fields of artificial intelligence and semiconductor inspection technology, and discloses a wafer defect identification method based on feature fusion. The method includes: acquiring a first type of image and a second type of image of the wafer inspection area and performing alignment processing; extracting features from the first type of image and the second type of image to generate first type features and second type features; fusing the first type features and the second type features to generate fused features; performing filtering processing on the fused features to select target fused features that meet a confidence threshold condition; and determining the defect location and corresponding process layer information based on the target fused features. This invention enhances the characterization ability of wafer surface defects and improves the identification accuracy of the inspection system by fusing features from different types of images; reduces false detections and missed detections and improves inspection reliability by selecting target fused features that meet a confidence threshold; accurately determines the process layer where the defect is located through hierarchical identification based on target fused features, achieving precise defect localization; and, combined with multimodal feature analysis, enables the inspection system to adapt to different defect types, improving quality control capabilities in the wafer manufacturing process.

[0086] In one embodiment, S10 includes:

[0087] S101, the wafer inspection area is located by the motion platform, and the coordinate data of the wafer inspection area is generated;

[0088] S102, Adjust the electron beam focal length of the electron optics module and set the operating parameters of the transmission mode in the electron optics module;

[0089] S103, Based on the transmission mode of the electro-optical module, acquire multiple sets of first-type images from different angles;

[0090] S104, Based on the scanning transmission mode of the electron optics module, a second type of image of the annular dark field is acquired, and distortion correction processing is performed on the second type of image based on the coordinate data of the wafer detection area;

[0091] S105, Spatial alignment is performed between the first type image and the second type image.

[0092] In this embodiment, during wafer inspection, data from different imaging modes needs to be acquired and spatially aligned for subsequent feature extraction and fusion. Because the acquisition methods, imaging principles, and resolutions of the first and second type images differ, precise positioning, focus adjustment, multi-angle sampling, and correction processing are required during acquisition to ensure data consistency and comparability. Furthermore, since different types of images may have different fields of view, distortions, and scaling ratios, spatial alignment is essential to ensure their features can be analyzed within the same reference coordinate system.

[0093] A motion platform is used to precisely position the wafer inspection area, ensuring that the imaging equipment can be aligned with the area to be inspected. The spatial coordinates of the wafer are acquired through a high-precision position sensor, and coordinate data of the wafer inspection area is generated according to inspection requirements. Coordinate transformation technology is used to convert the physical coordinates of the wafer into the coordinate system of the inspection equipment, ensuring the accuracy of subsequent data acquisition.

[0094] The electron beam focal length of the electron optics module directly affects image quality and needs to be adjusted according to the wafer surface characteristics to ensure optimal image sharpness. Operating parameters of the transmission mode include electron beam intensity, depth of focus, and scanning speed; optimization of these parameters affects the quality and resolution of the first type of image. An automatic focal length adjustment mechanism is employed, performing real-time optimization based on the acquired preview image to obtain the optimal transmission image.

[0095] In transmission mode, the electron beam penetrates the wafer material and forms a high-resolution transmission electron microscopy image on the detector. Multiple sets of Type I images from different angles are acquired to enhance the detection capability for defects in different directions. A multi-angle imaging method is employed to ensure the acquisition of complete morphological information of defects, improving the ability to identify defects with complex structures.

[0096] In scanning transmission mode, the electron beam scans the wafer surface along a preset trajectory, detecting the scattered signals generated by the interaction between electrons and materials. A ring-shaped dark-field detection method is employed to enhance imaging capabilities for low-contrast defects and improve detection accuracy. By combining the coordinate data of the wafer detection area, distortion correction is performed on the acquired second-type image to eliminate perspective distortion and geometric deformation caused by electron beam scanning.

[0097] A feature matching algorithm is used to extract common feature points from the first and second type images, and registration is performed based on these points. Affine transformation or non-rigid registration methods are employed to ensure that the images of different types are fused and analyzed within the same reference coordinate system. By optimizing alignment accuracy, subsequent feature extraction and fusion steps are ensured to be performed at a consistent spatial scale, improving data consistency.

[0098] To ensure the accuracy of image acquisition and alignment, various optimization methods can be employed. For example, autofocus and adaptive exposure adjustment techniques can be used to ensure consistent image sharpness across different imaging modes. During image alignment, feature-point-based matching algorithms, such as SIFT (Scale Invariant Feature Transform) or ORB (Fast Robust Feature Transform), can be used for feature point detection, and the RANSAC algorithm can be used to remove mismatched points, improving alignment accuracy. Furthermore, deep learning methods, such as neural network-based alignment models, can be combined to automatically optimize the alignment effect of different modalities through end-to-end training. For applications requiring high precision, optical distortion correction algorithms can be incorporated to eliminate geometric errors caused by electron beam scanning and lens distortion, improving the quality of the aligned data.

[0099] This embodiment uses a high-precision motion platform to position the wafer inspection area, ensuring that the acquired images have a stable spatial reference benchmark; it ensures imaging quality and improves defect detectability by adjusting the focal length and optimizing the transmission mode of the electro-optical module; it enhances the ability to identify different types of defects through multi-angle acquisition and annular dark-field imaging; and it ensures that different types of images can be analyzed in the same coordinate system through distortion correction and spatial alignment, improving the accuracy of feature extraction and fusion, and ultimately enhancing the accuracy and robustness of wafer defect detection.

[0100] In one embodiment, S20 includes:

[0101] S201, Construct a transmission feature extraction network, which includes a residual connection module and a dilated convolution module;

[0102] S202, Construct a scanning transmission feature extraction network, which includes a multi-scale convolution module and a contrast enhancement module;

[0103] S203, the first type of image is a transmission electron microscope image, and the texture features of particle defects and the morphological features of crack defects in the transmission electron microscope image are extracted by the transmission feature extraction network.

[0104] S204, the second type of image is a scanning transmission electron microscopy image, and the hierarchical structure features of defects in the scanning transmission electron microscopy image are extracted by the scanning transmission feature extraction network.

[0105] In this embodiment, in the wafer defect detection task, different types of images contain different information. Therefore, feature extraction needs to be performed on the first type of image and the second type of image separately to extract their unique defect information. Since transmission electron microscopy (TEM) images can provide high-resolution information about the internal structure of materials, while scanning transmission electron microscopy (STEM) images can provide high-contrast morphological information, different feature extraction networks are constructed for the two types of images to extract the key features of defects to the greatest extent possible and to ensure the effectiveness of subsequent data fusion.

[0106] A transmission feature extraction network was constructed based on a convolutional neural network (CNN), and the network architecture was optimized to adapt to the characteristics of transmission electron microscopy images. Residual connections were introduced into the network structure to ensure the stability of deep network training and enhance the detection capability of low-contrast defects. Dilated convolutions were used to expand the receptive field to capture defect distribution information within a larger area, thereby improving the feature extraction capability for granular and crack defects.

[0107] A convolutional neural network-based scanning transmission electron microscopy (STEM) feature extraction network was constructed, and the network structure was optimized to suit the characteristics of STEM images. By incorporating convolutional kernels of different scales, the model's ability to detect defects of varying sizes was improved, adapting to the defect distribution across different process layers on the wafer surface. To address the high contrast of defects in STEM mode, an adaptive contrast enhancement mechanism was introduced to improve the robustness of feature extraction, making even minute defects more apparent.

[0108] A transmission feature extraction network is employed to extract features from transmission electron microscopy images, separating the texture features of particulate defects and the morphological features of crack defects. Texture feature extraction for particulate defects utilizes multi-layer convolution and pooling operations to enhance the perception of local texture changes, thereby improving the accuracy of particulate contamination detection. Morphological feature extraction for crack defects combines structural tensor analysis, enabling the network to identify the directional characteristics of cracks and improving the accuracy of crack detection.

[0109] A scanning transmission electron microscopy (STEM) feature extraction network is employed to extract features from STEM images, with a focus on extracting the hierarchical structural features of defects. A multi-scale feature fusion method is used to improve the ability to identify cross-layer defects, enabling the system to distinguish defect morphologies in different process layers. Gaussian filtering and gradient mapping methods are used to optimize the feature representation of the STEM images, ensuring accurate extraction of defects at different depths.

[0110] To address diverse detection needs, various methods can be employed to optimize the feature extraction process. For instance, in deep learning optimization, adaptive learning rate adjustment mechanisms can be incorporated to enable the model to better adapt to different image types. In convolutional network design, lightweight network architectures, such as MobileNet or EfficientNet, can be combined to improve computational efficiency and reduce computational resource consumption. For scenarios requiring high precision, Transformer structures can be used, employing global attention mechanisms to optimize feature extraction performance. Furthermore, in the post-processing stage of feature extraction, feature selection methods, such as L1 regularization or PCA dimensionality reduction, can be integrated to remove redundant features and improve the effectiveness of feature representation.

[0111] This embodiment constructs independent transmission electron microscopy (TEM) and scanning transmission electron microscopy (STEM) feature extraction networks to extract key features from TEM images and STEM images, respectively. This allows data from different image types to maximize their advantages and improve the accuracy of defect identification. Optimization strategies such as residual connections, dilated convolutions, multi-scale convolutions, and contrast enhancement further enhance the feature extraction effect, improving the system's ability to identify low-contrast defects and cross-process layer defects. Furthermore, the feature optimization strategies improve the model's robustness, enabling the detection system to adapt to different process environments and enhancing quality control capabilities in wafer manufacturing.

[0112] In one embodiment, S30 includes:

[0113] S301, the first type of feature and the second type of feature are concatenated along the channel dimension to generate an initial fused feature;

[0114] S302, the initial fusion feature input channel attention module generates a first type of feature weight map and a second type of feature weight map;

[0115] S303, Multiply the first type of feature with the first type of feature weight map to generate a weighted first type of feature;

[0116] S304, Multiply the second type of feature with the second type of feature weight map to generate a weighted second type of feature;

[0117] S305, the weighted first type feature and the weighted second type feature are input into the nonlinear activation layer to generate intermediate fusion features;

[0118] S306, calculate the channel similarity matrix between the first type of feature and the second type of feature through the distance metric module;

[0119] S307, Filter channel combinations with similarity below a preset threshold according to the channel similarity matrix;

[0120] S308, the filtered channel combination and intermediate fusion features are superimposed along the channel dimension to generate the final fusion feature.

[0121] In this embodiment, in the wafer defect detection task, transmission electron microscopy (TEM) images and scanning TEM images extract different types of features. To fully utilize these two types of data, feature fusion is required, enabling the final model to more accurately describe the spatial distribution, hierarchical information, and morphological features of defects. The fusion process involves not only simple stitching but also attention weight allocation, weighted feature generation, similarity calculation, and feature selection. This ensures that the final fused features retain key information from both types of features while removing redundant or low-relevance features.

[0122] The first type of features and the second type of features are concatenated along the channel dimension to form the initial fused features, ensuring that feature information from different sources can participate in subsequent calculations. Since the distribution of the first type of features and the second type of features in the feature space may differ, normalization processing can be performed before concatenation to avoid the fusion imbalance caused by the excessively large numerical range of one type of feature.

[0123] The initial fused features are input into the channel attention module to calculate the importance weights of the first and second type features. This module generates first and second type feature weight maps, which are used to weight the features in subsequent steps. An adaptive attention mechanism is employed, enabling the model to dynamically adjust the feature fusion weights based on the characteristics of the input data, thus improving the flexibility of the fusion effect.

[0124] Multiplying the first type of feature with its weight map yields a weighted first type of feature, enhancing its expressive power for the target defect. Multiplying the second type of feature with its weight map yields a weighted second type of feature, allowing the fused feature to better adapt to complex defect morphologies. This weighting mechanism ensures that more important features are amplified, while less important information is weakened, thereby improving the effectiveness of the fused feature.

[0125] Weighted first-type features and weighted second-type features are input into a nonlinear activation layer to generate intermediate fused features. Nonlinear transformations are introduced using activation functions such as ReLU and Swish to improve the model's expressive power, enabling the fused features to more effectively represent complex defect information.

[0126] The distance metric module calculates the channel similarity matrix between the first type of features and the second type of features, analyzing the redundancy and complementarity between different channels. Specific metric methods, such as cosine similarity, Euclidean distance, or Mahalanobis distance, are employed to ensure that the selected feature channels retain useful information to the maximum extent. Channel combinations with similarity below a preset threshold are filtered out to remove redundant or noisy features, further optimizing the quality of the fused features.

[0127] The filtered channel combinations are superimposed with intermediate fusion features along the channel dimension to form the final fusion feature, which is then used for subsequent feature analysis, defect detection, and hierarchical processing. In this way, the fusion feature not only includes key information from both the first and second types of features but has also undergone optimized filtering to ensure that the data ultimately used for defect analysis is more accurate.

[0128] To optimize the feature fusion process, various methods can be employed to improve the quality of the fused features. For example, in the channel concatenation stage, feature normalization methods can be introduced to make the data distribution of different features more consistent, thereby improving fusion stability. In the attention mechanism calculation stage, an adaptive weighting strategy can be used, allowing the model to dynamically allocate weights instead of fixed linear weights. In the channel selection stage, selection criteria based on information entropy or feature contribution can be optimized to improve the effectiveness of the final fused features. Furthermore, for situations with limited computational resources, lightweight network architectures, such as MobileNet or EfficientNet, can be combined to improve computational efficiency and reduce storage overhead.

[0129] This embodiment achieves efficient feature fusion through various techniques, including channel splicing, channel attention mechanisms, feature weighting calculation, nonlinear transformation, similarity measurement, and channel filtering. This allows the fused features to retain key information to the maximum extent, remove redundant data, and improve the model's detection accuracy. The adaptive attention mechanism ensures that the feature fusion process dynamically adapts to the detection needs of different types of defects, improving the model's generalization ability. Furthermore, channel filtering optimization reduces interference from low-relevance features, significantly improving the reliability of the final defect detection.

[0130] In one embodiment, S40 includes:

[0131] S401, Generate a predicted probability distribution of fused features through a defect classification model, wherein the probability distribution represents the prediction confidence of each defect category;

[0132] S402, calculate the confidence score of each fusion feature based on the predicted probability distribution, wherein the confidence score is the maximum probability value in the predicted probability distribution;

[0133] S403, select target fusion features with confidence scores higher than the preset confidence threshold.

[0134] In this embodiment, after fusing the first type of features and the second type of features, the resulting fused features may contain a certain proportion of uncertain information or misclassified samples. Therefore, it is necessary to perform filtering on the fused features to ensure that the data ultimately used for defect analysis and hierarchical identification has a high confidence level. This process mainly evaluates the fused features through a defect classification model and calculates a confidence score based on the predicted probability distribution to screen out reliable target fused features and reduce the impact of low-quality features on subsequent detection accuracy.

[0135] The fused features are inferred using a defect classification model to obtain the predicted probability distribution for each defect category. The predicted probability distribution represents the model's confidence level in different defect categories, typically consisting of probability values ​​for multiple categories, with the sum of all probabilities equal to 1. Deep neural networks (DNNs) or convolutional neural networks (CNNs) are employed for classification inference to improve classification accuracy and generalization ability.

[0136] The maximum probability value is extracted from the predicted probability distribution and used as the confidence score for the fused feature. The confidence score reflects the reliability of the classification model for that feature; a higher value indicates more stable category determination for that feature. Using the maximum probability as the confidence score directly reflects the reliability of the classification results and avoids uncertainty caused by similar probabilities across multiple categories.

[0137] Set a predefined confidence threshold to filter out target fusion features with confidence scores higher than this threshold. Only high-confidence samples are retained to ensure the high reliability of features used in subsequent defect identification. Features below the threshold may belong to regions of model classification uncertainty and are usually discarded or subjected to secondary validation to reduce the impact of misclassification.

[0138] To optimize the feature fusion selection process, various methods can be employed to improve selection accuracy. For example, in the probability distribution generation stage, an attention-based classification network can be used, assigning different weights to different feature regions to improve classification accuracy. In the confidence score calculation stage, methods such as information entropy or Markov distance can be combined to optimize the confidence calculation strategy and reduce classification uncertainty. Furthermore, during the selection of target fusion features, a dynamic threshold adjustment mechanism can be incorporated, allowing the threshold to be adaptively adjusted to improve the selection effect for different batches of data.

[0139] This embodiment generates a predicted probability distribution through a defect classification model, calculates the confidence score of the fused features, and filters data based on a set threshold to ensure high reliability of the data used for subsequent defect analysis. This effectively reduces interference from low-confidence samples, improves the accuracy of defect identification, optimizes data quality, and enhances the model's stability in production environments. Furthermore, by combining adaptive threshold adjustment, the filtering process can be further optimized, improving the model's generalization ability across different process environments.

[0140] In one embodiment, the above S50 includes:

[0141] S501, Match the target fusion feature with a pre-stored process layer template, wherein the process layer template contains structural feature parameters of each process layer of the wafer;

[0142] S502, determine the process layer number where the defect is located based on the matching result, and the process layer number corresponds one-to-one with the layer segment in the wafer manufacturing process.

[0143] S503, extract the defect location coordinates from the target fusion features, and convert the defect location coordinates into physical coordinates of the wafer detection area, wherein the physical coordinates are two-dimensional positions on the wafer surface;

[0144] S504, bind the defect location coordinates with the corresponding process layer number to generate a table relating defect location and process layer.

[0145] In this embodiment, during wafer defect detection, defects may appear in different process layers. Accurately identifying the location of the defect and its corresponding process layer is crucial for subsequent production adjustments and quality control. This step analyzes the target fusion features and combines them with pre-stored process layer templates to determine the specific layer information of the defect. Simultaneously, the spatial coordinates of the defect are converted into actual physical coordinates to facilitate subsequent defect management and process optimization.

[0146] The pre-stored process layer templates contain structural feature parameters of each process layer on the wafer, such as material properties, etching depth, and deposition thickness. The target fusion features contain defect information after feature extraction, fusion, and filtering, and need to be matched with the process layer templates to determine the process layer to which the defect belongs. Feature similarity matching algorithms, such as those based on Euclidean distance, cosine similarity, or Mahalanobis distance, are used to ensure that the target fusion features match the most suitable process layer template.

[0147] The matching process identifies the process layer template whose fusion features most closely match the target, and obtains the corresponding process layer number. This process layer number corresponds one-to-one with the layers in the wafer manufacturing process, ensuring the accuracy of the identified defect layer information. A hierarchical classification model, combined with deep learning and rule matching techniques, improves the accuracy of process layer matching and avoids hierarchical identification errors caused by feature bias.

[0148] The location information of defects is extracted from the target fusion features to determine the coordinates of the defects in the feature space. A coordinate transformation model is used to map the defect location to the physical coordinates of the wafer inspection area, enabling the detected defects to be accurately located on the actual wafer surface. The physical coordinates are two-dimensional, typically represented by (X,Y) coordinates, ensuring that the actual location information of the defects can be used for subsequent production adjustments and defect visualization.

[0149] By combining the physical coordinates of defects and their process layer numbers, a correlation table is constructed between defect locations and process layers. This table records the spatial location of each defect and its associated process layer, providing data support for subsequent defect analysis and process optimization. The correlation table can be stored in a database or transmitted to a production management system for real-time quality control and traceability analysis.

[0150] To optimize defect hierarchy identification and coordinate mapping, various methods can be employed to improve identification accuracy. For example, in the target fusion feature and process layer template matching stage, a deep learning-based hierarchical classification model can be combined to optimize the accuracy of hierarchy matching. In the defect location coordinate transformation stage, coordinate transformation matrices or calibration-based coordinate mapping algorithms can be used to improve the accuracy of defect physical coordinates. Furthermore, in the defect location and process layer association table generation stage, database optimization techniques can be incorporated to make data storage and retrieval more efficient.

[0151] This embodiment accurately identifies the process level of defects by matching target fusion features with process layer templates; it achieves precise defect location through defect location coordinate transformation; and it ensures defect information traceability and improves the automation level of production management by constructing an association table between defect location and process layer. This effectively improves the accuracy of defect level identification, optimizes defect management, and increases production yield and process optimization efficiency.

[0152] In one embodiment, after S504 above, the method further includes:

[0153] S505, the defect location coordinates and the corresponding process layer number are superimposed onto the wafer surface image in the form of visual markers, the visual markers including defect outline boxes and process layer number labels;

[0154] S506, color-code the defect outline according to the process layer number, with different colors corresponding to different process layers;

[0155] S507 combines the superimposed wafer surface image with the color-coded defect outline to generate a layered detection result image;

[0156] S508, convert the association table between defect location and process layer into a standardized data format, wherein the standardized data format includes defect location coordinates, process layer number and confidence score;

[0157] S509, the association table in the standardized data format is transmitted to the production management module through a predefined communication protocol.

[0158] In this embodiment, after completing the layered identification of defects, in order to improve the readability and traceability of defect detection results, the defect information needs to be visualized and stored and transmitted to the production management module in a standardized data format for production process adjustment and optimization. This step involves visually labeling the defect location coordinates and process layer numbers, applying color coding for layered display, and standardizing the data to support subsequent data analysis and process optimization.

[0159] Defect location coordinates and process layer numbers are overlaid on the wafer surface image, allowing defect information to be visually presented in the inspection result image. Visual markers use defect outlines and process layer number labels to clearly distinguish different defects and their respective process layers on the same image. High-resolution image rendering ensures that the marker information remains clearly visible at different magnifications, improving readability.

[0160] Based on the process layer number, the defect outline is color-coded to ensure that defects in different process layers can be distinguished by color. The color coding strategy can employ a predefined color mapping table, such as using a color distribution scheme in the RGB or HSV color space, ensuring that all defects within the same process layer have a consistent color identifier. Combined with a color contrast enhancement algorithm, this ensures that the color coding remains clearly visible under different lighting or display conditions.

[0161] The superimposed wafer surface image is combined with color-coded defect outlines to generate a complete layered inspection result image. Multi-channel fusion technology is employed to ensure seamless integration of the original wafer surface image and defect marking information, avoiding image blurring or data confusion caused by information overlay. Dynamic interactive visualization support is provided, allowing users to view, zoom in, and filter defect data from different process layers within the production monitoring system, improving the flexibility of visualization analysis.

[0162] The association table between defect locations and process layers is converted into a standardized data format to ensure that the data is compatible with different production management systems.

[0163] Standardized data formats can include:

[0164] Defect location coordinates (two-dimensional physical coordinates, used for precise positioning);

[0165] Process layer number (indicating the process layer segment to which the defect belongs);

[0166] Confidence score (reflecting the reliability of the defect identification result).

[0167] Standardized data formats such as JSON, XML, or CSV are used to adapt to different data storage and transmission needs.

[0168] A standardized data format association table is transmitted to the production management module via a predefined communication protocol for further production optimization and quality traceability. Data transmission is performed using RESTful APIs, MQTT, or industrial bus protocols to improve real-time performance and reliability. Combined with data encryption and authentication mechanisms, data security during transmission is ensured to prevent the leakage of sensitive production information.

[0169] For different application scenarios, various methods can be used to optimize the visualization annotation, color encoding, data format conversion, and data transmission processes. For example, in the visualization annotation stage, object detection frameworks from computer vision (such as YOLO and Faster R-CNN) can be used to ensure the accuracy of defect contour boxes. In the color encoding stage, dynamic color mapping algorithms can be used to ensure clear color differences between different levels, improving readability. In the standardized data conversion stage, database-based automatic format conversion modules can be used to ensure seamless data integration with different production management systems. Furthermore, in the data transmission stage, Industry 4.0 standard data interface protocols, such as OPC UA or Modbus, can be used to improve data compatibility and real-time performance.

[0170] This embodiment utilizes technologies such as visual annotation, color coding, standardized data conversion, and data transmission to achieve intuitive display, categorized management, and efficient storage of defect information. This enables the production management system to acquire defect information in real time and make dynamic adjustments. It effectively improves the readability and traceability of defect information, optimizes production management, and increases wafer manufacturing yield and efficiency.

[0171] In one embodiment, a wafer defect identification device based on feature fusion is provided, which corresponds one-to-one with the wafer defect identification method based on feature fusion described in the above embodiments. (Refer to...) Figure 3 , Figure 3This is a schematic diagram of the functional modules of a preferred embodiment of the wafer defect identification device based on feature fusion of the present invention. The modules include an image acquisition and alignment module 10, a feature extraction module 20, a feature fusion module 30, a feature screening and filtering module 40, and a defect location and process layer identification module 50. Detailed descriptions of each functional module are as follows:

[0172] The image acquisition and alignment module 10 is used to acquire a first type of image and a second type of image of the wafer inspection area, and to perform alignment processing on the first type of image and the second type of image;

[0173] Feature extraction module 20 is used to extract features from the first type image and the second type image respectively, and generate first type features and second type features;

[0174] Feature fusion module 30 is used to fuse the first type of feature and the second type of feature to generate fused feature;

[0175] The feature selection and filtering module 40 is used to perform filtering processing on the fused features and select target fused features that meet the confidence threshold conditions;

[0176] The defect location and process layer identification module 50 is used to determine the defect location and corresponding process layer information based on the target fusion features.

[0177] In one embodiment, the image acquisition and alignment module 10 is specifically used for:

[0178] The wafer inspection area is located using a motion platform, and coordinate data of the wafer inspection area is generated.

[0179] Adjust the electron beam focal length of the electron optics module and set the operating parameters of the transmission mode in the electron optics module;

[0180] Based on the transmission mode of the electro-optical module, multiple sets of first-type images from different angles are acquired;

[0181] Based on the scanning transmission mode of the electro-optics module, a second type of image of the annular dark field is acquired, and distortion correction processing is performed on the second type of image based on the coordinate data of the wafer detection area.

[0182] Spatial alignment is performed between the first type of image and the second type of image.

[0183] In one embodiment, the feature extraction module 20 is specifically used for:

[0184] A transmission feature extraction network is constructed, which includes a residual connection module and a dilated convolution module.

[0185] A scanning transmission feature extraction network is constructed, which includes a multi-scale convolution module and a contrast enhancement module;

[0186] The first type of image is a transmission electron microscope image, and the texture features of particle defects and the morphological features of crack defects in the transmission electron microscope image are extracted by the transmission feature extraction network.

[0187] The second type of image is a scanning transmission electron microscopy image, and the hierarchical structural features of defects in the scanning transmission electron microscopy image are extracted by the scanning transmission feature extraction network.

[0188] In one embodiment, the feature fusion module 30 is specifically used for:

[0189] The first type of feature and the second type of feature are concatenated along the channel dimension to generate the initial fused feature;

[0190] The initial fused features are input into the channel attention module to generate a first type of feature weight map and a second type of feature weight map;

[0191] Multiply the first type of feature with the first type of feature weight map to generate a weighted first type of feature;

[0192] Multiply the second type of feature with the second type of feature weight map to generate a weighted second type of feature;

[0193] The weighted first-type features and the weighted second-type features are input into a nonlinear activation layer to generate intermediate fused features;

[0194] The channel similarity matrix between the first type of feature and the second type of feature is calculated using the distance metric module;

[0195] Based on the channel similarity matrix, channel combinations with similarity below a preset threshold are selected;

[0196] The filtered channel combinations are superimposed with the intermediate fusion features along the channel dimension to generate the final fusion feature.

[0197] In one embodiment, the feature selection and filtering module 40 is specifically used for:

[0198] A predicted probability distribution of fused features is generated through a defect classification model, whereby the probability distribution represents the prediction confidence of each defect category.

[0199] The confidence score for each fusion feature is calculated based on the predicted probability distribution, where the confidence score is the maximum probability value in the predicted probability distribution.

[0200] Select target fusion features with confidence scores higher than the preset confidence threshold.

[0201] In one embodiment, the defect location and process layer identification module 50 is specifically used for:

[0202] The target fusion feature is matched with a pre-stored process layer template, which contains structural feature parameters of each process layer of the wafer;

[0203] The process layer number where the defect is located is determined based on the matching results. The process layer number corresponds one-to-one with the layer segment in the wafer manufacturing process.

[0204] Defect location coordinates are extracted from the target fusion features and converted into physical coordinates of the wafer detection area, where the physical coordinates are two-dimensional positions on the wafer surface.

[0205] The defect location coordinates are bound to the corresponding process layer number to generate an association table between the defect location and the process layer.

[0206] In one embodiment, the defect location and process layer identification module 50 is specifically used for:

[0207] The defect location coordinates and the corresponding process layer number are superimposed onto the wafer surface image in the form of visual markers, the visual markers including defect outline boxes and process layer number labels;

[0208] The defect outline is color-coded according to the process layer number, with different colors corresponding to different process layers;

[0209] The superimposed wafer surface image is combined with the color-coded defect outline to generate a layered detection result image;

[0210] The association table between defect location and process layer is converted into a standardized data format, which includes defect location coordinates, process layer number and confidence score.

[0211] The association table in the standardized data format is transmitted to the production management module through a predefined communication protocol.

[0212] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external user terminals via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a feature fusion-based wafer defect identification method on the server side.

[0213] In one embodiment, a computer device is provided, which may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the user-side functions or steps of a feature fusion-based wafer defect identification method.

[0214] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0215] Acquire a first type of image and a second type of image of the wafer inspection area, and perform alignment processing on the first type of image and the second type of image;

[0216] Feature extraction is performed on the first type of image and the second type of image respectively to generate first type features and second type features;

[0217] The first type of feature and the second type of feature are combined to generate a fused feature;

[0218] The fused features are filtered, and target fused features that meet the confidence threshold conditions are selected.

[0219] Based on the target fusion features, the defect location and corresponding process layer information are determined.

[0220] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0221] Acquire a first type of image and a second type of image of the wafer inspection area, and perform alignment processing on the first type of image and the second type of image;

[0222] Feature extraction is performed on the first type of image and the second type of image respectively to generate first type features and second type features;

[0223] The first type of feature and the second type of feature are combined to generate a fused feature;

[0224] The fused features are filtered, and target fused features that meet the confidence threshold conditions are selected.

[0225] Based on the target fusion features, the defect location and corresponding process layer information are determined.

[0226] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and user side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0227] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0228] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0229] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A wafer defect identification method based on feature fusion, characterized in that, Includes the following steps: A first type of image and a second type of image of the wafer inspection area are acquired. The first type of image is a transmission electron microscopy image and the second type of image is a scanning transmission electron microscopy image. Alignment processing is performed on the first type of image and the second type of image. Feature extraction is performed on the first type of image and the second type of image respectively to generate first type features and second type features; The process of fusing the first type of feature and the second type of feature to generate a fused feature includes: concatenating the first type of feature and the second type of feature along the channel dimension to generate an initial fused feature; inputting the initial fused feature into a channel attention module to generate a first type of feature weight map and a second type of feature weight map; multiplying the first type of feature with the first type of feature weight map to generate a weighted first type of feature; multiplying the second type of feature with the second type of feature weight map to generate a weighted second type of feature; inputting the weighted first type of feature and the weighted second type of feature into a nonlinear activation layer to generate an intermediate fused feature; calculating the channel similarity matrix between the first type of feature and the second type of feature through a distance metric module; filtering channel combinations with similarity below a preset threshold based on the channel similarity matrix; and superimposing the filtered channel combinations with the intermediate fused feature along the channel dimension to generate the final fused feature. The fused features are filtered, and target fused features that meet the confidence threshold conditions are selected. Based on the target fusion features, the defect location and the corresponding process layer number are determined.

2. The wafer defect identification method based on feature fusion as described in claim 1, characterized in that, Acquire a first type of image and a second type of image of the wafer inspection area, and perform alignment processing on the first type of image and the second type of image, including: The wafer inspection area is located using a motion platform, and coordinate data of the wafer inspection area is generated. Adjust the electron beam focal length of the electron optics module and set the operating parameters of the transmission mode in the electron optics module; Based on the transmission mode of the electro-optical module, multiple sets of first-type images from different angles are acquired; Based on the scanning transmission mode of the electro-optics module, a second type of image of the annular dark field is acquired, and distortion correction processing is performed on the second type of image based on the coordinate data of the wafer detection area. Spatial alignment is performed between the first type of image and the second type of image.

3. The wafer defect identification method based on feature fusion as described in claim 1, characterized in that, Feature extraction is performed on the first type of image and the second type of image respectively to generate first type features and second type features, including: A transmission feature extraction network is constructed, which includes a residual connection module and a dilated convolution module. A scanning transmission feature extraction network is constructed, which includes a multi-scale convolution module and a contrast enhancement module; The first type of image is a transmission electron microscope image, and the texture features of particle defects and the morphological features of crack defects in the transmission electron microscope image are extracted by the transmission feature extraction network. The second type of image is a scanning transmission electron microscopy image, and the hierarchical structural features of defects in the scanning transmission electron microscopy image are extracted by the scanning transmission feature extraction network.

4. The wafer defect identification method based on feature fusion as described in claim 1, characterized in that, The fused features are filtered to select target fused features that meet the confidence threshold conditions, including: A predicted probability distribution of fused features is generated through a defect classification model, whereby the probability distribution represents the prediction confidence of each defect category. The confidence score for each fusion feature is calculated based on the predicted probability distribution, where the confidence score is the maximum probability value in the predicted probability distribution. Select target fusion features with confidence scores higher than the preset confidence threshold.

5. The wafer defect identification method based on feature fusion as described in claim 1, characterized in that, Determining the defect location and corresponding process layer information based on the target fusion features includes: The target fusion feature is matched with a pre-stored process layer template, which contains structural feature parameters of each process layer of the wafer; The process layer number where the defect is located is determined based on the matching results. The process layer number corresponds one-to-one with the layer segment in the wafer manufacturing process. Defect location coordinates are extracted from the target fusion features and converted into physical coordinates of the wafer detection area, where the physical coordinates are two-dimensional positions on the wafer surface. The defect location coordinates are bound to the corresponding process layer number to generate an association table between the defect location and the process layer.

6. The wafer defect identification method based on feature fusion as described in claim 5, characterized in that, After binding the defect location coordinates to the corresponding process layer number to generate an association table between defect location and process layer, the process also includes: The defect location coordinates and the corresponding process layer number are superimposed onto the wafer surface image in the form of visual markers, the visual markers including defect outline boxes and process layer number labels; The defect outline is color-coded according to the process layer number, with different colors corresponding to different process layers. The superimposed wafer surface image is combined with the color-coded defect outline to generate a layered detection result image; The association table between defect location and process layer is converted into a standardized data format, which includes defect location coordinates, process layer number and confidence score. The association table in the standardized data format is transmitted to the production management module through a predefined communication protocol.

7. A wafer defect identification device based on feature fusion, characterized in that, The wafer defect identification device based on feature fusion includes: The image acquisition and alignment module is used to acquire a first type of image and a second type of image of the wafer inspection area. The first type of image is a transmission electron microscopy image and the second type of image is a scanning transmission electron microscopy image. The module also performs alignment processing on the first type of image and the second type of image. The feature extraction module is used to extract features from the first type of image and the second type of image respectively, and generate first type features and second type features; A feature fusion module is used to fuse the first type of features and the second type of features to generate fused features, including: concatenating the first type of features and the second type of features along the channel dimension to generate an initial fused feature; inputting the initial fused feature into a channel attention module to generate a first type of feature weight map and a second type of feature weight map; multiplying the first type of features with the first type of feature weight map to generate a weighted first type of feature; multiplying the second type of features with the second type of feature weight map to generate a weighted second type of feature; inputting the weighted first type of feature and the weighted second type of feature into a nonlinear activation layer to generate an intermediate fused feature; calculating the channel similarity matrix between the first type of features and the second type of features through a distance metric module; filtering channel combinations with similarity below a preset threshold according to the channel similarity matrix; and superimposing the filtered channel combinations with the intermediate fused features along the channel dimension to generate the final fused feature. The feature selection and filtering module is used to perform filtering processing on the fused features and select target fused features that meet the confidence threshold conditions. The defect location and process layer identification module is used to determine the defect location and the corresponding process layer number based on the target fusion features.

8. A computer device, characterized in that, The computer device includes a memory, a processor, and a feature fusion-based wafer defect identification program stored in the memory and executable on the processor. When executed by the processor, the feature fusion-based wafer defect identification program implements the steps of the feature fusion-based wafer defect identification method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a wafer defect identification program based on feature fusion, which, when executed by a processor, implements the steps of the wafer defect identification method based on feature fusion as described in any one of claims 1-6.

Images