Method and system for automatically identifying wafer internal defect image of 3D stacked chip

Abstract

The invention discloses an automatic identification method and system for a wafer internal defect image of a 3D stacked chip, and belongs to the technical field of semiconductor manufacturing and detection. According to the method, optical, X-ray and ultrasonic image data are synchronously acquired based on a multi-modal imaging technology, imaging parameters are dynamically adjusted to adapt to different wafer levels and material characteristics, multi-modal features are extracted in combination with layered filtering and denoising, multi-resolution registration and a self-supervised deep learning method, and micron-sized defects are positioned by using an attention mechanism. And further constructing a defect-process parameter correlation model through reinforcement learning, generating a closed-loop process optimization instruction, and transmitting the closed-loop process optimization instruction to an execution system. The system comprises a multi-modal imaging module, a noise suppression module, a deep learning analysis module and a process optimization module, and supports edge computing deployment. According to the invention, the internal defect detection efficiency and precision of the multi-layer stacked chip are significantly improved, real-time closed-loop control of detection-analysis is realized, the wafer manufacturing quality risk is reduced, and the method is suitable for intelligent defect detection of an advanced packaging production line.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor manufacturing and detection technology, and specifically to a method and system for automatically identifying internal defects in 3D stacked chip wafers based on multimodal imaging and deep learning. Background Art

[0002] Currently, internal defect detection for wafers containing three-dimensional stacked chips still relies heavily on single-modal imaging technology and manually-led analysis processes. Optical imaging is widely used to identify surface scratches or particle contamination due to its high-resolution characteristics, but the inherent limitations of its physical penetration capabilities prevent it from capturing deep defects such as interlayer bonding cracks and microbubbles. While X-ray tomography can generate three-dimensional structural images, its high equipment complexity and low imaging efficiency make it difficult to popularize in mass production lines. Ultrasonic testing is limited by sound wave attenuation and resolution, making it unreliable for identifying submicron defects. More critically, existing detection systems generally use static parameter configurations (such as fixed imaging modes and preset filter thresholds), which are unable to adapt to different material properties and process conditions in real time, resulting in significant fluctuations in detection accuracy.

[0003] In recent years, technological development has achieved breakthroughs around multimodal data fusion and intelligent algorithms. Early supervised learning models (such as classifiers based on convolutional neural networks) achieved efficient recognition of known defects through large amounts of manually labeled data, but they suffered from serious generalization defects when faced with new defect types or complex interlayer structures. To reduce reliance on annotations, unsupervised methods (such as autoencoders and contrastive learning) have gradually been introduced, but their feature extraction processes lack the ability to adapt to differences in cross-modal imaging principles, resulting in insufficient spatial registration accuracy and a high incidence of false detections under noise interference. Despite attempts to introduce attention mechanisms or feature pyramid networks to improve model sensitivity, existing algorithms have not yet achieved substantial breakthroughs in the positioning stability of tiny defects (such as nanoscale metal residues) and the ability to model three-dimensional interlayer correlations.

[0004] Technical difficulties are concentrated in three dimensions: First, the difficulty of expanding the detection dimension—existing single-modal equipment struggles to simultaneously achieve a balance between high penetration (such as deep silicon-based structures) and high resolution (such as surface submicron defects). The lack of a dynamic parameter optimization mechanism exacerbates this contradiction. Second, the bottleneck of algorithm robustness—the differences in noise characteristics of multimodal data and cross-dimensional registration errors lead to a sharp increase in algorithm complexity. Traditional processing methods (such as rigid transformation and fixed threshold segmentation) cannot balance computational efficiency and accuracy. Third, the system closed-loop control gap—the lack of an automated correlation model between defect identification results and process parameter adjustments. Key decisions rely on manual intervention, resulting in delayed detection response and blind spots in quality control. These pain points seriously hinder the improvement of the yield of 3D stacked chips and the speed of technological iteration. Summary of the Invention

[0005] This invention relates to a method and system for automatically identifying internal defects in 3D stacked chip wafers. The system aims to address the difficulty of existing inspection technologies in balancing accuracy and efficiency in multi-layer stacked structures. By integrating multimodal imaging with intelligent algorithms, the invention enables precise identification of complex internal defects and process optimization.

[0006] In one aspect, the present invention specifically provides a method for automatically identifying internal defect images of a 3D stacked chip wafer, characterized by comprising the following steps: S1: Uses multimodal imaging techniques including optical imaging, X-ray imaging, and ultrasound imaging to acquire wafer image data, and adjusts imaging parameters in real time based on the structural characteristics and material properties of different wafer layers, such as density, transmittance, and acoustic impedance of each layer. S2: Preprocessing the multimodal image data includes suppressing noise signals through layered filtering, improving the saliency of defect area features using a contrast enhancement algorithm, and performing spatial registration of cross-modal images to ensure geometric consistency of different imaging data; S3: The pre-processed image data is input into a deep learning model, which extracts multi-level features through a self-supervised learning algorithm and uses an attention mechanism to identify the location and type of defects within the wafer. S4: Correlate and analyze the defect identification results with historical process parameters, generate process optimization suggestions based on preset decision strategies, and transmit the optimization suggestions to the manufacturing execution system through a programmable interface.

[0007] Furthermore, a method for automatically identifying internal defect images of a 3D stacked chip wafer further includes that the real-time adjustment of imaging parameters in step S1 further includes: The incident angle and polarization mode of the optical imaging light source are autonomously adjusted according to the reflection characteristics of the wafer surface to light; the phase sensitivity and energy parameters of X-ray imaging are adjusted in combination with the penetration characteristics of the material inside the wafer; and the probe frequency and sound beam focusing parameters of different detection depths are adapted based on the propagation attenuation characteristics of ultrasound.

[0008] Furthermore, a method for automatically identifying internal defect images of a 3D stacked chip wafer further includes that the spatial registration operation of the cross-modal images in step S2 further includes: Common feature points in different imaging data are extracted as registration benchmarks; spatial deformation caused by different imaging principles is corrected through non-rigid transformation algorithms; multi-resolution interpolation compensation is performed on pixel data in missing matching areas to maintain image integrity.

[0009] Furthermore, a method for automatically identifying internal defect images of a 3D stacked chip wafer further includes: the self-supervised learning algorithm method of the deep learning model in step S3 further includes: In the pre-training stage, a comparative learning strategy is used to constrain the feature similarity of unlabeled data to generate a robust low-level feature extraction module; in the fine-tuning stage, the model is optimized for the target task using labeled data, and the ability to recognize tiny defects is enhanced through attention mapping; in the inference stage, inter-layer defect association rules are established in combination with the wafer-level structural features, and a structured inspection report is output.

[0010] Furthermore, a method for automatically identifying internal defect images of a 3D stacked chip wafer further includes: the process of generating process optimization suggestions based on the preset decision strategy in step S4 further includes: Construct a historical data association matrix between defect types and process parameters to identify abnormal parameter combinations; use simulation models to predict the potential impact of different parameter adjustment schemes on yield; and output the scheme whose predicted results meet the preset optimization goals as executable instructions.

[0011] Furthermore, a method for automatically identifying internal defect images of 3D stacked chip wafers also includes verifying the safety boundary conditions of the instructions based on the equipment operating status and environmental variables before the parameter adjustment instructions are issued; if the verification result is a risk overflow, the calibration mode is triggered or the instructions are transferred to the manual review process.

[0012] Furthermore, a method for automatically identifying internal defect images of a 3D stacked chip wafer further includes that the hierarchical filtering to suppress noise signals using a contrast enhancement algorithm in step S2 further includes: A composite noise reduction process of guided filtering and bilateral filtering is performed on the highlight reflective areas in the optical image; an iterative reconstruction algorithm is used to perform spatial filtering to eliminate the ring artifacts in the X-ray image; and wavelet threshold segmentation and feature filtering are performed to separate the scattered noise in the ultrasound imaging.

[0013] Furthermore, a method for automatically identifying internal defect images of a 3D stacked chip wafer further includes: step S4 further includes: Generate a process anomaly tracing path based on the defect distribution heat map and integrate the path into the visual analysis interface; monitor the drift of key parameters in real time, and trigger an alarm or shutdown command when the drift exceeds the adaptive threshold.

[0014] In one aspect, the present invention specifically provides a system for automatically identifying internal defects of 3D stacked chip wafers, comprising: It includes the following modules connected in sequence: Multimodal imaging module: integrates optical imaging equipment, X-ray imager and ultrasonic probe array, and is configured to dynamically adjust imaging parameters according to the wafer hierarchical structure and material properties and output multi-source image data; preprocessing execution module: connected to the output end of the multimodal imaging module, includes a noise suppression unit, a contrast enhancement unit and a registration algorithm unit, and is configured to perform denoising, enhancement and geometric alignment operations on the imaging data; defect recognition module: connected to the output end of the preprocessing execution module, deploys a deep learning model and an attention focusing network, and is configured to analyze image data and locate defect positions; process optimization module: connected to the output end of the defect recognition module, integrates a decision engine and a communication interface, and is configured to generate process adjustment instructions and transmit the instructions to the production control system.

[0015] Furthermore, a system for automatically identifying internal defects of wafers of 3D stacked chips further includes: Calibration self-test module: embedded in the multimodal imaging module, used to regularly perform imaging parameter calibration and hardware status monitoring; safety verification module: connected before the process optimization module, used to evaluate the execution risk of parameter adjustment plans and screen safety instructions; redundant control module: deployed in the instruction output link of the process optimization module, and ensures the reliability of control signals through a multi-node voting mechanism.

[0016] Beneficial effects The 3D stacked chip wafer defect detection method and system of the present invention integrates multimodal imaging, self-supervised intelligent algorithms, and closed-loop process optimization mechanisms to achieve the following core values from the perspectives of technological breakthroughs and production line adaptability: 1. Accurate imaging of defects at all levels: Through cross-modal collaboration of optics, X-rays, and ultrasound, imaging parameters are adjusted in real time to match material properties (density, transmittance, and acoustic impedance). This allows for the simultaneous capture of both micron-level surface roughness and deep nanoscale defects (such as TSV holes and bonding cracks), addressing imaging blind spots in low-density materials and complex stacked structures, significantly improving the detection rate of hidden defects.

[0017] 2. Unsupervised Feature Fusion and Noise-Resistant Registration: This algorithm extracts common features of cross-modal defects based on self-supervised contrastive learning. It combines non-rigid transformation registration with a layered denoising strategy to reduce noise interference while preserving the edge features of small defects. This significantly enhances the algorithm's ability to generalize and identify unknown defect types, reducing reliance on manual annotation and false positives.

[0018] 3. Closed-Loop Process Optimization and Efficient Decision-Making: This system builds a multidimensional correlation model between defect distribution and lithography, bonding, and deposition parameters. It dynamically generates process optimization instructions through reinforcement learning and integrates a redundancy mechanism to verify the reliability of these instructions. The system triggers real-time production line alarms and traces the source of defects, achieving a minute-by-minute closed-loop control system of "detection-feedback-adjustment," mitigating the risk of yield fluctuations.

[0019] 4. Flexible Deployment and Rapid Adaptation: This system supports seamless switching between lightweight edge model execution and in-depth cloud-based analysis, adapting to high-speed mass production testing and multi-process R&D scenarios. Based on historical data transfer learning, it can quickly adapt to the testing requirements of new packaging materials or processes, shortening production line upgrade cycles. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 This is a schematic overall block diagram of the system architecture of the present invention; Figure 2 This is a schematic diagram of dynamic adjustment of multimodal imaging parameters according to a specific embodiment of the present invention; Figure 3 1 is a schematic diagram of a multimodal image preprocessing process according to a specific embodiment of the present invention; Figure 4 This is a schematic diagram of the defect recognition model architecture (CNN+RNN combination) of a specific embodiment of the present invention; Figure 5 is a schematic diagram of self-supervised learning and attention mechanism according to a specific embodiment of the present invention; Figure 6 This is a schematic diagram of a process for generating a defect analysis report by reinforcement learning according to a specific embodiment of the present invention; Figure 7 This is a logical diagram of an early warning mechanism according to a specific embodiment of the present invention; Figure 8 1 is a schematic diagram of the internal structure of a multimodal imaging module according to a specific embodiment of the present invention; Figure 9 This is a detailed flowchart of an image preprocessing module according to a specific embodiment of the present invention; Figure 10 This is a schematic diagram of a distributed computing architecture of a defect identification module according to a specific embodiment of the present invention; Figure 11 This is a functional diagram of a data storage module according to a specific embodiment of the present invention; Figure 12 is a schematic diagram of a model compression technology (knowledge distillation) according to a specific embodiment of the present invention; Figure 13 This is a flow chart of a historical defect data statistical analysis interface according to a specific embodiment of the present invention; Figure 14This is a schematic diagram of an application scenario of the integration of a system and MES according to a specific embodiment of the present invention; Figure 15 It is a schematic diagram of a system deployment architecture of the present invention. DETAILED DESCRIPTION

[0022] Name Explanation 1. The Multi-Modal Wafer Defect Detection System (MMWDDS) is an integrated system independently developed for the detection of multi-layer heterogeneous structures of 3D stacked chips. It integrates three imaging modalities, optical, X-ray, and ultrasonic, and intelligent algorithms. Through hardware collaboration and parameter adaptation, it achieves full-level defect coverage from the wafer surface to the internal layers. Its X-ray imaging submodule is based on formula (1) Calculate the breakdown voltage, where is Planck's constant, which represents the quantized unit of photon energy; is the speed of light, which is the basic physical constant for the propagation of electromagnetic waves; is the wavelength of X-rays (in nm), which determines the penetrating power of photons; The electron charge is the basic unit of charge. Purpose: By accurately calculating the penetration voltage, X-rays effectively penetrate silicon, metal, and other multilayer materials, capturing the internal structural details of TSVs. Results: Detection of 0.5μm-level TSV voids and interlayer dielectric defects is achieved, increasing coverage by 50% compared to traditional single-mode inspection and reducing the missed detection rate from 25% to 5%.

[0023] 2. The Cross-Modal Feature Fusion Engine (CMFFE) is the core algorithm module for solving the dimensional differences among optical, X-ray, and ultrasound image data. Dynamic weighted fusion of multimodal features. To fuse the feature vectors, the defect information of the three modalities is integrated; is the attention weight of each modality , automatically assigned according to defect type (such as interlayer defect enhancement ultrasonic modal weight); is the unimodal eigenvector (Corresponding to optical, X-ray, and ultrasonic). Through dynamic weight allocation, the complementarity of cross-modal defect features is enhanced, solving the problem of insufficient single-modal information. The result: In copper bond interface crack detection, the defect edge feature response strength is increased by 2.5 times, the detection accuracy rate is increased from 82% to 100%, and the ability to identify low-contrast defects is significantly enhanced.

[0024] 3. The Interlayer Defect Correlation Analysis Module (IDCAM) is designed for the vertical interconnection structure of 3D stacked chips. It captures the spatial-sequence correlation of multi-layer defects based on a bidirectional LSTM network. The core formula is (3) .in is the defect association probability, which outputs a probability value between 0 and 1 through the Sigmoid function; It is the hidden state of the current layer and stores the abstract features of the defects of the current layer; For cell states, long-term dependency information of interlayer defects is conveyed; is the weight matrix, used for feature transformation; This is a bias term that adjusts the activation threshold. Purpose: This function exploits the correlation between defects in upper and lower layers, avoiding the isolation of single-layer detection. Results: When a void is detected in the upper-layer TSV, the probability of predicting a crack at the lower-layer bond interface is increased by 45% compared to independent detection. The missed detection rate for interlayer defects is reduced from 30% to 12%, effectively identifying chain defects caused by multi-layer interconnects.

[0025] 4. The Self-Supervised Contrastive Learning Engine (SSCLE) is a key module to address the scarcity of label data in 3D stacked chip detection. Optimize the feature embedding space. It is a contrast loss that measures the difference in feature similarity between positive and negative sample pairs; is the sample logarithm, is a positive sample (feature vector of the same defect from different perspectives), are negative samples (feature vectors of different defects); is a temperature parameter that controls the density of feature distribution. Purpose: Leverages unlabeled data to enhance model generalization and reduce reliance on manual annotation. Results: In detecting bubble defects in novel dielectric materials, the recognition accuracy reached 85% (this type was not included in the training set), a 22% improvement over supervised learning solutions. This reduces data annotation costs by 70%, significantly shortening the model training cycle.

[0026] 5. The Defect-Process Mapping Database (DPMDB) is a database that associates process parameters and defect features based on reinforcement learning. It stores historical defect data and optimal process adjustment plans. The core formula is (5) .in The optimal action is the recommended process parameter adjustment solution (e.g. bonding pressure +5%, exposure time -2%). The discount factor balances immediate and long-term rewards; The state-action value function evaluates the expected benefits under future states. Function: Generates intelligent process adjustment strategies through historical data training, enabling automatic mapping of inspection results to production parameters. Results: When the bond interface defect rate for five consecutive wafers exceeds 3%, the optimal solution is automatically selected to adjust the bond pressure from 80N to 84N. This reduces the defect rate to 1.2% within 10 wafers, increases the yield by 4.7%, and shortens the process adjustment response time from 30 minutes to 20 seconds, a 60% improvement in efficiency compared to manual parameter adjustment, significantly reducing trial-and-error costs and quality risks.

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0028] Example 1 provides an automatic wafer defect identification system for 3D stacked chip manufacturing. To address the problems in the existing technology of multi-layer heterogeneous structure detection, such as insufficient coverage of surface and deep defects (such as limited optical imaging penetration leading to missed detection of interlayer defects, insufficient ultrasonic detection resolution making it difficult to identify sub-micron defects), severe cross-modal data noise interference, and disconnection between detection results and process control, the system realizes full-process intelligence from defect detection to process optimization through the deep integration of multi-modal imaging collaborative mechanism, intelligent algorithm optimization and closed-loop control strategy.

[0029] The multimodal imaging module of the system ( Figure 8 , U1) integrated optical imaging submodule ( Figure 8 , U2), X-ray phase contrast imaging submodule ( Figure 8 , U3) and multi-frequency ultrasound probe submodule ( Figure 8 , U4), each submodule adjusts the decision unit through parameters ( Figure 2, U1) receives the wafer material characteristic parameters (density, transmittance, acoustic impedance) and dynamically adapts the imaging parameters according to the physical properties of the material. For wafer surface defect detection, the optical imaging submodule is equipped with a 200x telecentric microscope and a laser ranging autofocus system (accuracy ±0.1μm). By dynamically adjusting the incident angle and polarization mode of the light source, it effectively suppresses the edge blurring caused by mirror reflection and achieves clear imaging of 0.8μm surface scratches. For deep TSV structures, the X-ray phase contrast imaging submodule calculates the penetration voltage based on formula (1). For the 300μm silicon layer and 50μm copper interconnect layer, the voltage parameter is set to 150kV (retaining a 20% safety margin based on the theoretical value of 124kV). Phase contrast imaging technology is used to enhance the contrast between low-density materials and metal interfaces, achieving accurate detection of 0.5μm voids inside TSVs. The multi-frequency ultrasonic probe submodule uses 25MHz phased array technology. Based on the ultrasonic attenuation formula (6) Adaptively adjusting focused acoustic beam parameters resolves the conflict between the limited penetration of high-frequency probes and the low resolution of low-frequency probes, effectively identifying 0.3μm bond cracks within a depth of 100μm. Compared to traditional single-modal inspection solutions, this design increases the detection rate of deep defects by 50% and reduces the missed detection rate from 25% to 5%, covering the entire structure layer from silicon substrates to metal layers and dielectric layers.

[0030] In the multimodal image preprocessing stage, the highlight reflection noise of optical images, the ring artifacts of X-ray images and the scattering noise of ultrasound images are firstly eliminated by using a guided filtering and bilateral filtering composite algorithm ( Figure 9 , S1) performs noise reduction on the optical image, attenuating the noise in the highlight area while retaining the gradient information of the defect edge, so that the edge feature retention rate reaches 93%; it uses the iterative reconstruction algorithm (ART) to eliminate the ring artifacts of the X-ray image, and combines the wavelet threshold segmentation technology to separate the scattered noise and interlayer interface signal of the ultrasonic image, thereby increasing the signal-to-noise ratio of the microbubble defect by 2.5 times. Then, the non-rigid B-spline transform algorithm ( Figure 3 , S3), extracting optical markers, X-ray metal column features, and ultrasonic interlayer reflection interfaces as common benchmarks, performing geometric deformation correction on cross-modal images, and processing missing areas through the bicubic interpolation algorithm to achieve submicron registration accuracy (position deviation < 1μm), providing high-quality input data for subsequent deep learning models.

[0031] The self-supervised deep learning model deployed in the defect recognition module addresses the problems of scarcity of new defect annotation data and difficulty in identifying inter-layer defect correlations through comparative learning strategies ( Figure 5, U1) uses unlabeled multimodal data for pre-training, and combines the contrast loss function (Formula 4) to constrain the feature similarity of the same defect in different modalities, reducing the manual labeling cost by 70%. In the feature extraction stage, the attention focus network is connected after the ResNet50 backbone network, and the weight distribution formula is (7) Strengthen the characteristic response of micron-level defects, and increase the edge characteristic response strength of 1μm-level metal residue by 2 times; build an inter-layer defect correlation model based on a bidirectional LSTM network ( Figure 4 , R302), the cross-layer defect correlation probability is calculated by the correlation probability formula (3). When TSV voids are detected in the upper layer, the prediction accuracy of the lower layer bonding crack is improved by 45%, effectively avoiding the isolated missed detection problem of single-layer detection.

[0032] The process optimization module addresses the problems of low efficiency and high trial-and-error cost of traditional manual parameter adjustment. First, historical defect data and process parameters (bonding pressure, exposure time, deposition rate, etc.) are collected to construct a defect type-parameter combination correlation matrix ( Figure 13 , historical data statistics interface), identified high-frequency abnormal parameter combinations such as a 20% increase in interface crack rate when the bonding pressure is less than 80N. The process simulation model then predicted the impact of parameter adjustment on yield. When the bonding pressure was automatically adjusted from 80N to 84N, the defect rate was reduced to 1.2% within 10 wafers. Before the command was output, the safety verification module ( Figure 8 , U6) verifies the boundary conditions of the device status and environmental variables, and the redundant control module uses a triple node voting mechanism ( Figure 8 , R703) ensures 99.99% control signal reliability. This closed-loop mechanism reduces process adjustment response time from 30 minutes to 20 seconds, improves chip manufacturing yield by 4.7%, reduces trial-and-error costs by 60%, and enables minute-by-minute intelligent process optimization.

[0033] In this embodiment, dynamic adaptation of multimodal imaging parameters ( Figure 2 , R102-R104), cross-modal noise suppression and high-precision registration ( Figure 9 , S1-S3), self-supervised feature fusion, and reinforcement learning process optimization, systematically addressing core issues in 3D stacked chip inspection, including the penetration-resolution contradiction, insufficient algorithm robustness, and lack of closed-loop control. Actual measurements show that the system has a 99% accuracy rate for detecting 0.5μm-level TSV voids and 1μm-level bonding cracks, supporting high-speed inspection of 60 wafers per hour at the edge. Figure 15 , edge device performance parameters), significantly improving the intelligent detection level of advanced packaging production lines and providing a high-reliability technical solution for high-density integrated circuit manufacturing.

[0034] Example 2 provides an edge-end lightweight detection solution for high-density packaging production lines. To address the problems of low model inference efficiency (traditional deep learning model edge-end processing delay > 100ms), insufficient real-time fusion capability of multimodal data, and poor adaptability to complex process scenarios in existing systems in edge computing environments, this solution achieves high-speed deployment and flexible adaptation of the detection system through edge-cloud collaborative architecture, lightweight model design, and dynamic parameter adaptation technology.

[0035] The system adopts a distributed architecture of "real-time processing at the edge + deep analysis in the cloud" ( Figure 15 , U1-U2), where the edge device ( Figure 15 , U1) integrated lightweight multi-modal imaging module ( Figure 8 , U1) and pre-processing unit, equipped with low-power FPGA chip (computing power 20TOPS) and high-speed data interface (transmission rate 10Gbps). In view of the limitation of edge computing power, through knowledge distillation technology ( Figure 12 ) compresses the deep learning model, compressing the parameters of the ResNet50 backbone network to 1 / 4 of the original model and the floating point operations (FLOPs) from 4.1×10 9 Reduced to 9.8×10 8 , while maintaining 98% of the defect feature extraction capability. The edge pre-processing unit uses an adaptive filtering algorithm ( Figure 9 , S1) performs real-time noise reduction on optical, X-ray, and ultrasound images, and combines a fast feature point matching algorithm (registration time < 20ms / frame) to achieve geometric alignment of cross-modal images, with a data compression rate of 75%, effectively reducing cloud transmission pressure.

[0036] Example 3 provides a dynamic process optimization solution based on reinforcement learning. To address the problems of existing detection systems in complex process parameter adjustment, such as strong reliance on manual decision-making (adjustment cycle > 30 minutes), low prediction accuracy of multi-parameter coupling effects (yield fluctuation ±2%), and delayed response to abnormal working conditions, an intelligent closed loop from detection results to production control is achieved by constructing a multidimensional defect-process parameter correlation model, a dynamic optimization algorithm, and a real-time risk assessment mechanism.

[0037] The process optimization module of the system ( Figure 6 ,U4) integrates reinforcement learning decision engine and process simulation model, based on defect-process mapping database (DPMDB, Figure 13 , U1) Construct the correlation matrix between historical defect data and process parameters (bonding pressure, exposure time, deposition rate, etc.). For the interlayer bonding defects in 3D stacked chip manufacturing, the optimal adjustment strategy is first generated dynamically through formula (5), where is the immediate reward function (decrease in defect rate), The engine automatically identifies abnormal parameter combinations (e.g., bonding pressure 80N, temperature 180°C) and predicts through process simulation models that if the pressure is adjusted to 84N and the temperature is raised to 190°C, the defect rate can be reduced to 1.2% within 10 wafers. Figure 6 , S1).

[0038] In order to improve the optimization accuracy in multi-parameter coupling scenarios, the system introduces a hierarchical decision-making mechanism ( Figure 4 , R302): At the feature layer, the attention mechanism is used to strengthen the weight distribution of key parameters (such as the bonding pressure that affects the bonding strength between layers) (weight coefficient > 0.7); at the strategy layer, the experience replay technology (Replay Buffer) is used to store historical successful parameter adjustment cases (capacity 100,000), and transfer learning is used to accelerate the strategy convergence under new working conditions, reducing the first parameter optimization time from 15 minutes to 3 minutes. Real-time risk assessment module ( Figure 7 , U1) Simultaneously verify the safety boundaries of the adjustment plan, and build safety constraints based on variables such as equipment load (current < 80% of the rated value), ambient temperature and humidity (temperature 22±2℃, humidity 40±5%). If the bonding temperature adjustment exceeds the equipment upper limit (200℃), manual review is triggered to ensure the reliability of the process adjustment.

[0039] In the cross-process collaborative optimization scenario, the system combines the defect distribution heat map ( Figure 13 , R1201) and MES production data ( Figure 14 , U2), establish a full chain mapping of "defect detection → process tracing → parameter linkage adjustment". For example, when microbubble defects are detected in the TSV filling process of three consecutive batches of wafers, the system automatically associates them with the deposition rate (current value 5μm / s) and the precursor concentration (current value 95%). Through the reinforcement learning model, it recommends reducing the deposition rate to 4.5μm / s and increasing the concentration to 98%. Simulation verification shows that this type of defect rate can be reduced by 60%. The adjustment instructions are sent through the redundant control module ( Figure 8 , R703)’s triple-node voting mechanism (with an accuracy rate of 99.99%) is sent to production equipment, forming a fully closed-loop control of detection, analysis, and optimization.

[0040] Example 4 provides a multimodal imaging hardware collaboration solution based on real-time calibration. To address the imaging parameter drift (such as X-ray tube voltage attenuation ±5%, ultrasound probe frequency deviation ±2MHz) and inter-modal synchronization error (time deviation >10ms) caused by long-term operation of existing equipment, an embedded calibration self-test module and a hardware collaborative control algorithm are used to achieve high-precision calibration and real-time synchronization of the multimodal imaging system.

[0041] The multimodal imaging module ( Figure 8 , U1) built-in calibration self-test module ( Figure 2 , U5), including optical calibration plate (accuracy ±0.2μm), X-ray equivalent attenuation plate (thickness error ±1%) and ultrasonic standard test block (defect size standard ±0.1μm). Before the production line starts every day, the calibration self-check module automatically triggers the calibration process: Optical imaging submodule ( Figure 8 , U2) Use the laser rangefinder (±0.1μm) to perform three-dimensional modeling of the characteristic points of the calibration plate and generate the focal length-illuminance compensation coefficient (compensation accuracy ±3%); X-ray phase contrast imaging submodule ( Figure 8 , U3) calculates the theoretical penetration voltage based on formula (1), fits it with the actual imaging contrast, and dynamically corrects the voltage parameters (correction step ±1kV); Ultrasonic probe submodule ( Figure 8 , U4) The focus parameters of the acoustic beam are inverted by the standard test block, and the focus offset of the 25MHz probe is controlled within ±5μm.

[0042] During real-time detection, the hardware collaborative control algorithm ( Figure 2 , R101) monitors the temperature drift (sensor accuracy ±0.5°C) and vibration noise (gyroscope ±0.05°) of each mode in real time, predicts the parameter drift through Kalman filtering and dynamically compensates. When the X-ray tube temperature is detected to exceed 50°C, the duty cycle is automatically reduced to 70% and the water cooling cycle is started to ensure voltage stability of ±0.5%; if the vibration amplitude of the ultrasound probe is greater than 0.1g, the image stabilization algorithm is triggered ( Figure 9 , S1), the displacement error is controlled within ±2 pixels through inter-frame motion compensation.

[0043] Field measurements have shown that this embodiment improves the long-term accuracy of a multimodal imaging system by 80%, reduces the X-ray penetration voltage drift from ±5% to ±1%, controls the ultrasound probe frequency deviation to ±0.5 MHz, and maintains inter-modal time synchronization error below 5 ms. The calibration self-check process takes less than 3 minutes, a 90% improvement over traditional manual calibration. This effectively addresses the issue of hardware performance degradation during long-term operation and provides a stable data source for high-precision defect detection.

[0044] Example 5 provides a multimodal image enhancement solution that integrates a generative adversarial network (GAN). To address the problem that low-contrast defects (such as interlayer dielectric bubbles below 1μm and microcracks on TSV edges) are easily lost in traditional filtering methods (signal-to-noise ratio improvement <1.5 times), this solution achieves enhanced recognition of tiny defects through adversarial learning and cross-modal feature fusion.

[0045] The pre-processing execution module ( Figure 3 , U2) New GAN enhancement unit ( Figure 9, S4), including generator G and discriminator D: Generator G is based on U-Net architecture, and inputs the feature fusion map of ultrasound image and X-ray image ( Figure 5 , U2), outputs the enhanced inter-layer defect feature map; the discriminator D distinguishes the real defect image from the generated image through adversarial training, and improves the generator's ability to restore micro-defect edges (gradient < 5%).

[0046] In the defect enhancement stage, the ultrasonic image is first subjected to wavelet transform to extract the low-frequency interlayer signal, which is then fused with the phase gradient feature of the X-ray image at the channel level and input into the generator G to generate a high-contrast defect map ( Figure 9 , R806). For submicron scratches in optical images, the conditional GAN (cGAN) introduces focal length and lighting parameters as conditional inputs to generate clear images without reflection interference, which increases the edge response intensity of 0.8μm scratches by 3 times ( Figure 2 , R102). The enhanced multimodal images are then non-rigidly registered ( Figure 3 , S3), the overlap of defect features increased from 92% to 98%, providing more complete edge and texture information for subsequent deep learning models.

[0047] Example 6 provides a three-dimensional localization solution for interlayer defects based on a 3D convolutional neural network (3D-CNN). This solution addresses the problem of insufficient recognition of the three-dimensional spatial correlation of defects in the vertical interconnect structure of 3D stacked chips (e.g., the vertical spacing between TSV voids and underlying bonding cracks is less than 5μm). Through three-dimensional feature modeling and spatiotemporal correlation analysis, it achieves three-dimensional localization and chain risk assessment of multi-layer defects.

[0048] The defect recognition module ( Figure 4 , U2) deploy 3D-CNN backbone network ( Figure 10 , U3), input the pre-processed multimodal image sequence (layer spacing 0.5μm, a total of 20 layers), and extract the spatial context features of inter-layer defects (such as TSV diameter change rate, inter-layer medium density gradient) through 3D convolution kernel (3×3×3). Figure 5 , U3) introduces three-dimensional position encoding, which increases the model's sensitivity to defect positions in the Z-axis direction (stacking direction) by 40%, effectively identifying tiny defects with interlayer distances less than 2μm (such as composite defects of interlayer bubbles and metal column offsets).

[0049] Interlayer defect correlation analysis module ( Figure 4 , R302) Based on the improved bidirectional LSTM network, a three-dimensional defect correlation matrix is constructed (i, j are plane coordinates, k is the layer number), and the spatial correlation probability of cross-layer defects is calculated using formula (3). When a TSV void (diameter > 1.5 μm) is detected in the upper layer 10, the model automatically searches for the bonding interface at the corresponding position in the lower layer 9. If the crack length is > 2 μm, the correlation probability is > 0.85, triggering a multi-level defect warning ( Figure 7 , U2), and through the 3D visualization interface ( Figure 13 , R1204) presents a three-dimensional distribution of defects.

[0050] Example 7 provides a full-process traceability and visualization control solution based on digital twins. To address the problems of the existing system defect analysis results being out of touch with the production process (traceability delay > 1 hour) and the low efficiency of multi-source data integration (data association time > 10 minutes), a wafer-level digital twin model and visualization analysis platform are constructed to achieve deep integration of defect detection and process control.

[0051] The process optimization module ( Figure 6 , U4) integrated digital twin engine ( Figure 14 , U1), multimodal detection data based on wafer ID ( Figure 11 , U3), process parameters ( Figure 11 , U4) and MES production records ( Figure 14 , U2), builds a 3D digital twin model with more than 200 parameters (accuracy ±2%). When an abnormal bonding interface defect rate (>3%) is detected for a batch of wafers, the digital twin engine automatically backtracks the bonding pressure curve (fluctuation ±5N), ultrasonic inspection B-scan image (abnormal interlayer reflectivity area) and TSV filling thickness (deviation ±3μm) of the previous process, and uses the defect-process correlation matrix ( Figure 13 , R1202) locate the key influencing parameters (bonding pressure weight > 0.6).

[0052] Example 8 provides a blockchain solution for the integrity and traceability of test data. It addresses the problems of tampering risks in multimodal data storage in existing systems (data tampering detection delay > 24 hours) and low cross-departmental traceability efficiency (traceability link construction time > 30 minutes). Through the deep integration of blockchain technology and test data, the tamper-proof recording and efficient traceability of the entire test process data are achieved.

[0053] The data storage module ( Figure 11 , U1) integrated blockchain node ( Figure 1 , R5), using alliance chain architecture (number of nodes ≥ 5) for multimodal image data ( Figure 11 , U2), defect recognition results ( Figure 11 , U3) and process adjustment instructions ( Figure 11, U4) to store the evidence on the chain. During the data collection phase, the edge device ( Figure 15 , U1) performs hash operation (SHA-256) on the original image to generate a unique data fingerprint (length 256 bits), which is combined with the device ID ( Figure 2 , U1), detection timestamp (accuracy ±1ms) is packaged into blocks; pre-processed image features, defect coordinates and other structured data are transmitted through smart contracts ( Figure 6 , U4) is automatically associated with the corresponding block to ensure the timing consistency of the data chain (timestamp error < 5ms).

[0054] In the data traceability scenario, when it is necessary to verify the defect handling process of a batch of wafers (ID: WAFER_20250418_001), the blockchain browser ( Figure 13 , U3) Input wafer ID, the system automatically retrieves the associated blocks and presents the multimodal original images of the wafer in sequence ( Figure 9 , R801), preprocessing parameters ( Figure 3 , S1-S3), defect identification report ( Figure 4 , R304) and process adjustment records ( Figure 6 , R504), tracing delay <10 seconds. To address the risk of data tampering, blockchain nodes verify data integrity in real time through a consensus algorithm (PoS). If an abnormal hash value is detected (matching degree <99.99%), an alarm is immediately triggered and the abnormal block is locked ( Figure 7 , U2), ensuring data credibility of 99.999%.

[0055] Example 9 provides an energy efficiency optimization solution for edge computing devices. To address the problems of excessive power consumption (operating power consumption > 150W) and uneven distribution of computing resources (single-node load fluctuation > 40%) in existing edge detection equipment, dynamic voltage and frequency scaling (DVFS) and load balancing algorithms are used to achieve low-power and efficient operation of edge devices.

[0056] The edge device ( Figure 15 , U1) equipped with heterogeneous computing architecture (FPGA+ARM), integrated energy efficiency management module ( Figure 8 , U6) and load monitoring sensor (accuracy ±5%). In the data preprocessing stage, the energy efficiency management module uses formula (8) Dynamically adjust the FPGA voltage (V) and frequency (f): When low-complexity image data is detected (such as a defect-free wafer with less than 100 feature points), the voltage is reduced from 1.2V to 0.9V and the frequency is reduced from 200MHz to 150MHz, reducing power consumption by 40%. If processing images with complex interlayer defects (more than 500 feature points), the computing power is temporarily increased to the peak value (20TOPS), and then automatically returns to energy-saving mode (response time less than 2ms) after processing is completed.

[0057] Load balancing algorithm ( Figure 10 ,U4) Based on the real-time computing power utilization of edge nodes (threshold set at 70%) and the task queue length (threshold 50 frames), detection tasks are dynamically allocated through a polling mechanism: when the load of node A reaches 80%, subsequent tasks are automatically diverted to node B (load < 50%), ensuring that the load balance of each node is > 95%. Combined with the lightweight model after knowledge distillation ( Figure 12 , student model), the throughput of a single node at the edge reaches 60 slices / hour, the end-to-end delay is less than 100ms, and the operating power consumption is stable within 50W (70% lower than the traditional solution), meeting the energy efficiency requirements of 24-hour continuous detection.

[0058] Example 10 provides an automated testing solution covering hardware, algorithm, and system. To address the problems of time-consuming parameter calibration of the detection system when introducing a new packaging process (single process calibration > 2 hours) and lack of virtual verification capabilities (defect type coverage < 80%), a virtual simulation platform and a hardware-in-the-loop testing mechanism are constructed to achieve rapid verification and process adaptation of the detection system.

[0059] The system integrates a virtual simulation module ( Figure 14 , U1) and hardware-in-the-loop test platform ( Figure 1 , R9), generates virtual wafer data containing 100+ defect types based on 3D stacked chip process simulation software (accuracy ±2%) ( Figure 2 , R101), covering typical defects such as TSV voids (size 0.5-5μm), interlayer cracks (length 1-10μm), and metal residues (area 2-20μm²). During the test, virtual wafer data is input into the multimodal imaging module ( Figure 8 , U1), by simulating physical processes such as optical reflection, X-ray attenuation, and ultrasonic scattering ( Figure 2 , R102-R104), generate equivalent real detection images (with an error of less than 3% from the actual imaging) to verify the defect recognition algorithm ( Figure 4 , U2)’s generalization ability.

[0060] Hardware-in-the-loop testing mechanism ( Figure 10U3) A programmable attenuator (accuracy ±1%) was used to simulate hardware degradation scenarios such as X-ray tube aging and ultrasonic probe wear, testing the system's detection stability under parameter drift (voltage attenuation 5%, frequency deviation 2MHz). When validating detection solutions for new dielectric materials (such as boron nitride), simply updating material parameters (density, acoustic impedance) within the virtual simulation platform allows for rapid evaluation of the compatibility between imaging parameters (ultrasound frequency from 25MHz to 50MHz) and the algorithm model (feature extraction layer parameter adjustment), reducing the time it takes to introduce a new process from 72 hours to 8 hours.

[0061] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0062] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for automatically identifying internal defect images of 3D stacked chip wafers, characterized in that: The following steps are involved: S1: Uses multimodal imaging techniques including optical imaging, X-ray imaging, and ultrasound imaging to acquire wafer image data, and adjusts imaging parameters in real time based on the structural characteristics and material properties of different wafer layers, such as density, transmittance, and acoustic impedance of each layer. S2: Preprocessing the multimodal image data includes suppressing noise signals through layered filtering, improving the saliency of defect area features using a contrast enhancement algorithm, and performing spatial registration of cross-modal images to ensure geometric consistency of different imaging data; S3: The pre-processed multimodal image data is fed into a deep learning model, which extracts multi-level features through a self-supervised learning algorithm and uses an attention mechanism to identify the location and type of defects within the wafer. S4: Correlate and analyze the defect identification results with historical process parameters, generate process optimization suggestions based on preset decision strategies, and transmit the optimization suggestions to the manufacturing execution system through a programmable interface.

2. The method according to claim 1, characterized in that The real-time adjustment of imaging parameters in step S1 further includes: The incident angle and polarization mode of the optical imaging light source are autonomously adjusted according to the reflection characteristics of the wafer surface to light; the phase sensitivity and energy parameters of X-ray imaging are adjusted in combination with the penetration characteristics of the material inside the wafer; and the probe frequency and sound beam focusing parameters of different detection depths are adapted based on the propagation attenuation characteristics of ultrasound.

3. The method according to claim 1, characterized in that The spatial registration operation of the cross-modal images in step S2 further includes: Common feature points in different imaging data are extracted as registration benchmarks; spatial deformation caused by different imaging principles is corrected through non-rigid transformation algorithms; multi-resolution interpolation compensation is performed on pixel data in missing matching areas to maintain image integrity.

4. The method according to claim 1, wherein The self-supervised learning algorithm method of the deep learning model in step S3 further includes: In the pre-training stage, a comparative learning strategy is used to constrain the feature similarity of unlabeled data to generate a robust low-level feature extraction module; in the fine-tuning stage, the model is optimized for the target task using labeled data, and the ability to recognize tiny defects is enhanced through attention mapping; in the inference stage, inter-layer defect association rules are established in combination with the wafer-level structural features, and a structured inspection report is output.

5. The method according to claim 1, wherein The process of generating process optimization suggestions by presetting the decision strategy in step S4 also includes: Construct a historical data association matrix between defect types and process parameters to identify abnormal parameter combinations; use simulation models to predict the potential impact of different parameter adjustment schemes on yield; and output the scheme whose predicted results meet the preset optimization goals as executable instructions.

6. The method according to claim 1, characterized in that Further including: Before issuing a parameter adjustment instruction, the safety boundary conditions of the instruction are verified based on the device operating status and environmental variables; if the verification result is a risk overflow, the calibration mode is triggered or the instruction is transferred to the manual review process.

7. The method according to claim 1, characterized in that The hierarchical filtering and noise signal suppression in step S2 using a contrast enhancement algorithm further comprises: A composite noise reduction process of guided filtering and bilateral filtering is performed on the highlight reflective areas in the optical image; an iterative reconstruction algorithm is used to perform spatial filtering to eliminate the ring artifacts in the X-ray image; and wavelet threshold segmentation and feature filtering are performed to separate the scattered noise in the ultrasound imaging.

8. The method according to claim 1, characterized in that The step S4 further includes: Generate a process anomaly tracing path based on the defect distribution heat map and integrate the path into the visual analysis interface; monitor the drift of key parameters in real time, and trigger an alarm or shutdown command when the drift exceeds the adaptive threshold.

9. A system for automatically identifying internal defects of 3D stacked chip wafers for executing the method according to any one of claims 1 to 8, characterized in that: It includes the following modules connected in sequence: Multimodal imaging module: integrates optical imaging equipment, X-ray imager and ultrasonic probe array, and is configured to dynamically adjust imaging parameters according to the wafer hierarchical structure and material properties and output multi-source image data; preprocessing execution module: connected to the output end of the multimodal imaging module, includes a noise suppression unit, a contrast enhancement unit and a registration algorithm unit, and is configured to perform denoising, enhancement and geometric alignment operations on the imaging data; defect recognition module: connected to the output end of the preprocessing execution module, deploys a deep learning model and an attention focusing network, and is configured to analyze image data and locate defect positions; process optimization module: connected to the output end of the defect recognition module, integrates a decision engine and a communication interface, and is configured to generate process adjustment instructions and transmit the instructions to the production control system.

10. The system according to claim 9, characterized in that Further including: Calibration self-test module: embedded in the multimodal imaging module, used to regularly perform imaging parameter calibration and hardware status monitoring; safety verification module: connected before the process optimization module, used to evaluate the execution risk of parameter adjustment plans and screen safety instructions; redundant control module: deployed in the instruction output link of the process optimization module, and ensures the reliability of control signals through a multi-node voting mechanism.

Images