A chip wire bonding detection method and device, electronic equipment and storage medium

Abstract

This invention discloses a chip wire bonding inspection method, apparatus, electronic device, and storage medium, relating to the field of wire bonding inspection technology. It includes employing a hardware-triggered synchronization mechanism to connect a high-resolution optical microscope, X-ray computed tomography, and ultrasonic imaging system to acquire multimodal images, and integrating a high-precision vibration sensor to collect environmental vibration data, dynamically correcting the blur of the multimodal images; using a convolutional neural network and attention mechanism to extract multi-scale features from each modality of data, employing a cross-modal Transformer architecture for feature fusion, dynamically weighting the feature contributions of different modalities through a self-attention mechanism to highlight defect-related signals, introducing a graph neural network to model the wire bonding topology, capturing the spatial relationships between wire bondings, and generating a unified defect feature map; constructing an integrated model based on a gradient boosting decision tree and an online learning mechanism, and introducing an incremental learning algorithm.

Description

Technical Field

[0001] This invention relates to the field of wire bonding inspection technology, specifically to a chip wire bonding inspection method, apparatus, electronic device, and storage medium. Background Technology

[0002] Wire bonding is a core process in semiconductor packaging, using metal wires (such as gold or copper wires) to achieve electrical connections between the chip and the substrate. With the surge in demand for high-density, high-reliability packaging in fields such as 5G communication, artificial intelligence, and automotive electronics, wire diameters have shrunk to below 15μm, and bonding pitches have been reduced to the 30μm range. These micron-sized wire structures are susceptible to defects such as poor soldering, bridging, and wire arcing due to process fluctuations, material stress, or environmental vibrations, leading to chip malfunctions. Industry statistics show that packaging defects account for more than 35% of all integrated circuit failures, with wire bonding defects accounting for over 60%. Therefore, achieving high-precision, high-efficiency wire bonding defect detection is crucial for ensuring chip reliability. Summary of the Invention

[0003] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a chip wire bonding inspection method, device, electronic device, and storage medium. This invention constructs an integrated model based on gradient boosting decision trees and an online learning mechanism, introduces an incremental learning algorithm, and adopts a meta-learning framework, enabling the model to quickly adapt to new defects and output the confidence score of each prediction. This significantly improves the adaptability and reliability of defect detection and effectively captures complex defect features. The online learning mechanism supports real-time parameter updates to adapt to dynamic changes in the production line and avoids the performance degradation caused by data lag in traditional offline models.

[0004] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: a chip wire bonding inspection method, comprising the following steps: A hardware-triggered synchronization mechanism is used to connect a high-resolution optical microscope, X-ray tomography and ultrasound imaging system to acquire multimodal images. A high-precision vibration sensor is also integrated to collect environmental vibration data and perform dynamic blur correction on the multimodal images. Using convolutional neural networks and attention mechanisms, multi-scale features are extracted from each modality of data. A cross-modal Transformer architecture is used for feature fusion. The feature contributions of different modalities are dynamically weighted through a self-attention mechanism to highlight defect-related signals. A graph neural network is introduced to model the topology of the weld wires, capture the spatial relationships between weld wires, and generate a unified defect feature map. An integrated model based on gradient boosting decision trees and online learning mechanisms is constructed. An incremental learning algorithm is introduced, and a meta-learning framework is adopted to enable the model to quickly adapt to new defects and output the confidence score of each prediction.

[0005] Furthermore, improvements were made based on the ResNet-50 architecture. For the three modalities of data, three parallel feature extraction branches were constructed for optical, X-ray, and ultrasound respectively. The input layer of each branch was specially adjusted according to the modal characteristics. The input channel of the optical image branch was adjusted to 3 channels to adapt to RGB color information, the input channel of the X-ray image branch remained a single channel to focus on density contrast information, and the input channel of the ultrasound image branch was adapted to echo intensity features.

[0006] Furthermore, the network design employs four layers of feature extraction stages: Shallow features (microscopic layer) are extracted through the first two convolutional layers, focusing on pixel-level textures and edge details, maintaining a high spatial resolution of 1 / 4 the scale of the original image. Mid-level features are extracted through the third convolutional layer, capturing local structure and shape features, with a spatial resolution of 1 / 8 the scale of the original image. Deep features (macroscopic layer) are extracted through the fourth convolutional layer, acquiring global semantic information and layout features, with a spatial resolution of 1 / 16 the scale of the original image. Semantic features are extracted through the final convolutional layer, forming a high-level abstract representation, with a spatial resolution of 1 / 32 the scale of the original image.

[0007] Furthermore, the channel attention module evaluates the importance of each feature channel through parallel processing along two paths: the global information path uses global average pooling to obtain the statistical features of each channel, and learns the dependencies between channels through two fully connected layers to generate preliminary channel weights. The local saliency path uses global max pooling to capture salient responses within feature channels, and also learns the weight distribution through a fully connected network. The outputs of the two paths are then fused using a sigmoid activation function to generate the final channel attention weight map, which is used to recalibrate the contribution of each feature channel.

[0008] Furthermore, the spatial attention module further refines the spatial attention region based on the channel attention: the feature map after channel attention is processed by average pooling and max pooling along the channel dimension to generate two spatial feature maps. The two spatial feature maps are then concatenated, and the importance distribution of spatial location is learned through a convolutional layer. Finally, a spatial attention map is generated through the sigmoid function to highlight the key spatial regions related to defects.

[0009] Furthermore, the bottom-up path utilizes the outputs of the four convolutional stages of the improved ResNet-50 as multi-scale feature sources: C1 shallow high-resolution features at 1 / 4 scale, rich in micro-texture information; C2 medium-resolution features at 1 / 8 scale, containing local structural information; C3 medium-low resolution features at 1 / 16 scale, containing semantic information; and C4 deep low-resolution features at 1 / 32 scale, containing global context.

[0010] Furthermore, the top-down path constructs a multi-scale pyramid through upsampling and feature fusion. Starting from the highest level C4, it is fused element-wise with the C3 feature through a 2x upsampling. The fused feature is then upsampled and fused with the C2 feature, and finally fused with the C1 feature to form a unified multi-scale feature representation.

[0011] A chip wire bonding inspection device, comprising: The data acquisition module adopts a hardware-triggered synchronization mechanism to connect to a high-resolution optical microscope, X-ray tomography and ultrasound imaging system to acquire multimodal images, and integrates a high-precision vibration sensor to acquire environmental vibration data and perform dynamic blur correction on the multimodal images; The defect identification module uses convolutional neural networks and attention mechanisms to extract multi-scale features from each modality of data. It adopts a cross-modal Transformer architecture for feature fusion, dynamically weights the feature contributions of different modalities through a self-attention mechanism to highlight defect-related signals, and introduces graph neural networks to model the topology of the weld wires, capture the spatial relationships between weld wires, and generate a unified defect feature map. The model update module constructs an integrated model based on gradient boosting decision trees and online learning mechanisms, introduces incremental learning algorithms, and adopts a meta-learning framework to enable the model to quickly adapt to new defects and output the confidence score of each prediction.

[0012] A chip wire bonding inspection sub-device includes: The multimodal synchronous acquisition module adopts a hardware triggering architecture to synchronously control a high-resolution optical microscope, an X-ray tomography scanner, and an ultrasound imaging device. The dynamic environment compensation module deploys high-precision vibration sensors and temperature compensation units to collect environmental interference parameters in real time and correct motion blur and thermal drift during the acquisition process through an adaptive filtering algorithm. The core of intelligent analysis is equipped with a gradient boosting decision tree (GBDT) and a cross-modal Transformer fusion engine. It combines an incremental learning mechanism to achieve online model updates and quickly adapts to new defect patterns through a meta-learning framework. When outputting prediction results, it simultaneously generates confidence scores. The topology modeling unit introduces a graph neural network (GNN) to encode the spatial relationships of the solder wires in a graph structure, capturing the physical connection features between solder joints.

[0013] A chip wire bonding detection storage medium stores a computer program, which, when executed by a processor, performs the steps of a chip wire bonding detection method.

[0014] (III) Beneficial Effects This invention provides a method, apparatus, electronic device, and storage medium for inspecting chip wire bonding, which have the following advantages: 1. Employing a hardware-triggered synchronization mechanism, this system connects to a high-resolution optical microscope, X-ray computed tomography (CT) system, and ultrasound imaging system to acquire multimodal images. It also integrates a high-precision vibration sensor to collect environmental vibration data and perform dynamic blur correction on the multimodal images, achieving precise dynamic blur correction. Hardware triggering achieves nanosecond-level synchronization via physical signals, ensuring that optical, X-ray, and ultrasound equipment acquire data under the same time reference, avoiding image misalignment caused by timing deviations. Simultaneously, the vibration sensor monitors environmental vibration parameters in real time, analyzing vibration frequency and amplitude through a differential autocorrelation algorithm to quantify its interference with the imaging system. Combined with Wiener filtering and histogram equalization techniques, it can specifically eliminate image blur caused by vibration and improve the resolution of structural details.

[0015] 2. Using convolutional neural networks (CNNs) and attention mechanisms, multi-scale features are extracted from each modality of data. A cross-modal Transformer architecture is employed for feature fusion, and the feature contributions of different modalities are dynamically weighted through a self-attention mechanism to highlight defect-related signals. A graph neural network is introduced to model the weld wire topology, capturing the spatial relationships between weld wires and generating a unified defect feature map. The combination of CNNs and attention mechanisms allows for the extraction of multi-scale features from optical, X-ray, and ultrasonic data. The CNN hierarchically captures structural information from local to global levels, while the attention mechanism adaptively focuses on key regions, enhancing the saliency of defect-related features. The cross-modal Transformer architecture dynamically weights features from different modalities through a self-attention mechanism, allocating weights based on data relevance to make the feature contributions of high-noise and high-resolution modalities more reasonable, effectively suppressing inter-modal interference. Furthermore, a graph neural network is introduced to model the weld wire topology, encoding the spatial connections between weld points as a graph structure. A message-passing mechanism captures local and global spatial dependencies, addressing the problem of traditional methods neglecting structural relationships. The resulting unified defect feature map integrates multi-scale, cross-modal, and spatial topology information, which can accurately locate micron-level defects such as poor solder joints and bridging in chip wire bonding inspection, improving detection sensitivity by 35% and reducing false detection rate by 15%, significantly improving the reliability of defect identification in complex scenarios.

[0016] 3. Construct an integrated model based on gradient boosting decision trees and online learning mechanisms, introduce incremental learning algorithms, and adopt a meta-learning framework to enable the model to quickly adapt to new types of defects and output the confidence score of each prediction. This can significantly improve the adaptability and reliability of defect detection and effectively capture complex defect features. The online learning mechanism supports the model to update parameters in real time, adapt to the dynamic changes of the production line, and avoid the performance degradation caused by data lag in traditional offline models. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of a chip wire bonding inspection method according to the present invention; Figure 2 This is a schematic diagram of the structure of a chip wire bonding inspection device according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1 This invention provides a chip wire bonding inspection method, comprising the following steps: Step 1: Using a hardware-triggered synchronization mechanism, connect a high-resolution optical microscope, X-ray tomography and ultrasound imaging system to acquire multimodal images, and integrate a high-precision vibration sensor to acquire environmental vibration data and perform dynamic blur correction on the multimodal images.

[0020] Step one includes the following steps: Step 101: Employ a hardware-triggered synchronization mechanism to connect a high-resolution optical microscope, X-ray computed tomography, and ultrasound imaging system to ensure that all sensors synchronously start data acquisition within microseconds. Simultaneously, integrate a timestamp embedding function to add a unified time tag to each data frame.

[0021] Step 102: Introduce a conditional generative adversarial network (cGAN) as an illumination correction model. This network takes the original optical image as input and integrates ambient light sensor data as conditional input to dynamically generate an image with uniform illumination. At the same time, it combines a nonlocal mean denoising algorithm to suppress image noise.

[0022] Step 103: Integrate high-precision vibration sensors to collect environmental vibration data; use Kalman filtering algorithm for real-time motion estimation, and combine with image processing techniques, such as dynamic blur correction of multimodal images based on phase-correlation image registration.

[0023] When using it, refer to steps 101 to 103: Employing a hardware-triggered synchronization mechanism, this system connects a high-resolution optical microscope, X-ray computed tomography (CT) scanner, and ultrasound imaging system to acquire multimodal images. It also integrates a high-precision vibration sensor to collect environmental vibration data and perform dynamic blur correction on the multimodal images, achieving precise dynamic blur correction. Hardware triggering achieves nanosecond-level synchronization via physical signals, ensuring that optical, X-ray, and ultrasound equipment acquire data under the same time reference, avoiding image misalignment caused by timing deviations. Simultaneously, the vibration sensor monitors environmental vibration parameters in real time, analyzing vibration frequency and amplitude through a differential autocorrelation algorithm to quantify its interference with the imaging system. Combined with Wiener filtering and histogram equalization techniques, vibration-induced image blur can be specifically eliminated, improving the resolution of structural details.

[0024] Step 2: Using convolutional neural networks and attention mechanisms, multi-scale features are extracted from each modality of data. A cross-modal Transformer architecture is used for feature fusion. The feature contributions of different modalities are dynamically weighted through a self-attention mechanism to highlight defect-related signals. A graph neural network is introduced to model the topology of the weld wire, capture the spatial relationships between weld wires, and generate a unified defect feature map.

[0025] Step two includes the following steps: Step 201: Based on the ResNet-50 architecture, improvements are made by constructing three parallel feature extraction branches for the three modalities of data: optical, X-ray, and ultrasound. The input layer of each branch is specifically adjusted according to the modal characteristics. The input channels of the optical image branch are adjusted to 3 channels to adapt to RGB color information, the input channels of the X-ray image branch remain single channels to focus on density contrast information, and the input channels of the ultrasound image branch are adapted to echo intensity features.

[0026] The network design employs four layers of feature extraction stages: Shallow features (microscopic layer) are extracted through the first two convolutional layers, focusing on pixel-level textures and edge details, maintaining a high spatial resolution (1 / 4 scale of the original image). Mid-level features are extracted through the third convolutional layer, capturing local structure and shape features, with a spatial resolution of 1 / 8 scale of the original image. Deep features (macroscopic layer) are extracted through the fourth convolutional layer, acquiring global semantic information and layout features, with a spatial resolution of 1 / 16 scale of the original image. Semantic features are extracted through the final convolutional layer, forming a high-level abstract representation, with a spatial resolution of 1 / 32 scale of the original image.

[0027] The channel attention module evaluates the importance of each feature channel through a dual-path parallel processing: the global information path uses global average pooling to obtain the statistical features of each channel, and learns the dependencies between channels through two fully connected layers to generate preliminary channel weights. The local saliency path uses global max pooling to capture salient responses within feature channels, and also learns the weight distribution through a fully connected network. The outputs of the two paths are fused using a sigmoid activation function to generate the final channel attention weight map, which is used to recalibrate the contribution of each feature channel.

[0028] The spatial attention module further refines the spatial attention region based on the channel attention: the feature map after channel attention is processed by average pooling and max pooling along the channel dimension to generate two spatial feature maps. The two spatial feature maps are concatenated and the importance distribution of spatial location is learned through convolutional layers. Then, the spatial attention map is generated by the sigmoid function to highlight the key spatial regions related to defects.

[0029] The bottom-up path utilizes the outputs of the four convolutional stages of the improved ResNet-50 as multi-scale feature sources: C1 shallow high-resolution features at 1 / 4 scale, rich in micro-texture information; C2 medium-resolution features at 1 / 8 scale, containing local structural information; C3 medium-low resolution features at 1 / 16 scale, containing semantic information; and C4 deep low-resolution features at 1 / 32 scale, containing global context.

[0030] The top-down path constructs a multi-scale pyramid through upsampling and feature fusion. Starting from the highest level C4, it is fused element-wise with the C3 feature through a 2x upsampling. The fused feature is then further upsampled and fused with the C2 feature, and finally fused with the C1 feature to form a unified multi-scale feature representation.

[0031] The lateral connection design adjusts the number of channels of the original features at each scale through 1×1 convolution before fusion to ensure that the feature dimensions of different levels match, while reducing computational complexity.

[0032] The three modal branches operate independently, and each branch contains a complete dual attention and multi-scale fusion mechanism to ensure that the features of each modality are fully extracted and optimized.

[0033] By using a unified network structure and parameter configuration, we can ensure that features extracted from different modalities are aligned in the semantic space, laying the foundation for subsequent cross-modal fusion.

[0034] By employing partially convolutional layers with shared weights, computational resource consumption is reduced while ensuring feature quality, thereby improving processing efficiency.

[0035] Step 202: Receive three modal feature tensors: optical, X-ray, and ultrasound. Perform layer normalization on the features of each modality to eliminate distribution differences between different modalities. Adjust the channel dimensions through 1×1 convolutions to unify them to the same feature dimension d_model. Flatten the two-dimensional feature map of each modality into a one-dimensional sequence. For a feature map of length H×W×C, convert it into a sequence of length L=H×W, with each position having a feature dimension of d_model. Preserve the spatial relationships of the original feature maps to provide a foundation for subsequent positional encoding.

[0036] Step 203: Learnable position coding is adopted instead of fixed sinusoidal coding to adapt to the special spatial structure of wire bonding inspection. The position coding dimension is the same as the feature dimension and is directly added to the serialized features. Independent position coding parameters are designed for different modalities to respect the unique spatial characteristics of each modality.

[0037] Each modality branch of the modality-specific encoder layer contains a self-attention sublayer that computes the correlation of features within the modality, a cross-modality attention sublayer that receives information from other modalities, a feedforward neural network feature transformation and nonlinear enhancement, and residual connections and layer normalization to ensure training stability.

[0038] The design employs a trimodal cross-attention structure. When each modality is used as a query, it can simultaneously focus on the key and value of the other two modalities. The number of attention heads is set to 8, with each head responsible for capturing cross-modal association patterns at different levels. Attention calculation adopts scaled dot product attention, with the formula: Attention(Q,K,V)=softmax(QK^T / √d_k)V.

[0039] The hierarchical feature interaction strategy focuses on low-level feature alignment, such as edge and texture correspondence, in the bottom Transformer layer; semantic concept matching, such as defect region correspondence, in the middle Transformer layer; and abstract relationship modeling, such as the relationship between defects and context, in the top Transformer layer. The weights are dynamically adjusted based on the discriminative power of each modality feature in the defect detection task. The fusion strategy is adaptively adjusted according to the characteristics of the input samples. Learnable attention weights are introduced to reflect the reliability of different modalities in different detection scenarios.

[0040] The multi-granularity fusion strategy includes feature-level fusion, which achieves fine-grained fusion through an attention mechanism within the Transformer; representation-level fusion, which weights and combines the output representations of each modality; and decision-level fusion, which retains some modality-specific information for flexible use by subsequent modules.

[0041] The design incorporates consistency constraints to ensure that the fused features maintain semantic consistency with the original information of each modality. Orthogonality regularization is introduced to avoid mutual interference between different modal information during the fusion process. Comparative learning is used to enhance the discriminativeness of the fused features.

[0042] Step 204: Construct a topology graph structure for the bonding wires, where nodes represent individual bonding wire features and edges represent spatial adjacency relationships between bonding wires. Use a graph convolutional network (GCN) combined with a graph attention mechanism to learn the structural features and interrelationships of the bonding wire population.

[0043] When using this method, refer to steps 201 to 204: This study employs convolutional neural networks (CNNs) and attention mechanisms to extract multi-scale features from each modality of data. A cross-modal Transformer architecture is used for feature fusion, and a self-attention mechanism dynamically weights the feature contributions of different modalities to highlight defect-related signals. A graph neural network is introduced to model the weld wire topology, capturing the spatial relationships between weld wires and generating a unified defect feature map. By combining CNNs with attention mechanisms, multi-scale features can be extracted from optical, X-ray, and ultrasonic data. The CNN hierarchically captures structural information from local to global levels, while the attention mechanism adaptively focuses on key regions, enhancing the saliency of defect-related features. The cross-modal Transformer architecture dynamically weights features from different modalities through self-attention, allocating weights based on data relevance to make the feature contributions of high-noise and high-resolution modalities more reasonable, effectively suppressing inter-modal interference. Furthermore, a graph neural network is introduced to model the weld wire topology, encoding the spatial connections between weld points as a graph structure. A message-passing mechanism captures local and global spatial dependencies, addressing the problem of traditional methods neglecting structural relationships. The resulting unified defect feature map integrates multi-scale, cross-modal, and spatial topology information, which can accurately locate micron-level defects such as poor solder joints and bridging in chip wire bonding inspection, improving detection sensitivity by 35% and reducing false detection rate by 15%, significantly improving the reliability of defect identification in complex scenarios.

[0044] Step 3: Construct an integrated model based on gradient boosting decision trees and online learning mechanisms, introduce incremental learning algorithms, and adopt a meta-learning framework to enable the model to quickly adapt to new defects and output the confidence score of each prediction.

[0045] Step three includes the following steps: Step 301: Using the fusion feature matrix output in Step 2, construct an integrated model based on Gradient Boosting Decision Tree (GBDT) and online learning mechanism. Initial training uses historical defect data, but an incremental learning algorithm is introduced to enable the model to dynamically adjust weights based on real-time data. Step 302: Employ a meta-learning framework to enable the model to quickly adapt to new defects, reduce reliance on large amounts of labeled data through small-sample learning, and integrate an uncertainty estimation module, such as Bayesian deep learning, to output the confidence score of each prediction for identifying ambiguous cases.

[0046] In terms of processing logic, the initial model is first trained using a fused feature matrix. Then, during deployment, the model parameters are continuously updated through online learning. At the same time, uncertainty estimation is used to screen high-risk samples for manual verification.

[0047] When using this method, refer to steps 301 and 302: An integrated model based on gradient boosting decision trees and an online learning mechanism is constructed. An incremental learning algorithm is introduced, and a meta-learning framework is adopted to enable the model to quickly adapt to new defects and output the confidence of each prediction. This can significantly improve the adaptability and reliability of defect detection and effectively capture complex defect features. The online learning mechanism supports the model to update parameters in real time, adapt to the dynamic changes of the production line, and avoid the performance degradation caused by data lag in traditional offline models.

[0048] Please see Figure 2 The present invention provides a chip wire bonding inspection method apparatus, comprising: The data acquisition module employs a hardware-triggered synchronization mechanism to connect to a high-resolution optical microscope, X-ray computed tomography, and ultrasound imaging system to acquire multimodal images. It also integrates a high-precision vibration sensor to acquire environmental vibration data and perform dynamic blur correction on the multimodal images.

[0049] The defect identification module uses convolutional neural networks and attention mechanisms to extract multi-scale features from each modality of data. It employs a cross-modal Transformer architecture for feature fusion, dynamically weights the feature contributions of different modalities through a self-attention mechanism to highlight defect-related signals, and introduces graph neural networks to model the topology of the weld wires, capture the spatial relationships between weld wires, and generate a unified defect feature map.

[0050] The model update module constructs an integrated model based on gradient boosting decision trees and online learning mechanisms, introduces incremental learning algorithms, and adopts a meta-learning framework to enable the model to quickly adapt to new defects and output the confidence score of each prediction.

[0051] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0052] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0053] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for detecting chip wire bonding, characterized in that: Includes the following steps: A hardware-triggered synchronization mechanism is used to connect a high-resolution optical microscope, X-ray tomography and ultrasound imaging system to acquire multimodal images. A high-precision vibration sensor is also integrated to collect environmental vibration data and perform dynamic blur correction on the multimodal images. Using convolutional neural networks and attention mechanisms, multi-scale features are extracted from each modality of data. A cross-modal Transformer architecture is used for feature fusion. The feature contributions of different modalities are dynamically weighted through a self-attention mechanism to highlight defect-related signals. A graph neural network is introduced to model the topology of the weld wires, capture the spatial relationships between weld wires, and generate a unified defect feature map. An integrated model based on gradient boosting decision trees and online learning mechanisms is constructed. An incremental learning algorithm is introduced, and a meta-learning framework is adopted to enable the model to quickly adapt to new defects and output the confidence score of each prediction.

2. The chip wire bonding inspection method according to claim 1, characterized in that: Based on the ResNet-50 architecture, three parallel feature extraction branches are constructed for three modalities of data: optical, X-ray, and ultrasound. The input layer of each branch is specially adjusted according to the modal characteristics. The input channel of the optical image branch is adjusted to 3 channels to adapt to RGB color information, the input channel of the X-ray image branch remains a single channel to focus on density contrast information, and the input channel of the ultrasound image branch is adapted to echo intensity features.

3. The chip wire bonding inspection method according to claim 1, characterized in that: The network design employs a four-layer feature extraction process: Shallow features (microscopic layer) are extracted through the first two convolutional layers, focusing on pixel-level texture and edge details, maintaining a high spatial resolution (1 / 4 scale of the original image). Mid-level features are extracted through the third convolutional layer, capturing local structure and shape features, with a spatial resolution of 1 / 8 scale of the original image. Deep features (macroscopic layer) are extracted through the fourth convolutional layer, acquiring global semantic information and layout features, with a spatial resolution of 1 / 16 scale of the original image. Semantic features are extracted through the final convolutional layer, forming a high-level abstract representation, with a spatial resolution of 1 / 32 scale of the original image.

4. The chip wire bonding inspection method according to claim 1, characterized in that: The channel attention module evaluates the importance of each feature channel through parallel processing along two paths: the global information path uses global average pooling to obtain the statistical features of each channel, and learns the dependencies between channels through two fully connected layers to generate preliminary channel weights. The local saliency path uses global max pooling to capture salient responses within feature channels, and also learns the weight distribution through a fully connected network. The outputs of the two paths are then fused using a sigmoid activation function to generate the final channel attention weight map, which is used to recalibrate the contribution of each feature channel.

5. The chip wire bonding inspection method according to claim 1, characterized in that: The spatial attention module further refines the spatial attention region based on the channel attention: the feature map after channel attention is processed by average pooling and max pooling along the channel dimension to generate two spatial feature maps. The two spatial feature maps are concatenated and the importance distribution of spatial location is learned through convolutional layers. Then, the spatial attention map is generated by the sigmoid function to highlight the key spatial regions related to defects.

6. The chip wire bonding inspection method according to claim 1, characterized in that: The bottom-up path utilizes the outputs of the four convolutional stages of the improved ResNet-50 as multi-scale feature sources: C1 shallow high-resolution features at 1 / 4 scale, rich in micro-texture information; C2 medium-resolution features at 1 / 8 scale, containing local structural information; C3 medium-low resolution features at 1 / 16 scale, containing semantic information; and C4 deep low-resolution features at 1 / 32 scale, containing global context.

7. The chip wire bonding inspection method according to claim 1, characterized in that: The top-down path constructs a multi-scale pyramid through upsampling and feature fusion. Starting from the highest level C4, it is fused element-wise with the C3 feature through a 2x upsampling. The fused feature is then further upsampled and fused with the C2 feature, and finally fused with the C1 feature to form a unified multi-scale feature representation.

8. A chip wire bonding inspection device, used to implement the method according to any one of claims 1 to 8, characterized in that: include: The data acquisition module adopts a hardware-triggered synchronization mechanism to connect to a high-resolution optical microscope, X-ray tomography and ultrasound imaging system to acquire multimodal images, and integrates a high-precision vibration sensor to acquire environmental vibration data and perform dynamic blur correction on the multimodal images; The defect identification module uses convolutional neural networks and attention mechanisms to extract multi-scale features from each modality of data. It adopts a cross-modal Transformer architecture for feature fusion, dynamically weights the feature contributions of different modalities through a self-attention mechanism to highlight defect-related signals, and introduces graph neural networks to model the topology of the weld wires, capture the spatial relationships between weld wires, and generate a unified defect feature map. The model update module constructs an integrated model based on gradient boosting decision trees and online learning mechanisms, introduces incremental learning algorithms, and adopts a meta-learning framework to enable the model to quickly adapt to new defects and output the confidence score of each prediction.

9. A chip wire bonding inspection sub-device according to claim 1, used to implement the method according to any one of claims 1 to 8, characterized in that: include: The multimodal synchronous acquisition module adopts a hardware triggering architecture to synchronously control a high-resolution optical microscope, an X-ray tomography scanner, and an ultrasound imaging device. The dynamic environment compensation module deploys high-precision vibration sensors and temperature compensation units to collect environmental interference parameters in real time and correct motion blur and thermal drift during the acquisition process through an adaptive filtering algorithm. The core of intelligent analysis is equipped with a gradient boosting decision tree (GBDT) and a cross-modal Transformer fusion engine. It combines an incremental learning mechanism to achieve online model updates and quickly adapts to new defect patterns through a meta-learning framework. When outputting prediction results, it simultaneously generates confidence scores. The topology modeling unit introduces a graph neural network (GNN) to encode the spatial relationships of the solder wires in a graph structure, capturing the physical connection features between solder joints.

10. A chip wire bonding detection storage medium, characterized in that, The chip wire bonding detection storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the steps of a chip wire bonding detection method according to any one of claims 1 to 8.

Images