A method and system for automatically detecting a solder ball bubble defect of a ball grid array package chip

Abstract

The application discloses a kind of ball grid array package chip solder ball bubble defect automatic detection method and system, it is related to chip intelligent detection technical field, the method is guided by designing semantic cross-layer fusion framework, realizes multi-scale feature balanced fusion in neck network by introducing feature alignment and redistribution module, strengthens small target details and semantic information using local semantic enhancement module, and the positioning accuracy of bubble is optimized in combination with smooth geometry positioning loss function.The application significantly improves the detection capability of small, fuzzy and occluded bubbles, has strong robustness and generalization, and can be widely used in solder ball bubble, material porosity and surface defect industrial vision detection scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent chip inspection technology, and in particular to an automatic detection method and system for solder ball bubble defects in ball grid array (BGA) packaged chips. Background Technology

[0002] As electronic packaging technology advances towards higher density and performance, ball grid array (BGA) packaging has become one of the mainstream forms of integrated circuit packaging due to its high pin density and excellent thermal and electrical performance. However, during the soldering process, BGA chips are prone to bubble defects forming inside the solder balls or at the interface. These bubbles can seriously affect the mechanical strength, electrical connectivity, and long-term reliability of the solder joint, and even lead to premature device failure. Therefore, efficient and accurate detection of bubbles inside BGA solder balls is a key step in ensuring chip packaging quality.

[0003] Currently, the industry mainly uses X-ray imaging technology for non-destructive testing of BGA solder balls. However, bubbles in X-ray images often appear as tiny, blurry, and low-contrast targets, and are frequently affected by multiple interferences such as internal metal particles in the solder ball, lead wire obstruction, and image speckle noise. This makes manual visual inspection inefficient and inconsistent, and difficult to meet the speed and accuracy requirements of modern production lines.

[0004] In recent years, deep learning-based target detection methods have demonstrated significant advantages in industrial visual inspection. In particular, single-stage detection algorithms, represented by the YOLO series, have been widely used in defect detection due to their end-to-end structure, high inference speed, and good real-time performance. However, existing YOLO algorithms and their feature pyramid networks (such as FPN, PAN, and BiFPN) still have significant shortcomings when dealing with small targets, targets with blurred edges, and complex background interference: First, information attenuation easily occurs when deep semantic features are transmitted to shallow layers, while shallow detail features are easily masked by deep features during fusion, resulting in insufficient representation of small target features; Second, multi-scale feature fusion often relies on simple channel concatenation or weighted superposition, failing to achieve balanced and adaptive fusion of cross-layer features; Third, traditional cross-union loss functions based on rectangular boxes (such as GIoU and CIoU) are prone to gradient instability and inaccurate localization when dealing with small targets that are approximately circular or have blurred boundaries, affecting model convergence and detection accuracy.

[0005] Therefore, for detection tasks such as BGA solder ball bubbles, which are characterized by small size, blurred edges, and complex backgrounds, there is an urgent need for a detection method that can significantly improve feature fusion efficiency, enhance the semantic expression of small targets, and optimize the localization loss function while maintaining real-time detection, so as to achieve high-precision and robust automated bubble defect detection. Summary of the Invention

[0006] To address this, embodiments of the present invention provide an automatic detection method and system for solder ball bubble defects in ball grid array packaged chips, which solves the problems in the prior art caused by insufficient feature fusion, semantic information attenuation, inaccurate positioning, and poor robustness of the detection model due to the small size of the bubble target, blurred edges, and complex background.

[0007] To address the aforementioned technical problems, embodiments of the present invention provide an automatic detection method for solder ball bubble defects in ball grid array packaged chips, the method comprising:

[0008] Acquire X-ray images of the ball grid array packaged chip to be inspected;

[0009] The X-ray image is input into a trained bubble defect detection model; wherein the bubble defect detection model adopts an improved YOLOv8 network architecture, comprising:

[0010] The backbone network is used to extract multi-level features from the input image and output multiple initial feature maps at different scales.

[0011] A neck network, connected after the backbone network, is used for cross-layer fusion and enhancement of the multiple initial feature maps of different scales; the neck network includes at least:

[0012] The global feature fusion module is used to receive and fuse the initial feature maps of all scales output by the backbone network, and generate a fused feature map with global semantic information through feature alignment and redistribution operations.

[0013] The local semantic enhancement module receives the initial feature map at a partial scale from the backbone network output and generates a feature map that combines detailed information and enhanced semantics through semantically guided cross-layer fusion and local detail enhancement operations.

[0014] The feature injection module is used to adaptively weight and fuse the feature map output by the global feature fusion module and the feature map output by the local semantic enhancement module.

[0015] The head network, connected after the neck network, is used to perform target classification and location regression based on the fused features output by the neck network, and output the category and bounding box information of the defect.

[0016] Based on the output of the bubble defect detection model, a detection result containing information on the number and location of bubble defects is obtained.

[0017] Preferably, the global feature fusion module includes a FAR module and an IFM module;

[0018] The FAR module is used to achieve scale alignment and channel reallocation of multi-scale features;

[0019] The IFM module consists of a convolutional layer with a 1×1 kernel, a structure reparameterization unit, and another convolutional layer with a 1×1 kernel connected sequentially. It is used for feature channel adjustment and multi-scale local feature extraction. The working process of the IFM module includes: adjusting the number of channels of the fused features output by the FAR module, extracting multi-scale local features and enhancing feature diversity through the structure reparameterization unit, and then adjusting the number of channels to the target number through the convolutional layer before outputting the global features.

[0020] Preferably, the FAR module consists of an average pooling operation and a Shuffle unit. The Shuffle unit includes a channel splitting operation and an interleaving stitching operation. The processing includes:

[0021] Multiple feature maps adjusted to the same scale through average pooling are divided into fixed-length sub-blocks along the channel dimension. Then, according to a preset staggered fusion strategy, sub-blocks are sequentially selected from feature maps of different scales and spliced ​​along the channel dimension to generate a global fused feature map.

[0022] Preferably, the preset interleaving fusion strategy is a 15-step loop strategy. ; in the During the step fusion, according to the formula Select source feature map Sub-block extraction and splicing are performed, among which Indicates the first The scale index selected for the step. This is the modulo operator.

[0023] Preferably, the local semantic enhancement module includes a Conv convolutional layer, a semantically guided fusion module, a max pooling layer, an average pooling layer, and a Sigmoid activation function; the semantically guided fusion module is configured to: receive two feature maps to be fused, upsample the deep feature map, perform average pooling and max pooling in parallel, process them separately through convolutional layers and activation functions, add them together, and generate channel weights through the Sigmoid function; multiply the channel weights with the shallow feature map, and then add them together to output the semantically enhanced feature map.

[0024] Preferably, the processing procedure of the feature injection module includes:

[0025] The output feature map of the global feature fusion module is divided into two parts. The first part is processed by a 1×1 convolutional layer, an h_sigmoid activation function, and bilinear interpolation. The second part is processed by a 1×1 convolutional layer and bilinear interpolation.

[0026] The output features of the local feature fusion module are processed by a convolutional layer with a kernel of 1×1, multiplied by the first weight map, added to the second output feature map, and finally output by the RepVGGBlock module.

[0027] Preferably, before inputting the X-ray image into the trained bubble defect detection model, a model training step is also included:

[0028] A dataset of images of solder ball bubble traps in ball grid array packaged chips was constructed and divided into training set, validation set and test set according to a preset ratio;

[0029] Initialize the parameters of the bubble defect detection model;

[0030] The model is iteratively trained using the training set, where the loss function used is the total loss function, the expression of which is:

[0031] ;

[0032] Where L represents the total loss function, Represents classification loss. This represents the weight coefficients corresponding to the classification loss. This represents the smoothed geometric positioning loss. This represents the weighting coefficients corresponding to the smoothed geometric localization loss. Indicates the distribution focus loss. This represents the weight coefficient corresponding to the focal point of the distribution;

[0033] When the model's performance on the validation set meets the preset conditions, training is stopped, and the trained bubble defect detection model is obtained.

[0034] Preferably, the expression for the classification loss is:

[0035] ;

[0036] in, Indicates the first The true label of each sample The model predicts the first... The probability that a sample has a bubble defect. This represents the number of positive samples within the batch.

[0037] The expression for the smooth geometric localization loss is:

[0038] ;

[0039] in, Indicates the intersection-union ratio of circles. Represents the frequency modulation function. Indicates a penalty item. Represents the second-order Wasserstein distance. and Let represent the Gaussian distribution models of the ground truth bounding box and the predicted bounding box, respectively. For balance coefficient, It is a constant value;

[0040] The expression for the distribution focus loss is:

[0041] ;

[0042] in, Represents the true coordinate values ​​of the sample; and For integer bucket boundaries, That is to The integer obtained by rounding down. ,Right now The previous integer; and For label distribution values, , ; , This indicates that the bounding box coordinates predicted by the model fall within the integer bucket. and The probability of it; This represents the cross-entropy loss.

[0043] This invention also provides an automatic detection system for solder ball bubble defects in ball grid array (BGA) packaged chips. This system is used to implement the aforementioned automatic detection method for solder ball bubble defects in BGA packaged chips, specifically including:

[0044] The image acquisition module is used to acquire X-ray images of the ball grid array packaged chip to be inspected;

[0045] A defect detection module is used to input the X-ray image into a trained bubble defect detection model; wherein the bubble defect detection model adopts an improved YOLOv8 network architecture and includes, in sequence:

[0046] The backbone network is used to extract multi-level features from the input image and output multiple initial feature maps at different scales.

[0047] A neck network, connected after the backbone network, is used for cross-layer fusion and enhancement of the multiple initial feature maps of different scales; the neck network includes at least:

[0048] The global feature fusion module is used to receive and fuse the initial feature maps of all scales output by the backbone network, and generate a fused feature map with global semantic information through feature alignment and redistribution operations.

[0049] The local semantic enhancement module receives the initial feature map at a partial scale from the backbone network output and generates a feature map that combines detailed information and enhanced semantics through semantically guided cross-layer fusion and local detail enhancement operations.

[0050] The feature injection module is used to adaptively weight and fuse the feature map output by the global feature fusion module and the feature map output by the local semantic enhancement module.

[0051] The head network, connected after the neck network, is used to perform target classification and location regression based on the fused features output by the neck network, and output the category and bounding box information of the defect.

[0052] The result output module is used to obtain the detection result containing the number and location information of bubble defects based on the output of the bubble defect detection model.

[0053] This invention also provides an electronic device, which includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the above-described automatic detection method for solder ball bubble defects in ball grid array packaged chips.

[0054] As can be seen from the above technical solutions, this invention application has the following beneficial effects:

[0055] (1) This invention reconstructs the YOLOv8 neck network, designs a semantically guided cross-layer fusion (SGC) framework, and introduces a Local Semantic Enhancement (LSE) module and a Feature Alignment and Reassignment (FAR) module. The LSE module enhances the detailed features of the bubble and suppresses noise in complex backgrounds through semantically guided fusion and spatial attention mechanisms; the FAR module adopts a channel staggered splicing strategy to achieve balanced fusion of multi-scale features and avoids shallow details being masked by deep semantics. This dual-path mechanism enables the model to effectively capture weak small target signals and significantly reduce missed detections and false detections in X-ray images with small bubble size, blurred edges, lead occlusion, and speckle noise.

[0056] (2) To address the geometric characteristics of bubble-like targets, this invention proposes a Smooth Geometric Localization Loss (SGL), which integrates the Circular WIoU loss and the Normalized Wasserstein distance loss. Compared to the traditional IoU loss based on rectangular boxes, this loss better fits the approximate circular outline of the bubble and provides more accurate geometric constraints. Simultaneously, the introduced Wasserstein distance metric still provides a smooth gradient even when the predicted box slightly deviates from the ground truth box, effectively alleviating the gradient vanishing and convergence instability problems commonly encountered in training small targets, making the predicted box fit the actual bubble boundary more closely.

[0057] (3) The method of this invention is not only specifically optimized for BGA solder ball bubble detection tasks, but its core improved modules and loss function design are also universal. The SGC fusion framework and SGL loss can be transferred to other detection scenarios with small, blurred edges and near-circular characteristics, such as the identification of pores inside materials, the detection of micro-lesions in medical images, and the identification of surface defects in industrial products. While maintaining the real-time inference speed advantage of the YOLO series, the model significantly improves the detection accuracy and robustness in the above-mentioned complex scenarios, and has strong engineering implementation potential and wide industry application value. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Referring to the drawings will make the features and advantages of the present invention clearer. The drawings are illustrative and should not be construed as limiting the present invention in any way. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0059] Figure 1 This is a flowchart of an automatic detection method for solder ball bubble defects in ball grid array packaged chips provided by the present invention;

[0060] Figure 2 This is a schematic diagram of the overall structure of the bubble defect detection model in this invention;

[0061] Figure 3 This is a structural diagram of the FAR module in this invention;

[0062] Figure 4 This is a schematic diagram of the fusion of the FAR module in this invention;

[0063] Figure 5 This is a structural diagram of the local semantic enhancement module in this invention;

[0064] Figure 6 This is a structural diagram of the semantic guidance fusion module in this invention;

[0065] Figure 7 This is a schematic diagram of the detection results in this invention;

[0066] Figure 8 This is a block diagram of an automatic detection system for solder ball bubble defects in ball grid array packaged chips provided by the present invention.

[0067] Explanation of reference numerals in the accompanying drawings: 100, Image acquisition module; 200, Defect detection module; 300, Result output module. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] Example 1: To address the problems in existing technologies such as insufficient feature fusion, semantic information attenuation, inaccurate localization, and poor robustness of detection models caused by the small size of the bubble target, blurred edges, and complex backgrounds. For example... Figure 1 As shown, this invention proposes an automatic detection method for solder ball bubble defects in ball grid array packaged chips. The method includes:

[0070] S1: Acquire X-ray images of the ball grid array packaged chip to be inspected;

[0071] S2: Input the X-ray image into the trained bubble defect detection model; wherein, the bubble defect detection model adopts an improved YOLOv8 network architecture, including:

[0072] The backbone network is used to extract multi-level features from the input image and output multiple initial feature maps at different scales.

[0073] The neck network, connected after the backbone network, is used for cross-layer fusion and enhancement of initial feature maps at multiple different scales; the neck network includes at least:

[0074] The global feature fusion module is used to receive and fuse the initial feature maps of all scales output by the backbone network, and generate a fused feature map with global semantic information through feature alignment and redistribution operations.

[0075] The local semantic enhancement module receives the initial feature map at a partial scale from the backbone network output and generates a feature map that combines detailed information and enhanced semantics through semantically guided cross-layer fusion and local detail enhancement operations.

[0076] The feature injection module is used to adaptively weight and fuse the feature map output by the global feature fusion module and the feature map output by the local semantic enhancement module.

[0077] The head network, connected after the neck network, is used to classify targets and regress their locations based on the fused features output by the neck network, and outputs the category and bounding box information of defects.

[0078] S3: Based on the output of the bubble defect detection model, obtain the detection results containing information on the number and location of bubble defects.

[0079] This invention proposes an automatic detection method for solder ball bubble defects in ball grid array (BGA) packaged chips. First, an X-ray image of the chip to be inspected is acquired to provide input data for defect detection. Then, the image is input into an improved YOLOv8 detection model: the backbone network of the model performs multi-level feature extraction on the image, outputting initial feature maps at multiple scales to form a multi-level feature representation; the neck network of the model is a key innovation, its global feature fusion module aligns and redistributes features at all scales to generate feature maps rich in global semantic information, providing a stable semantic foundation for solving the detection of small targets; simultaneously, the local semantic enhancement module performs semantically guided cross-layer fusion and detail enhancement on features at some scales, generating feature maps that enhance details and semantics, specifically improving the representation ability of small, blurry bubbles; the feature injection module adaptively weights and fuses the above two types of features, achieving complementary optimization of global semantics and local details; finally, the head network performs accurate classification and position regression based on the fused features, directly outputting the number and location information of bubble defects. This method, by constructing this collaboratively enhanced detection architecture, systematically solves the problem of missed and false detections of small bubbles caused by varying scales, blurred edges, and complex backgrounds.

[0080] In step S1, the present invention first acquires X-ray images of the BGA chip to be inspected and constructs a dataset. Specifically, X-ray images of BGA chips are collected from an actual production line, and the images should contain samples with and without bubble defects. Bubble defects in each image are precisely labeled, generating corresponding label files (usually bounding box coordinates and category information), thereby constructing a complete dataset of BGA chip solder ball bubble defects. Subsequently, the dataset is divided into a training set, a validation set, and a test set according to a preset ratio (e.g., 8:1:1) for subsequent model training, optimization, and performance evaluation.

[0081] In step S2, a bubble defect detection model based on the improved YOLOv8 is constructed and trained. The core of this invention lies in reconstructing the neck portion of the YOLOv8 network, introducing a semantically guided cross-layer (SGC) framework, and designing several key modules. The improved model can be called SGC-YOLO, and its overall structure diagram is shown below. Figure 2 .

[0082] The bubble defect detection model constructed in this invention includes, in sequence:

[0083] Backbone: Based on the YOLOv8 backbone structure, it contains multiple Conv modules, C2f modules and SPPF modules, which are used to perform multi-level feature extraction on the input X-ray image and output multiple initial feature maps at different scales (e.g., 4 scales, denoted as F2, F4, F6 and F9).

[0084] The neck network, connected after the backbone network, is the main improvement of this invention. It receives multi-scale feature maps output from the backbone network and performs deep fusion and enhancement. The neck network mainly includes: a global feature fusion module, a local semantic enhancement module, a feature injection module, and a C2f module.

[0085] Head Network: Connected to the neck network, it uses a single-scale detection head to perform target classification and bounding box regression based on the fused features output by the neck network, and finally outputs the category confidence and location information of the bubble defect.

[0086] The implementation of the key modules of the neck network is described in detail below with reference to the accompanying drawings.

[0087] (1) Global Feature Fusion Module

[0088] This module aims to fuse features from all scales of the backbone network to obtain global semantic information. It includes a Feature Alignment and Redistribution (FAR) module and an IFM module.

[0089] FAR module: its structure is shown below. Figure 3 See the fusion diagram. Figure 4 .

[0090] 1. Scale alignment: The four feature maps (F2, F4, F6, F9) output by the backbone network are adjusted to the same spatial scale (usually adjusted to the smallest scale) through average pooling.

[0091] 2. Channel segmentation and staggered stitching (Shuffle unit): This involves aligning each scaled feature map... Divide into fixed lengths along the channel dimension. The sub-block is represented as:

[0092] ;

[0093] in, For feature map Total number of channels This indicates channel splicing.

[0094] 3. Preset fusion strategy: Follows a 15-step cyclical fusion strategy. Perform the splicing. Total splicing steps The sum of the number of all feature map sub-blocks: .

[0095] 4. In the During the step fusion, according to the formula Select the source feature map index, and then extract the sub-block corresponding to the current index from that feature map. Perform concatenation and update the sub-block indexes of the source feature map. .

[0096] 5. Finally, the global fused feature map is obtained. This process achieves balanced and interleaved fusion of multi-scale features across the channel dimension, preventing shallow detail features from being obscured by deep semantic features.

[0097] The IFM module consists of a 1×1 convolutional layer, a structure reparameterization unit (RepVGGBlock module), and another 1×1 convolutional layer connected sequentially. It receives the output from the FAR module, first adjusts the number of channels using a 1×1 convolution, then uses RepVGGBlock to extract richer multi-scale local features and enhance feature diversity, and finally adjusts it to the target number of channels using another 1×1 convolution, outputting the final global feature map. .

[0098] (2) Local Semantic Enhancement (LSE) module

[0099] This module focuses on detail enhancement and semantically guided fusion of some shallow and intermediate features (such as the first three scales) from the backbone network output. It includes Conv convolutional layers, a semantically guided fusion module, max pooling layers, average pooling layers, and a sigmoid activation function. Its structure is shown in [link to module]. Figure 5 .

[0100] 1. Channel alignment: Adjust multiple input feature maps to the same number of channels using Conv convolutional layers.

[0101] 2. Semantic-Guided Fusion (SGF): The channel-aligned feature maps are input pairwise into the SGF module for progressive fusion. The structure of the SGF module is shown below. Figure 6 Its workflow is as follows:

[0102] For deep feature maps Perform bilinear upsampling to match the shallow feature map. Consistent spatial scale.

[0103] Average pooling (Avgpool) and max pooling (Maxpool) operations are performed in parallel on the upsampled deep feature maps.

[0104] The two pooled features are processed by a set of convolutional layers and ReLU activation functions, respectively.

[0105] The two processed features are added together and then activated by the Sigmoid function to generate a channel attention weight map.

[0106] The weight map is multiplied by the shallow feature map to guide the shallow features with semantic information, and then added back to the original shallow feature map to output a semantically enhanced feature map. .

[0107] 3. Spatial Attention Enhancement: The semantically guided feature map output by the SGF module is subjected to global average pooling (Avgpool) and global max pooling (Maxpool) to obtain two spatial feature descriptors. These two descriptors are then summed and passed through a convolutional layer and a sigmoid activation function to generate a spatial attention weight map. Finally, this weight map is multiplied by the semantically guided feature map to further enhance key regions and suppress background noise, outputting a locally enhanced feature map. .

[0108] (3) Feature Injection Module

[0109] This module is responsible for adaptively fusing global and local features.

[0110] 1. The feature map output by the global feature fusion module The passageway is divided into two parts.

[0111] 2. First part: A weight map is generated by sequentially passing the data through a 1×1 convolutional layer, an h_sigmoid activation function, and a bilinear interpolation upsampling operation. .

[0112] 3. Second part: The feature map is generated by sequentially passing it through a 1×1 convolutional layer and bilinear interpolation upsampling operation. .

[0113] 4. The feature map output by the local semantic enhancement module After adjustment by a 1×1 convolutional layer, and with the weight map Multiplying them yields weighted local features.

[0114] 5. Combine weighted local features with Add them together to achieve feature injection.

[0115] 6. Finally, the summed features are further integrated and enhanced using a RepVGGBlock module to output the final fused feature map of the neck network. .

[0116] 7. The features are then refined through a C2f module before being fed into the head network.

[0117] Furthermore, the SGC-YOLO model described above is trained using the constructed training set.

[0118] Optimizer: The Adam backpropagation algorithm is used to iteratively optimize the model parameters.

[0119] Loss function: A combined loss function is used until the total loss converges. Total loss function It consists of three parts:

[0120] ;

[0121] Where L represents the total loss function, Represents classification loss. This represents the weight coefficients corresponding to the classification loss. This represents the smoothed geometric positioning loss. This represents the weighting coefficients corresponding to the smoothed geometric localization loss. Indicates the distribution focus loss. This represents the weight coefficient corresponding to the focal point of the distribution.

[0122] Furthermore, classification loss The binary cross-entropy loss is used, and the formula is as follows:

[0123] ;

[0124] in, Indicates the first One-hot label (0 or 1) for each sample. Indicates the first The probability of each sample being predicted as a bubble defect. This represents the number of positive samples within the batch.

[0125] Smooth Geometric Loss: This invention proposes a Smooth Geometric Loss (SGL), which is a weighted sum of the Circular WIoU loss and the normalized Wasserstein distance loss, specifically optimized for small, near-circular targets such as bubbles.

[0126] ;

[0127] In the formula, ,in Indicates the intersection-union ratio of circles. This represents the frequency modulation function, used for stable training; This indicates a penalty. ,in Represents the second-order Wasserstein distance. and Let represent the Gaussian distribution models of the ground truth bounding box and the predicted bounding box, respectively. For balance coefficient, It is a constant value (e.g., 12.8).

[0128] Distribution focus loss : Used to optimize the regression of bounding box coordinates, making their distribution more focused near the true values:

[0129] ;

[0130] in, Represents the true coordinate values ​​of the sample; and For integer bucket boundaries, That is to The integer obtained by rounding down. ,Right now The previous integer; and For label distribution values, , ; , This indicates that the bounding box coordinates predicted by the model fall within the integer bucket. and The probability of it; This represents the cross-entropy loss.

[0131] In step S3, the trained model is used to detect bubble defects. Specifically, the X-ray image of the BGA chip to be detected is input into the trained SGC-YOLO model. After feature extraction by the backbone network, multi-scale fusion and enhancement by the neck network, and prediction by the head network, the model directly outputs the bounding box locations (center point coordinates, width and height) and corresponding confidence scores of all suspected bubble defects in the image. Based on a preset confidence threshold and / or non-maximum suppression (NMS) operation, the final bubble defect detection results are selected, including their number and specific location in the image. An example of the detection results is shown below. Figure 7 As shown.

[0132] Example 2: This invention provides an automatic detection system for solder ball bubble defects in ball grid array packaged chips, such as... Figure 8 As shown, this system is used to implement the automatic detection method for solder ball bubble defects in ball grid array packaged chips according to Embodiment 1 above. This system can be integrated into an industrial computer or server, and specifically includes:

[0133] Image acquisition module 100 is used to acquire X-ray images of the ball grid array packaged chip to be inspected;

[0134] The defect detection module 200 is used to input X-ray images into a trained bubble defect detection model; wherein, the bubble defect detection model adopts an improved YOLOv8 network architecture, and includes, in sequence:

[0135] The backbone network is used to extract multi-level features from the input image and output multiple initial feature maps at different scales.

[0136] The neck network, connected after the backbone network, is used for cross-layer fusion and enhancement of initial feature maps at multiple different scales; the neck network includes at least:

[0137] The global feature fusion module is used to receive and fuse the initial feature maps of all scales output by the backbone network, and generate a fused feature map with global semantic information through feature alignment and redistribution operations.

[0138] The local semantic enhancement module receives the initial feature map at a partial scale from the backbone network output and generates a feature map that combines detailed information and enhanced semantics through semantically guided cross-layer fusion and local detail enhancement operations.

[0139] The feature injection module is used to adaptively weight and fuse the feature map output by the global feature fusion module and the feature map output by the local semantic enhancement module.

[0140] The head network, connected after the neck network, is used to classify targets and regress their locations based on the fused features output by the neck network, and outputs the category and bounding box information of defects.

[0141] The result output module 300 is used to obtain the detection result containing the number and location information of bubble defects based on the output of the bubble defect detection model.

[0142] Furthermore, the image acquisition module 100 is connected to an X-ray imaging device and is used to acquire or receive X-ray images of the BGA chip to be inspected.

[0143] Furthermore, the defect detection module 200 is a core processing unit, embedded with an SGC-YOLO model trained according to the method described in Embodiment 1. This module receives images from the image acquisition module 100 and automatically completes the entire process of feature extraction, fusion, classification, and localization.

[0144] Furthermore, the result output module 300 receives the output from the defect detection module 200, processes the detection results (e.g., filtering low-confidence boxes, applying NMS), generates a structured detection report, including the number of defects, location coordinates, confidence level, etc., and can overlay it onto the original image through a graphical interface (e.g., ...). Figure 7 (or transmit to the production line control system).

[0145] Furthermore, the system of the present invention may also include a model update and management module: responsible for loading the model, managing the version, and performing incremental training or retraining on the model based on newly labeled data.

[0146] This embodiment provides an automatic detection system for solder ball bubble defects in ball grid array (BGA) packaged chips, used to implement the aforementioned automatic detection method for solder ball bubble defects in BGA packaged chips. Therefore, the specific implementation of the automatic detection system for solder ball bubble defects in BGA packaged chips can be found in the previous embodiment section of the automatic detection method for solder ball bubble defects in BGA packaged chips. For example, the image acquisition module 100, the defect detection module 200, and the result output module 300 are respectively used to implement steps S1, S2, and S3 in the above-mentioned automatic detection method for solder ball bubble defects in BGA packaged chips. Therefore, the specific implementation can be referred to the description of the corresponding embodiments. To avoid redundancy, it will not be repeated here.

[0147] Example 3: This embodiment of the invention provides an electronic device, which includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory stores instructions, and the processor executes the instructions stored in the memory to implement the above-described automatic detection method for solder ball bubble defects in ball grid array packaged chips. This electronic device can be an industrial computer, a server, an embedded device, or a cloud computing node.

[0148] As can be seen from the above specific implementation methods, the present invention reconstructs the neck network of YOLOv8, introduces the SGC framework and modules such as LSE and FAR, and combines it with a specially designed SGL loss function to form a complete BGA chip bubble defect detection solution.

[0149] 1. More efficient feature fusion: The FAR module balances multi-scale features through an interleaved channel splicing strategy to prevent the loss of shallow details; the LSE module enhances the feature representation of small targets through semantic guidance and spatial attention.

[0150] 2. More accurate positioning: The SGL loss function combines Circular WIoU and Wasserstein distance to better fit the geometry of the bubble, alleviating the gradient instability problem in small target positioning.

[0151] 3. Strong anti-interference ability: The global and local dual-path fusion mechanism enables the model to maintain high detection robustness even under complex backgrounds, noise and lead wire occlusions commonly found in X-ray images.

[0152] 4. High practicality: This method and system can be directly integrated into existing production lines to achieve online, automatic, and rapid detection of bubbles in BGA solder balls, significantly improving detection efficiency and consistency.

[0153] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0156] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. An automatic detection method for solder ball bubble defects in ball grid array packaged chips, characterized in that, include: Acquire X-ray images of the ball grid array packaged chip to be inspected; The X-ray image is input into a trained bubble defect detection model; wherein the bubble defect detection model adopts an improved YOLOv8 network architecture, comprising: The backbone network is used to extract multi-level features from the input image and output multiple initial feature maps at different scales. A neck network, connected after the backbone network, is used for cross-layer fusion and enhancement of the multiple initial feature maps of different scales; the neck network includes at least: The global feature fusion module is used to receive and fuse the initial feature maps of all scales output by the backbone network, and generate a fused feature map with global semantic information through feature alignment and redistribution operations. The local semantic enhancement module receives the initial feature map at a partial scale from the backbone network output and generates a feature map that combines detailed information and enhanced semantics through semantically guided cross-layer fusion and local detail enhancement operations. The feature injection module is used to adaptively weight and fuse the feature map output by the global feature fusion module and the feature map output by the local semantic enhancement module. The head network, connected after the neck network, is used to perform target classification and location regression based on the fused features output by the neck network, and output the category and bounding box information of the defect. Based on the output of the bubble defect detection model, a detection result containing information on the number and location of bubble defects is obtained.

2. The automatic detection method for solder ball bubble defects in ball grid array packaged chips according to claim 1, characterized in that, The global feature fusion module includes a FAR module and an IFM module; The FAR module is used to achieve scale alignment and channel reallocation of multi-scale features; The IFM module consists of a convolutional layer with a 1×1 kernel, a structure reparameterization unit, and another convolutional layer with a 1×1 kernel connected sequentially. It is used for feature channel adjustment and multi-scale local feature extraction. The working process of the IFM module includes: adjusting the number of channels of the fused features output by the FAR module, extracting multi-scale local features and enhancing feature diversity through the structure reparameterization unit, and then adjusting the number of channels to the target number through the convolutional layer before outputting the global features.

3. The automatic detection method for solder ball bubble defects in ball grid array packaged chips according to claim 2, characterized in that, The FAR module consists of an average pooling operation and a Shuffle unit. The Shuffle unit includes channel splitting and interleaving operations, and the processing includes: Multiple feature maps adjusted to the same scale through average pooling are divided into fixed-length sub-blocks along the channel dimension. Then, according to a preset staggered fusion strategy, sub-blocks are sequentially selected from feature maps of different scales and spliced ​​along the channel dimension to generate a global fused feature map.

4. The automatic detection method for solder ball bubble defects in ball grid array packaged chips according to claim 3, characterized in that, The preset interleaving and fusion strategy is a 15-step loop strategy. ; in the During the step fusion, according to the formula Select source feature map Sub-block extraction and splicing are performed, among which Indicates the first The scale index selected for the step. This is the modulo operator.

5. The automatic detection method for solder ball bubble defects in ball grid array packaged chips according to claim 1, characterized in that, The local semantic enhancement module includes a Conv convolutional layer, a semantic-guided fusion module, a max-pooling layer, an average-pooling layer, and a Sigmoid activation function. The semantic-guided fusion module is configured to: receive two feature maps to be fused; upsample the deep feature map; perform average-pooling and max-pooling in parallel; process the deep feature map with convolutional layers and activation functions respectively; add the two maps together and generate channel weights using the Sigmoid function; multiply the channel weights by the shallow feature map; add the channel weights by the shallow feature map; and output the semantically enhanced feature map.

6. The automatic detection method for solder ball bubble defects in ball grid array packaged chips according to claim 1, characterized in that, The processing steps of the feature injection module include: The output feature map of the global feature fusion module is divided into two parts. The first part is processed by a 1×1 convolutional layer, an h_sigmoid activation function, and bilinear interpolation. The second part is processed by a 1×1 convolutional layer and bilinear interpolation. The output features of the local feature fusion module are processed by a convolutional layer with a kernel of 1×1, multiplied by the first weight map, added to the second output feature map, and finally output by the RepVGGBlock module.

7. The automatic detection method for solder ball bubble defects in ball grid array packaged chips according to claim 1, characterized in that, Before inputting the X-ray image into the trained bubble defect detection model, a model training step is also included: A dataset of images of solder ball bubble traps in ball grid array packaged chips was constructed and divided into training set, validation set and test set according to a preset ratio; Initialize the parameters of the bubble defect detection model; The model is iteratively trained using the training set, where the loss function used is the total loss function, the expression of which is: ； Where L represents the total loss function, Represents classification loss. This represents the weight coefficients corresponding to the classification loss. This represents the smoothed geometric positioning loss. This represents the weighting coefficients corresponding to the smoothed geometric localization loss. Indicates the distribution focus loss. This represents the weight coefficient corresponding to the focal point of the distribution; When the model's performance on the validation set meets the preset conditions, training is stopped, and the trained bubble defect detection model is obtained.

8. The automatic detection method for solder ball bubble defects in ball grid array packaged chips according to claim 7, characterized in that, The expression for the classification loss is: ； in, Indicates the first The true label of each sample The model predicts the first... The probability that a sample has a bubble defect. This represents the number of positive samples within the batch. The expression for the smooth geometric localization loss is: ； in, Indicates the intersection-union ratio of circles. Represents the frequency modulation function. Indicates a penalty item. Represents the second-order Wasserstein distance. and Let represent the Gaussian distribution models of the ground truth bounding box and the predicted bounding box, respectively. For balance coefficient, It is a constant value; The expression for the distribution focus loss is: ； in, Represents the true coordinate values ​​of the sample; and For integer bucket boundaries, That is to The integer obtained by rounding down. ,Right now The previous integer; and For label distribution values, , ; , This indicates that the bounding box coordinates predicted by the model fall within the integer bucket. and The probability of it; This represents the cross-entropy loss.

9. An automatic detection system for solder ball bubble defects in ball grid array packaged chips, characterized in that, The system is used to implement the automatic detection method for solder ball bubble defects in ball grid array packaged chips according to any one of claims 1 to 8, specifically including: The image acquisition module is used to acquire X-ray images of the ball grid array packaged chip to be inspected; A defect detection module is used to input the X-ray image into a trained bubble defect detection model; wherein the bubble defect detection model adopts an improved YOLOv8 network architecture and includes, in sequence: The backbone network is used to extract multi-level features from the input image and output multiple initial feature maps at different scales. A neck network, connected after the backbone network, is used for cross-layer fusion and enhancement of the multiple initial feature maps of different scales; the neck network includes at least: The global feature fusion module is used to receive and fuse the initial feature maps of all scales output by the backbone network, and generate a fused feature map with global semantic information through feature alignment and redistribution operations. The local semantic enhancement module receives the initial feature map at a partial scale from the backbone network output and generates a feature map that combines detailed information and enhanced semantics through semantically guided cross-layer fusion and local detail enhancement operations. The feature injection module is used to adaptively weight and fuse the feature map output by the global feature fusion module and the feature map output by the local semantic enhancement module. The head network, connected after the neck network, is used to perform target classification and location regression based on the fused features output by the neck network, and output the category and bounding box information of the defect. The result output module is used to obtain the detection result containing the number and location information of bubble defects based on the output of the bubble defect detection model.

10. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the automatic detection method for solder ball bubble defects of ball grid array packaged chips according to any one of claims 1 to 8.

Images