Method, apparatus, and electronic device for optical inspection of ball grid array component offset defects based on artificial intelligence

Abstract

An optical detection method, apparatus, and electronic device for offset defects in ball grid array (BGA) components based on artificial intelligence are disclosed. The method involves acquiring optical images using an automated optical inspection device, extracting global position features and edge pixel-level offset features from the optical images through global and local branches of a multi-scale feature fusion network, weighted fusion of the two features using an attention mechanism to generate fused features, and performing regression processing on these fused features to output the offset. Finally, the offset is compared with a preset threshold to determine whether the BGA component under test in the optical image is defective. This method aims to improve the accuracy and robustness of BGA offset detection.

Claims

1. An optical detection method for offset defects in ball grating array elements based on artificial intelligence, executed by a processor, characterized in that it includes: An optical image is acquired from an automated optical inspection device. This optical image includes a Ball Grid Array (BGA) element under test and its peripheral features on a circuit board. The optical image is processed by a multi-scale feature fusion network, which includes a global branch and a local branch. The global branch extracts a global position feature from the overall position of the BGA element under test in a global view of the optical image. The local branch extracts an edge pixel-level offset feature from a region of interest (ROI) in the optical image corresponding to the peripheral features. The global branch is used for coarse localization to quickly filter out suspected offset BGA elements under test, while the local branch is used for fine detection to perform pixel-level offset edge analysis. The global position feature and the edge pixel-level offset feature are weighted and fused using an attention mechanism through a feature fusion module of the multi-scale feature fusion network to generate a fused feature. The fused feature is fed into multiple fully connected layers of the multi-scale feature fusion network to perform regression processing to output an offset; and the offset is compared with a preset threshold. When the offset is greater than the preset threshold, the BGA component under test is determined to be defective.

2. The optical detection method for offset defects of ball grating array elements based on artificial intelligence as claimed in claim 1, characterized in that the offset includes an X-direction offset value, a Y-direction offset value, and an angle, the preset threshold includes an X-direction threshold, a Y-direction threshold, and an angle threshold, and the offset is determined to be greater than the preset threshold when one of the following comparison conditions is met: the X-direction offset value is greater than the X-direction threshold; the Y-direction offset value is greater than the Y-direction threshold; and the angle is greater than the angle threshold.

3. The optical detection method for offset defects of ball grating array elements based on artificial intelligence as claimed in claim 1, characterized in that the multi-scale feature fusion network is pre-generated through a training process, the training process comprising: dynamically increasing the weight of an offset sample using a dynamic sample weighting strategy (DSW) according to a sample class frequency, and combining a loss function to suppress the gradient influence of a normal sample, so as to balance a class bias between the normal sample and the offset sample, wherein the loss function comprises Focal Loss.

4. The optical detection method for offset defects of ball grid array elements based on artificial intelligence as claimed in claim 3, characterized in that the training process further includes performing a data augmentation process on the normal sample before using the dynamic sample weighting strategy. The data augmentation process uses an affine transformation to generate multiple virtual offset samples, which are used to simulate misalignment including X / Y offset and angular rotation.

5. The optical detection method for offset defects of ball grid array elements based on artificial intelligence as claimed in claim 1, characterized in that the peripheral features include at least one of silkscreen lines, positioning marks, and circuit patterns.

6. The optical detection method for offset defects of ball grid array elements based on artificial intelligence as claimed in claim 1, characterized in that the global branch system extracts the global position features using a convolutional neural network (ResNet-18) with a network depth of 18 layers.

7. The optical detection method for offset defects of ball grid array elements based on artificial intelligence as claimed in claim 1, characterized in that the local branch system uses a high-precision convolutional neural network to detect the pixel-level offset of the encapsulation edge, so as to extract the pixel-level offset features of the edge.

8. The optical detection method for offset defects of ball grating array elements based on artificial intelligence as claimed in claim 1, characterized in that the attention mechanism performs a weighted fusion system that applies an N% weight to the global position feature and an (100-N)% weight to the edge pixel-level offset feature, wherein, N is a positive integer.

9. An optical detection device for offset defects in ball grating array elements based on artificial intelligence, characterized in that it comprises: An image acquisition module is used to acquire an optical image from an automated optical inspection device. The optical image includes a ball grid array (BGA) element under test and its peripheral features on a circuit board. A feature extraction module, coupled to the image acquisition module, is used to extract a global position feature and an edge pixel-level offset feature from the optical image through a global branch and a local branch of a multi-scale feature fusion network. The global branch extracts the global position feature based on the overall position of the BGA element under test from a global view of the optical image. The local branch extracts the edge pixel-level offset feature from a region of interest (ROI) in the optical image corresponding to the peripheral features. The global branch is used for coarse localization to quickly filter out suspected offset BGA elements under test, and the local branch is used for fine detection to perform pixel-level offset edge analysis. A feature fusion module, coupled to the feature extraction module, is used to perform an attention mechanism... The system performs a weighted fusion of the global location feature and the edge pixel-level offset feature to generate a fused feature; an offset calculation module, coupled to the feature fusion module, feeds the fused feature into multiple fully connected layers of the multi-scale feature fusion network to perform regression processing and output an offset; and a determination module, coupled to the offset calculation module, compares the offset with a preset threshold, and determines the BGA component under test as defective when the offset is greater than the preset threshold.

10. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that, when the processor executes the computer program, it implements the optical detection method for offset defects of ball grid array elements based on artificial intelligence as described in claim 1.

Images