Electronic component X-ray detection method based on convolutional neural network
By adopting a multi-scale object detection model based on convolutional neural network in the X-ray detection of electronic components, combined with the coordinate attention mechanism and feature pyramid network, the problem of inefficiency of traditional X-ray inspection methods is solved, and efficient and accurate detection of internal structure defects of electronic components is achieved.
Patent Information
- Application Number
- CN202510090044.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-21
- Publication Date
- 2025-05-16
AI Technical Summary
Traditional X-ray inspection methods are inefficient in detecting internal structural defects of electronic components and are easily subjectively affected by the detection environment and personnel, resulting in low accuracy and efficiency.
A multi-scale object detection model based on convolutional neural network is adopted, combining the coordinate attention mechanism and feature pyramid network, and overfitting is avoided through feature batch elimination modules, and the loss function is improved to balance the loss value of positive and negative samples.
It significantly improves the feature extraction capability of X-ray imaging of electronic components, improves detection efficiency and accuracy, and ensures the controllability of component quality.
Smart Images
Figure CN120013897A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of electronic component testing, and in particular to an electronic component X-ray detection method based on a convolutional neural network and a destructive physical analysis X-ray inspection technology for electronic components. Background Art
[0002] Electronic components are the basic units of various electronic systems, and the reliability of electronic components is the basis for ensuring the reliability of electronic systems. In the field of electronic component manufacturing, quality monitoring of electronic components is an indispensable process. Under the increasing demand for component quality in electronic systems, destructive physical analysis (DPA) technology is an important method to ensure the quality of electronic components. Destructive physical analysis is used to monitor the production process during the production and processing of electronic components, especially the analysis and monitoring of key process quality. It plays an irreplaceable role in improving the reliability level of components that cannot be replaced by other tests and inspection methods. Among them, X-ray inspection is used in DPA to detect defects in the sealed tube shell, especially defects in the sealing process from the tube cover to the tube shell and internal defects, such as excess matter, incorrect inner lead connection, voids in the chip bonding material or in the glass when glass sealing is used. X-rays are required to check whether there are movable excess matter and structural defects inside the device under test before unsealing, to confirm whether particles are introduced or unnecessary structural damage is caused during the unsealing or cutting of the device under test, and to determine the cavity volume, cavity height, unsealing and section positioning. Traditional X-ray inspection methods are inefficient and easily affected by the test environment and the subjective influence of the testers, resulting in low accuracy and efficiency. Therefore, the artificial intelligence-assisted method using convolutional neural networks can effectively detect the internal structural defects of electronic components. By processing the obtained X-ray images, the deep learning model can quickly and accurately detect product defects, thereby greatly improving the inspection efficiency and further ensuring the controllable quality of components. Summary of the invention
[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an electronic component X-ray detection method based on a convolutional neural network. The method takes into account the difficulty of manually judging the pass degree of the X-ray inspection test of electronic components in the destructive physical analysis (DPA) test, adopts artificial intelligence means, and proposes a multi-scale target detection model based on a convolutional neural network. The model adopts a coordinate attention mechanism and a feature pyramid network to improve the scale range of the model's feature extraction of electronic component X-ray imaging, and uses a feature batch elimination module to avoid overfitting. Finally, the loss function is improved, and the balance of the loss values of positive and negative samples in the training process is ensured by adjusting the hyperparameters, which provides algorithm support for the intelligent identification of the destructive physical analysis (DPA) test of electronic components.
[0004] The present invention solves the technical problem by the following technical solutions:
[0005] An X-ray detection method for electronic components based on convolutional neural network, the detection method comprises the following steps:
[0006] Step 1), preprocessing the X-ray image of the electronic component to obtain a clear detection image, converting the detection image into a vector format for input into the loss function training network model;
[0007] Step 2), construct an improved multi-scale target detection network model based on coordinate attention and design a loss function to train the network model;
[0008] 2.1. Construct an improved multi-scale target detection network model based on coordinate attention;
[0009] 2.2. Improve the loss function classification weight and train the model to obtain an improved loss function training network model;
[0010] Step 3) Test the performance of the improved loss function training network model and save the optimal network model parameters.
[0011] Moreover, the specific process of step 1) preprocessing the X-ray image of the electronic component to obtain a clear detection image is as follows:
[0012] Collect X-ray images of electronic components to obtain detection images;
[0013] Mark the categories and locations of three types of defects in the inspection images in the data set: damaged deformation, bond wire breakage, and the presence of excess objects;
[0014] Perform data expansion on the marked detection images;
[0015] The expanded detection image data is denoised and clarified to enhance the contrast of the detection image and highlight the details. Finally, it is converted into a vector format of RGB3 channels for input into the loss function training network model.
[0016] Moreover, the specific process of constructing the improved coordinate attention-based multi-scale target detection network model in step 2.1 is:
[0017] A. Using Faster-RCNN as the basic network, multiple cascaded RCNN structures are introduced to continuously optimize the target recognition results;
[0018] B. Resnet101 residual structure is used as the feature extraction network to fuse the defect feature information extracted by convolution kernels of different sizes to achieve multi-scale feature extraction, which is more suitable for extracting subtle defect features of different shapes in the inspection images of components, such as round bubbles, long strips of redundant objects, etc., thereby improving the recognition effect of the network structure model on the target;
[0019] C. Integrate the coordinate attention mechanism into the Resnet101 feature extraction network to extract features of both position and spatial dimensions at the same time, generate attention factors for each semantic group, implement adaptive calibration of feature maps, and enhance the semantic representation of the model.
[0020] Moreover, the specific process of integrating the coordinate attention mechanism into the Resnet101 feature extraction network in step C is:
[0021] The detection image is X. The CA module first generates attention feature maps with horizontal and vertical coordinate information in the global pooling layer, as shown in formulas (1) and (2);
[0022]
[0023] The generated horizontal attention feature map f h and the vertical attention feature map f v Concatenation, convolution and activation function are performed to obtain feature f, and f is split to obtain two attention feature vectors. and Then for Z h and Z v Perform transformation activation to obtain m h =σ(F h (Z h )) and m v =σ(F w (Z v )), finally, m h and m v Multiply the input feature X at the corresponding position, and finally get the output attention feature Y, as shown in formula (3)
[0024] Y=X×m h ×m v (3).
[0025] Moreover, the specific process of improving the loss function classification weight and training the model in step 2.2 to obtain the improved loss function training network model is:
[0026] The loss function is a function used to evaluate the gap between the model prediction value and the true value, which consists of the problem classification loss function and the defect location loss function, as shown in formula (4).
[0027]
[0028] Among them, L loss is the overall loss function of the model, L loc is the positioning loss function, L cls is the classification loss function;
[0029] During training, the calculation method of the loss function is improved, and the hyperparameters are set to balance the weights of positive and negative samples, as shown in formula (6).
[0030]
[0031] Where α is a hyperparameter to balance the weights of different samples.
[0032] The target positioning loss function is shown in formula (7):
[0033]
[0034] Among them, t is the coordinate value of the actual target, is the predicted coordinate value.
[0035] Moreover, the specific process of testing the improved loss function training network model performance and saving the optimal network model parameters in step 3) is:
[0036] 3.1 Set the batch size to 32, the number of epochs to 500, the learning rate to 0.0001, the weight decay to 0.00001, and the improved loss function to train the network model;
[0037] 3.2 The improved loss function is tested by calculating the performance evaluation index to verify the performance of the network model trained and save the detection ability of the optimal network model parameters. The recall rate and precision rate of IOU=0.5, the average recall rate and average precision rate of the three categories of damaged deformation, bonding wire breakage and the presence of redundant objects are selected to evaluate the model detection ability.
[0038] The beneficial effects of the present invention are:
[0039] 1. The electronic component X-ray detection method based on convolutional neural network of the present invention aims at the problems of low efficiency of manual detection and high difficulty of discrimination in X-ray inspection, and proposes a multi-scale target detection network CABFE-RCNN (coordinate attention RCNN with batch feature erasing) based on R-CNN (Region-based Convolution Neural Networks). The CABFE-RCNN model includes a CA coordinate attention feature extraction structure based on Resnet101, an FPN feature pyramid network and a BFE feature batch elimination network. The coordinate attention mechanism is used to provide richer feature information for subsequent feature fusion of the network; the BFE feature batch elimination network is used to solve the overfitting problem by randomly eliminating the image extraction feature map area; and the improved loss function is used to solve the balance of positive and negative samples in model training and improve the learning ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 It is a schematic diagram of the overall structure of CABFE-RCNN of the present invention;
[0041] Figure 2 Schematic diagram of the combined convolution structure of the present invention;
[0042] Figure 3 It is a schematic diagram of the CA module structure of the present invention;
[0043] Figure 4 It is a schematic diagram of the BFE module structure of the present invention. DETAILED DESCRIPTION
[0044] The present invention is further described in detail below through specific examples. The following examples are only illustrative and not restrictive, and the protection scope of the present invention cannot be limited thereto.
[0045] An X-ray detection method for electronic components based on convolutional neural network, the steps of the detection method are:
[0046] Step 1), electronic component image preprocessing: preprocess the X-ray image of the electronic component to obtain a clear detection image, and convert the detection image into a vector format for input into the loss function training network model;
[0047] The X-ray imaging preprocessing algorithm is the primary work in the present invention. The images selected by this method are all obtained from experiments, totaling 3000 images with a resolution of 2152×1131. The categories and positions of the three types of defects in the data set, namely, damaged deformation, bond wire fracture, and redundant objects, are marked. In order to solve the problem of unbalanced data sets, random flipping, brightness transformation, and rotation scaling methods are used for data expansion. After expansion, 4500 images of four categories, namely, damaged deformation, bond wire fracture, redundant objects, and no defects, are obtained. In order to compress the amount of calculation and increase the amount of batch processing, the image is denoised by the mean filtering method to eliminate noise and make the image clearer, and contrast-limited adaptive histogram equalization (CLAHE) is performed to enhance the image contrast and highlight the details. Finally, it is converted into a vector format of RGB3 channels and input into the network.
[0048] Step 2) Build an improved multi-scale object detection network based on coordinate attention and design a loss function to train the network:
[0049] 2.1 Constructing an improved multi-scale object detection network based on coordinate attention:
[0050] A. Using Faster-RCNN as the basic network, multiple cascaded RCNN structures are introduced to achieve continuous optimization of target recognition results. The present invention uses Faster-RCNN (region convolutional neural network, CNN) as the basic network, introduces multiple cascaded RCNN structures, and includes 3 cascaded RCNN structures in total. The IoU (intersection over union) values of each structure are 0.5, 0.6 and 0.7 respectively. The IoU values in the cascaded structure gradually increase, and the target recognition results are continuously optimized. On this basis, further optimization is carried out for the problems of poor recognition effect and difficulty in feature extraction due to the difference in defect sizes of different types of electronic components in X-ray imaging and the large difference in feature structures.
[0051] B. Resnet101 residual structure is used as the feature extraction network to fuse the defect feature information extracted by convolution kernels of different sizes to achieve multi-scale feature extraction. It is more suitable for extracting subtle defect features of different shapes in the inspection images of components, such as round bubbles, long strips of redundant objects, etc., thereby improving the recognition effect of the network structure model on the target.
[0052] The present invention adopts the Resnet101 residual structure as the feature extraction network, which includes 1 convolution block conv1 and 4 residual blocks conv2, conv3, conv4 and conv5. The convolution block and residual block in the Resnet101 residual structure are improved. The convolution block and residual block use 3×3 and 5×5 convolution kernels of different sizes to perform convolution operations, and merge the features after the operation to form a merged convolution layer structure. This structure replaces the convolution block and residual block structure of the original Resnet101 feature extraction network, such as Figure 2 As shown in the figure, the defect feature information extracted by convolution kernels of different sizes is fused to realize multi-scale feature extraction, which is more suitable for extracting subtle defect features of different shapes in component X-ray imaging, such as round bubbles, long strips of redundant objects, etc., thereby improving the recognition effect of the network structure model on the target.
[0053] C. Integrate the coordinate attention mechanism into the Resnet101 feature extraction network to extract features of both position and spatial dimensions at the same time, generate attention factors for each semantic group, implement adaptive calibration of feature maps, and enhance the semantic representation of the model.
[0054] D. Since the size of defects such as cracks, excess materials, and broken bonding wires inside electronic components is small, in the original feature extraction process, accompanied by the process of downsampling and convolution, it is easy to lose a lot of key small-size feature information, which eventually leads to the omission of small features. Therefore, in order to make the model pay more attention to key positions such as cracks, excess materials, and broken bonding wires, and thus extract key position features to obtain the region of interest, the present invention integrates the coordinate attention mechanism into the Resnet101 feature extraction network to enhance the model's ability to learn related key features. It can not only extract features of position and spatial dimensions at the same time, but also generate attention factors for each semantic group, realize adaptive calibration feature maps, and enhance the semantic representation of the model. The schematic diagram of this module is shown in Figure 3 shown.
[0055] The detection image is X, and the CA module first generates attention feature maps with horizontal and vertical coordinate information in the global pooling layer, as shown in formulas (1) and (2).
[0056]
[0057] The generated horizontal attention feature map f h and the vertical attention feature map f v Concatenation, convolution and activation function are performed to obtain feature f, and f is split to obtain two attention feature vectors. and Then for Z h and Z vPerform transformation activation to obtain m h =σ(F h (Z h )) and m v =σ(F w (Z v )), finally, m h and m v Multiply the input feature X at the corresponding position respectively, and finally obtain the output attention feature Y, as shown in formula (3).
[0058] Y=X×m h ×m v (3)
[0059] According to the above formula, the CA coordinate attention mechanism can more specifically extract various relevant semantic features in the model and improve the output accuracy of the model. At the same time, it also has the characteristics of lightweight, which can improve the model detection accuracy without increasing the computational cost too much.
[0060] In order to further reduce the impact of overfitting on the model, the present invention adds a batch feature erasing (BFE) module to the backbone network. The schematic diagram of the module is as follows: Figure 4 As shown. The BFE module generates a mask feature mask during the model training process. The operation with the corresponding ROI feature map can eliminate the randomly extracted feature map area in the same batch, and then enhance the semantic expression of the feature area that has not been eliminated, so as to improve the model's learning ability for key features. Due to the wide variety of electronic components, the complex internal structure of the cavity, and the existence of multiple possible defects, the features that the model needs to recognize are relatively complex. The addition of the BFE module prevents the model from paying too much attention to certain fixed features during training. By increasing the diversity of feature learning, the generalization ability and robustness of the model are improved.
[0061] 2.2 Improve the loss function classification weight and train the model:
[0062] For the X-ray imaging inspection task of electronic components, two parts need to be completed: problem classification and defect location. The loss function is a function used to evaluate the gap between the model prediction value and the true value. It consists of a classification loss function and a location loss function, as shown in formula (4).
[0063]
[0064] Among them, L loss is the overall loss function of the model, L loc is the positioning loss function, L cls is the classification loss function. In traditional classification model training, the cross entropy loss function is generally used, as shown in formula (5).
[0065]
[0066] Where L CE is the cross entropy loss function used in traditional classification model training. y is the true value of the sample. If there is a problem with the device, y is 1, otherwise it is 0. is the model prediction value. It can be seen that this function does not take into account the imbalance of the number of samples. For categories with a small number of samples, the network is not sufficient to learn all of its features, which increases the difficulty of classification; and the uneven distribution of some sample feature areas will further increase the difficulty of classification. To address the above problems, this paper improves the calculation method of the loss function during training and sets hyperparameters to balance the weights of positive and negative samples, as shown in formula (6).
[0067]
[0068] Where α is a hyperparameter to balance the weights of different samples.
[0069] The target positioning loss function is shown in formula (7):
[0070]
[0071] Among them, t is the coordinate value of the actual target, is the predicted coordinate value. It can be seen that during the training process of the CABFE-RCNN model, the network forward propagation can balance the positive and negative samples by setting reasonable hyperparameters. If a problematic label image is misjudged as having no problem, the loss value will increase, thereby improving the model's adaptive learning ability, and ultimately improving the model's ability to learn samples.
[0072] Step 3) Test model performance and save the best network model parameters:
[0073] 3.1 Set the batch size to 32, the number of epochs to 500, the learning rate to 0.0001, the weight decay to 0.00001, and the improved loss function mentioned above to train the model.
[0074] 3.2 The detection ability of the model in this paper is verified by calculating the performance evaluation indicators. The recall rate with IOU=0.5, the precision rate, the average recall rate of three categories (damaged deformation, broken bonding wires, and the presence of redundant objects), and the average precision rate of three categories are selected to evaluate the detection ability of the model. According to the verification results, this method has achieved more superior detection accuracy overall. The average recall rate and average precision of the three categories of damaged deformation, broken bonding wires, and the presence of redundant objects reached 94.87% and 92.09%, which are better than several other existing algorithms.
[0075] Although the embodiments and drawings of the present invention are disclosed for illustrative purposes, those skilled in the art will appreciate that various substitutions, changes and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the scope of the present invention is not limited to the contents disclosed in the embodiments and drawings.
Claims
1. An X-ray detection method for electronic components based on convolutional neural network, characterized in that: The steps of this detection method are: Step 1), electronic component image preprocessing: preprocess the X-ray image of the electronic component to obtain a clear detection image, and convert the detection image into a vector format for input into the loss function training network model; Step 2), construct an improved multi-scale target detection network model based on coordinate attention and design a loss function to train the network model; 2.
1. Construct an improved multi-scale target detection network model based on coordinate attention; 2.
2. Improve the loss function classification weight and train the model to obtain an improved loss function training network model; Step 3) Test the performance of the improved loss function training network model and save the optimal network model parameters.
2. The electronic component X-ray detection method based on convolutional neural network according to claim 1, characterized in that: The specific process of step 1) preprocessing the X-ray image of the electronic component to obtain a clear detection image is as follows: Collect X-ray images of electronic components to obtain detection images; Mark the categories and locations of three types of defects in the inspection images in the data set: damaged deformation, bond wire breakage, and the presence of excess objects; Perform data expansion on the marked detection images; The expanded detection image data is denoised and clarified to enhance the contrast of the detection image and highlight the details. Finally, it is converted into a vector format of RGB3 channels for input into the loss function training network model.
3. The electronic component X-ray detection method based on convolutional neural network according to claim 1, characterized in that: The specific process of constructing the improved coordinate attention-based multi-scale target detection network model in step 2.1 is as follows: A. Using Faster-RCNN as the basic network, multiple cascaded RCNN structures are introduced to continuously optimize the target recognition results; B. Resnet101 residual structure is used as the feature extraction network to fuse the defect feature information extracted by convolution kernels of different sizes to achieve multi-scale feature extraction, which is more suitable for extracting subtle defect features of different shapes in the inspection images of components, such as round bubbles, long strips of redundant objects, etc., thereby improving the recognition effect of the network structure model on the target; C. Integrate the coordinate attention mechanism into the Resnet101 feature extraction network to extract features of both position and spatial dimensions at the same time, generate attention factors for each semantic group, implement adaptive calibration of feature maps, and enhance the semantic representation of the model.
4. The electronic component X-ray detection method based on convolutional neural network according to claim 3 is characterized in that: The specific process of integrating the coordinate attention mechanism into the Resnet101 feature extraction network in step C is: The detection image is X. The CA module first generates attention feature maps with horizontal and vertical coordinate information in the global pooling layer, as shown in formulas (1) and (2); The generated horizontal attention feature map f h and the vertical attention feature map f v Concatenation, convolution and activation function are performed to obtain feature f, and f is split to obtain two attention feature vectors. and Then for Z h and Z v Perform transformation activation and obtain and m v =σ(F w (Z v )), finally, m h and m v Multiply the input feature X at the corresponding position, and finally get the output attention feature Y, as shown in formula (3) Y=X×m h ×m v (3)。 5. The electronic component X-ray detection method based on convolutional neural network according to claim 3, characterized in that: The specific process of improving the loss function classification weight and training the model in step 2.2 to obtain the improved loss function training network model is as follows: The loss function is a function used to evaluate the gap between the model prediction value and the true value, which consists of the problem classification loss function and the defect location loss function, as shown in formula (4). Among them, L loss is the overall loss function of the model, L loc is the positioning loss function, L cls is the classification loss function; During training, the calculation method of the loss function is improved, and the hyperparameters are set to balance the weights of positive and negative samples, as shown in formula (6). Where α is a hyperparameter to balance the weights of different samples. The target positioning loss function is shown in formula (7): Among them, t is the coordinate value of the actual target, is the predicted coordinate value.
6. The electronic component X-ray detection method based on convolutional neural network according to claim 1, characterized in that: The specific process of testing the improved loss function training network model performance and saving the optimal network model parameters in step 3) is: 3.1 Set the batch size to 32, the number of epochs to 500, the learning rate to 0.0001, the weight decay to 0.00001, and the improved loss function to train the network model; 3.2 The improved loss function is tested by calculating the performance evaluation index to verify the performance of the network model trained and save the detection ability of the optimal network model parameters. The recall rate and precision rate of IOU=0.5, the average recall rate and average precision rate of the three categories of damaged deformation, bonding wire breakage and the presence of redundant objects are selected to evaluate the model detection ability.