Complex background anti-interference welding spot multi-class defect detection method
By designing a dual-model architecture combining Opencv's Unet neural network model, the problem of multi-category defect detection of BGA solder joints in complex background is solved, efficient and accurate defect detection is achieved, and detection efficiency and accuracy are improved.
Patent Information
- Application Number
- CN202411697080.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-03-18
AI Technical Summary
The prior art is difficult to efficiently detect multiple defects in BGA solder joints in complex contexts, especially other defects outside the cavity, such as defects, deformation, less balls, cracking, cold welding and bubble bursting.
A Unet neural network model combining Opencv is designed for dual-model architecture for solder joint segmentation and void recognition. By collecting and annotating the global map of solder joint distribution and solder joint images of BGA devices, the solder joint segmentation network and hollow identification network are trained, the morphological characteristics of solder joints and hollows are extracted, and the defect detection logic is designed according to the detection specifications.
Automatic detection of multiple defects in solder joints in complex contexts is realized, which improves the accuracy and efficiency of detection, reduces the risk of defect missed detection, and feeds back the detection information to the operator.
Smart Images
Figure CN119648649A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of industrial integrated circuit detection, and in particular to a complex background interference-resistant solder joint multi-category defect detection method. Background Art
[0002] With the increasing demand for high-density mounting of electronic components on circuits, the application of surface mount technology in the production process of electronic circuits has become increasingly widespread. As a mainstream packaging form among patch components, ball grid array has the advantages of high-density integration, excellent thermal and electrical properties, and easy automated production. However, quality defects are more likely to occur during BGA welding, including voids, defects, deformation, fewer balls, cracks, cold welding, etc., among which solder joint voids are the most common. At present, the electrical production line generally uses an X-ray machine plus manual visual inspection to detect, that is, the solder joints and internal voids are imaged by an X-ray machine, and then the defects are manually identified. This method is greatly affected by subjectivity and has low efficiency. According to the survey, there are many BGA solder joint defect detection algorithms based on X-ray images, which are mainly divided into two categories, namely traditional image recognition algorithms and deep learning algorithms. Research on traditional image recognition algorithms is mainly based on BGA solder joint defect detection under simple backgrounds, and solder joint defect detection is achieved through grayscale-based methods such as threshold segmentation and morphological feature extraction. This type of algorithm is limited to X-ray images with simple backgrounds, lacks application introduction in scenes with complex image backgrounds, and mainly solves the problem of solder joint void defect detection, and rarely involves other defect detection. Research on deep learning algorithms is mainly based on deep learning image semantic segmentation networks, which require setting the architecture of the neural network and algorithm parameters. Its recognition accuracy depends to a large extent on the effectiveness of the extracted features, and solder joint defect detection under complex backgrounds is still difficult to detect.
[0003] In the actual production process of electrical equipment, there are many types of BGA devices with small batches. In addition to voids, the types of defects include defects, deformation, missing balls, cracks, cold welding, bubble bursts, etc. In addition, due to the different quality of X-ray imaging devices, the image differentiation is also quite different. There are also interferences such as capacitors and through holes in the image background. Compared with the simple background in existing research, defect detection is more difficult. Therefore, there is an urgent need for an automatic detection technology for multiple types of defects in solder joints that is suitable for complex backgrounds. Summary of the invention
[0004] Purpose of the invention: The purpose of the present invention is to provide a method for detecting multiple types of defects in solder joints that is resistant to interference in complex backgrounds.
[0005] Technical solution: The present invention comprises the following steps:
[0006] (1) Collect the global distribution map of BGA device solder joints and solder joint images, and annotate the solder joints and voids respectively to form a solder joint segmentation sample set and a void defect sample set, so as to prepare for the training and verification of the solder joint segmentation network and the void recognition network;
[0007] (2) Design the network parameters of the solder joint segmentation network and the void recognition network according to the characteristics of the samples, and train and verify the solder joint segmentation network and the void recognition network;
[0008] (3) The two models obtained in step (2) are thermally loaded by using a model thermal loading method, and the global distribution map of the solder joints to be detected is used as the input of the solder joint segmentation model to perform solder joint segmentation. After obtaining a single solder joint, each solder joint is used as the input of the void recognition network in turn to extract void features;
[0009] (4) According to the design defect detection logic, including void defects, defects, deformation, missing balls, cold welding, cracking and bubble bursting;
[0010] (5) Feedback the defect information of all solder joints obtained in step (4) to the operator.
[0011] Furthermore, the step (1) includes collecting an existing global map of solder joint distribution of BGA-type devices, marking the solder joints therein to obtain solder joint information and solder joint label information, extracting a single solder joint map through the solder joint information, marking the void features inside a single solder joint, and obtaining a void defect sample set.
[0012] Furthermore, the step (2) includes training and validating according to the prepared sample set, and storing the model parameters after iterative convergence and accuracy is satisfied in preparation for model prediction.
[0013] Furthermore, the step (3) includes loading the solder joint segmentation model and the solder joint void recognition model by means of model hot loading, using the global image of the solder joint to be detected as a sample input of the solder joint segmentation model to obtain a single solder joint after segmentation, and then using each solder joint as a sample input of the void recognition model to obtain the void distribution inside each solder joint.
[0014] Furthermore, in step (3), the morphological features of a single solder joint and the morphological features of a cavity are extracted by using Opencv technology.
[0015] Furthermore, the void defect detection logic of step (4) includes, based on the void area and solder joint area extracted in step (3), if the ratio of the sum of all void areas in a single solder joint to the solder joint area is greater than a threshold, it is unqualified; if the ratio of the area of the largest void in a single solder joint to the solder joint area is greater than a threshold, it is unqualified.
[0016] Furthermore, the defect, deformation and cold welding detection logic of step (4) includes contour extraction based on Opencv, extraction of the shortest distance within the contour, and calculation of the ratio of the shortest diameter to the longest diameter. If the ratio is less than a threshold, it is unqualified.
[0017] Furthermore, the step (4) of the less-ball detection logic includes calculating the current solder joint area and the standard solder joint area based on the Opencv morphological feature extraction technology, that is, taking the ratio of the average area of a normal solder joint without defects. If the ratio is less than a threshold, it is unqualified.
[0018] Furthermore, the crack detection logic of step (4) includes performing ellipse fitting on the crack based on the feature fitting technology of Opencv, calculating the ratio of the major axis to the minor axis of the ellipse, and if the ratio is greater than a threshold, it is unqualified.
[0019] Furthermore, the bubble burst detection logic of step (4) includes calculating the bubble centroid based on Opencv morphological feature extraction technology, that is, the ratio of the distance between the void centroid extracted in step 3 and the solder point centroid and the solder point radius. If the ratio is greater than a threshold, it is considered that the bubble is close to the edge and has a tendency to burst, and is judged as unqualified.
[0020] Beneficial effect: Compared with the prior art, the present invention has the following significant advantages: the present invention adopts the Unet neural network model combined with Opencv to design a dual-model architecture for solder joint segmentation and void recognition, which, on the one hand, makes the solder joint segmentation more accurate under complex backgrounds, and on the other hand, can effectively identify the void features inside the solder joint; based on the extracted morphological features of single solder joints and voids inside solder joints, defect detection logic such as voids, defects, deformation, missing balls, cold welding, cracking, and bubble bursting is designed according to the detection specifications, which realizes automatic detection of multiple types of solder joint defects, and feeds back the defect information to the operator, ensuring the accuracy of defect detection while improving the defect detection efficiency, and effectively suppressing the risk of device damage after missed detection of solder joint defects. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 is a flow chart of the present invention;
[0022] Figure 2 This is the schematic diagram of multi-type defect detection in solder joints. DETAILED DESCRIPTION
[0023] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0024] like Figure 1 As shown, the present invention comprises the following steps:
[0025] (1) Collect the existing global distribution map of solder joints of BGA devices, annotate the solder joints to obtain solder joint information and solder joint label information, extract a single solder joint map based on the solder joint information, annotate the void features inside a single solder joint, and obtain a void defect sample set to prepare for the training of solder joint segmentation model and void recognition model;
[0026] (2) Design the network parameters for solder joint segmentation and solder joint void identification according to the sample characteristics, perform training and verification based on the prepared sample set, and store the model parameters after iterative convergence and accuracy is met in preparation for model prediction;
[0027] (3) Figure 2 As shown in the figure, the solder joint segmentation model and solder joint void recognition model are loaded by hot loading of the model, and the global image of the solder joint to be detected is used as the sample input of the solder joint segmentation model to obtain the single solder joint after segmentation, and then each solder joint is used as the sample input of the void recognition model to obtain the void distribution inside each solder joint. The Opencv morphological feature extraction technology is used to extract the void features of each solder joint and each solder joint to prepare for defect detection;
[0028] (4) According to the solder joint features and the void features inside the solder joint obtained in step (3), the defect detection logic is designed based on the "HF.131614S0110PCBA Inspection Specification", including voids, defects, deformation, missing balls, cold welding, cracking, bubble bursting, etc. The specific detection logic is as follows:
[0029] a. Void defect: Based on the void area and solder joint area extracted in step (3), if the ratio of the sum of all void areas in a single solder joint to the solder joint area is greater than the threshold, the solder joint is unqualified; if the ratio of the area of the largest void in a single solder joint to the solder joint area is greater than the threshold, the solder joint is unqualified;
[0030] b. Defect / deformation / cold welding: Calculate the ratio of the shortest diameter to the longest diameter based on Opencv contour extraction, shortest distance extraction within the contour and other technologies. If the ratio is less than the threshold, it is unqualified;
[0031] c. Fewer balls: Based on the Opencv morphological feature extraction technology, the current solder joint area and the standard solder joint area are calculated, and the ratio of the average area of the normal solder joint without defects is taken. If the ratio is less than the threshold, it is unqualified;
[0032] d. Cracking: Based on the feature fitting technology of Opencv, the cracks and the thin crack features have been extracted in step (3) of the void identification, and the ellipse fitting is performed to calculate the ratio of the major and minor axes of the ellipse. If the ratio is greater than the threshold, it is unqualified;
[0033] e. Bubble bursting: The bubble centroid is calculated based on Opencv morphological feature extraction technology, that is, the ratio of the distance between the void centroid extracted in step (3) and the solder point centroid to the solder point radius. If the ratio is greater than the threshold, it is considered that the bubble is close to the edge and has a tendency to burst, and is judged as unqualified.
[0034] (5) The operator confirms and records the returned solder joint defect detection information.
Claims
1. A method for detecting multiple types of defects in solder joints with anti-interference in complex backgrounds, characterized in that: The steps include: (1) Collect the global distribution map of BGA device solder joints and solder joint images, and annotate the solder joints and voids respectively to form a solder joint segmentation sample set and a void defect sample set, so as to prepare for the training and verification of the solder joint segmentation network and the void recognition network; (2) Design the network parameters of the solder joint segmentation network and the void recognition network according to the characteristics of the samples, and train and verify the solder joint segmentation network and the void recognition network; (3) The two models obtained in step (2) are thermally loaded by using a model thermal loading method, and the global distribution map of the solder joints to be detected is used as the input of the solder joint segmentation model to perform solder joint segmentation. After obtaining a single solder joint, each solder joint is used as the input of the void recognition network in turn to extract void features; (4) According to the design defect detection logic, including void defects, defects, deformation, missing balls, cold welding, cracking and bubble bursting; (5) Feedback the defect information of all solder joints obtained in step (4) to the operator.
2. The complex background anti-interference solder joint multi-class defect detection method according to claim 1 is characterized in that: The step (1) includes collecting an existing global map of solder joint distribution of BGA-type devices, annotating the solder joints therein to obtain solder joint information and solder joint label information, extracting a single solder joint map through the solder joint information, annotating the void features inside the single solder joint, and obtaining a void defect sample set.
3. The complex background anti-interference solder joint multi-class defect detection method according to claim 1 is characterized in that: The step (2) includes training and validating according to the prepared sample set, and storing the model parameters after iterative convergence and accuracy is satisfied in preparation for model prediction.
4. The complex background anti-interference solder joint multi-class defect detection method according to claim 1 is characterized in that: The step (3) includes loading the solder joint segmentation model and the solder joint void recognition model in a model hot loading manner, using the global image of the solder joint to be detected as a sample input of the solder joint segmentation model to obtain a single solder joint after segmentation, and then using each solder joint as a sample input of the void recognition model to obtain the void distribution inside each solder joint.
5. The complex background anti-interference solder joint multi-class defect detection method according to claim 1 is characterized in that: In the step (3), the morphological features of a single solder joint and the morphological features of a cavity are extracted by using Opencv technology.
6. The complex background anti-interference solder joint multi-class defect detection method according to claim 1 is characterized in that: The void defect detection logic of step (4) includes, based on the void area and solder joint area extracted in step (3), if the ratio of the sum of all void areas in a single solder joint to the solder joint area is greater than a threshold, it is unqualified; if the ratio of the area of the largest void in a single solder joint to the solder joint area is greater than a threshold, it is unqualified.
7. The complex background anti-interference solder joint multi-class defect detection method according to claim 1 is characterized in that: The defect, deformation and cold welding detection logic of step (4) includes contour extraction based on Opencv, extraction of the shortest distance within the contour, and calculation of the ratio of the shortest diameter to the longest diameter. If the ratio is less than a threshold, it is unqualified.
8. The complex background anti-interference solder joint multi-class defect detection method according to claim 1 is characterized in that: The step (4) of the less-ball detection logic includes calculating the current solder joint area and the standard solder joint area based on the Opencv morphological feature extraction technology, that is, taking the ratio of the average area of a normal solder joint without defects. If the ratio is less than a threshold, it is unqualified.
9. The complex background anti-interference solder joint multi-class defect detection method according to claim 1 is characterized in that: The crack detection logic of step (4) includes performing ellipse fitting on the crack based on the feature fitting technology of Opencv, calculating the ratio of the major and minor axes of the ellipse, and if the ratio is greater than a threshold, it is unqualified.
10. The complex background anti-interference solder joint multi-class defect detection method according to claim 1, characterized in that: The bubble burst detection logic of step (4) includes calculating the bubble centroid based on Opencv morphological feature extraction technology, that is, the ratio of the distance between the void centroid extracted in step (3) and the solder point centroid and the solder point radius. If the ratio is greater than a threshold, it is considered that the bubble is close to the edge and has a tendency to burst, and is judged as unqualified.