Intelligent automatic test system for welding spot holes and bridging defects

Through the intelligent automatic test system for solder joint voids and bridging defects, using data collection, void and bridging defect detection modules and confidence analysis, a structured inspection report is generated, which solves the problem of low accuracy in traditional inspection methods and realizes high-precision solder joint defect detection and management.

CN120823188AInactive Publication Date: 2025-10-21江门市科能电子有限公司
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202510992199.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-18
Publication Date
2025-10-21
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional methods for detecting voids and bridging defects in solder joints are not very accurate. Manual inspection is prone to omissions and misjudgments, while machine vision has limited ability to extract features from complex solder joints, making it difficult to meet the quality control requirements of high-precision manufacturing scenarios.

Method used

The intelligent automatic testing system for solder joint voids and bridging defects is comprised of a data acquisition module, a void defect detection module, a bridging defect detection module, a confidence analysis module, and a report generation module. The data acquisition module generates standard image data using an industrial camera and image preprocessing. The void defect detection module calculates the area ratio by identifying low-density areas. The bridging defect detection module calculates the minimum spacing between solder joints. The confidence analysis module performs secondary analysis. The report generation module produces a structured inspection report.

Benefits of technology

It improves the accuracy of solder joint void and bridging defect detection, reduces the false positive rate, realizes closed-loop management from detection to decision-making, and generates clear and actionable quality information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120823188A_ABST
    Figure CN120823188A_ABST
Patent Text Reader

Abstract

The invention provides a welding spot cavity and bridging defect intelligent automatic test system, and relates to the technical field of defect detection, and the method comprises the steps: a data collection module is used for collecting image data of a to-be-detected welding spot, carrying out the standardization processing, and obtaining standard image data; the void defect detection module is used for identifying a low-density region in a standard image data welding spot, calculating an area proportion, and if the area proportion exceeds a preset area threshold value, judging that the welding spot is a void defect; the bridging defect detection module is used for calculating the minimum distance between standard image data welding spots, and if the minimum distance is smaller than a preset distance threshold value, a bridging defect is judged; the confidence coefficient analysis module is used for obtaining the detection confidence coefficient and re-collecting image data for secondary analysis of a detection result lower than a confidence coefficient threshold value; and the report generation module is used for generating a structured detection report comprising the defect type and the defect severity level. The technical problem that in the prior art, welding spot cavities and bridging defect detection accuracy is not high is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of defect detection, and in particular to an intelligent automatic testing system for solder joint voids and bridging defects. Background Art

[0002] In the field of electronics manufacturing, the quality of solder joints directly affects the reliability and service life of electronic equipment. Among them, voids (low-density gaps inside solder joints) and bridging (improper connection of adjacent solder joints) are two common defects.

[0003] However, traditional inspection methods rely on manual visual inspection or simple machine vision. Human fatigue can lead to missed or misjudgment, and it's difficult to quantify the extent of defects. Traditional machine vision has limited feature extraction capabilities for complex solder joints, making it difficult to meet the quality control requirements of high-precision manufacturing scenarios. Consequently, the accuracy of detecting solder voids and bridging defects is low. Summary of the Invention

[0004] The present invention addresses the technical problem of low detection accuracy of solder joint voids and bridging defects in the prior art and provides an intelligent automatic testing system for solder joint voids and bridging defects.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: The present invention provides an intelligent automatic testing system for solder joint voids and bridging defects, comprising: The data acquisition module is used to collect image data of the solder joint to be tested, perform standardization processing, and obtain standard image data; A void defect detection module is used to identify low-density areas inside the solder joints in the standard image data, calculate the area ratio, and determine if it exceeds a preset area threshold as a void defect; A bridge defect detection module is used to calculate the minimum distance between solder joints in the standard image data, and if it is less than a preset distance threshold, it is determined to be a bridge defect; a confidence analysis module, configured to obtain the detection confidences of the cavity defect detection module and the bridge defect detection module respectively, and re-collect image data from different angles for secondary analysis of detection results below a confidence threshold; The report generation module is used to generate a structured detection report including defect type and defect severity level.

[0006] The beneficial effects of the present invention are: Compared with the existing technology, this application first collects image data of the solder joints to be tested through a data acquisition module, performs standardization processing, obtains standard image data, and converts the original, possibly interfering images into unified standard, high-quality image data, laying the foundation for the defect detection accuracy of the entire system. Secondly, through the void defect detection module, the low-density area inside the solder joints of the standard image data is identified and the area ratio is calculated. If it exceeds the preset area threshold, it is determined to be a void defect. Through artificial intelligence means, accurate identification and determination of void defects are achieved. Thirdly, the bridge defect detection module is used to calculate the minimum spacing between solder joints in the standard image data. If it is less than the preset distance threshold, it is determined to be a bridge defect. Based on the graph neural network, accurate identification and determination of bridge defects are achieved. Furthermore, the confidence analysis module is used to obtain the detection confidence of the void defect detection module and the bridge defect detection module respectively. For detection results below the confidence threshold, the image data is re-collected from different angles for secondary analysis, effectively filtering out low-confidence results, improving the accuracy and stability of detection, and avoiding misjudgments caused by single detection deviations. Finally, the report generation module is used to generate structured inspection reports including defect types and defect severity levels, converting complex inspection data into clear and actionable quality information, realizing closed-loop management of solder joint defects from detection to decision-making.

[0007] Through the above technical solution, this application collects image data of the solder joints to be tested, performs standardized processing, and then combines it with an artificial intelligence model to detect void defects and bridging defects. Through confidence analysis, unreliable detection results are re-analyzed to reduce the false positive rate, and ultimately generate a structured inspection report that includes defect type and defect severity level. This improves the accuracy of solder joint void and bridging defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a schematic diagram of the structure of the intelligent automatic testing system for solder joint voids and bridging defects provided by the present invention; Figure 2 This is a structural schematic diagram of the void defect detection module in the intelligent automatic testing system for solder joint voids and bridging defects provided by the present invention.

[0009] In the accompanying drawings, the components represented by the reference numerals are as follows: Data acquisition module 11, void defect detection module 12, bridge defect detection module 13, confidence analysis module 14, report generation module 15. DETAILED DESCRIPTION

[0010] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0011] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the specified features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0012] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or illustration". Any embodiment of the present invention described as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed herein.

[0013] Example 1, as Figure 1 As shown, an embodiment of the present invention provides an intelligent automatic testing system for solder joint voids and bridging defects, which includes: a data acquisition module 11, a void defect detection module 12, a bridging defect detection module 13, a confidence analysis module 14, and a report generation module 15.

[0014] The data acquisition module 11 is used to acquire image data of the solder joint to be tested, perform standardization processing, and obtain standard image data.

[0015] During the solder joint image data acquisition process, it may be affected by various factors such as light interference, equipment jitter, lens contamination, etc., resulting in noise, blur or insufficient contrast in the original image, thereby reducing the image data quality, interfering with the subsequent defect recognition algorithm's accurate extraction and analysis of solder joint features, and affecting the accuracy and reliability of defect detection.

[0016] In order to solve the above problems, the present invention collects image data of the solder joint to be tested, performs standardization processing, and obtains standard image data.

[0017] Specifically, the data acquisition module 11 includes: An industrial imaging unit is used to capture images of the solder joint area to be tested using an industrial camera to obtain original image data; The image preprocessing unit is used to perform grayscale conversion, noise reduction and histogram equalization processing on the original image data to generate standard image data.

[0018] In the present embodiment, an industrial camera is first used to capture images of the solder joint area to be tested, obtaining raw image data. For example, compared to ordinary cameras, industrial cameras have higher resolution, frame rate, and stability, and can clearly capture the subtle features of the solder joint, ensuring the quality of the raw image data and providing basic data for subsequent defect detection.

[0019] Secondly, grayscale conversion, noise reduction and histogram equalization are performed on the original image data to generate standard image data. For example, the raw image data acquired by the industrial imaging unit is standardized to eliminate interference, enhance useful information, and provide high-quality standard image data for the subsequent defect detection module. Specifically, the standardization process includes grayscale conversion, noise reduction, and histogram equalization. Grayscale conversion converts a color image into a grayscale image. A color image contains information from three channels: red, green, and blue. Grayscale conversion can simplify the image data dimension and reduce the amount of calculation while retaining the basic outline and brightness characteristics of the solder joint, facilitating subsequent feature extraction. Noise reduction is used to remove noise from the image. The raw image may introduce noise due to factors such as equipment performance and ambient light. This noise will interfere with the judgment of the solder joint features. Noise reduction algorithms (such as Gaussian filtering and median filtering) can be used to smooth the image and highlight the true features of the solder joint. Histogram equalization is used to adjust the contrast of the image and stretch the grayscale value range of the image to make the details in the dark area clearer and the features in the bright area more prominent, thereby enhancing the difference between the solder joint and the background, and between different areas within the solder joint, creating favorable conditions for subsequent defect identification.

[0020] In summary, compared to the existing technology, this application uses a data acquisition module to collect image data of the solder joint to be tested, perform standard processing, and obtain standard image data. In this way, the original, potentially noisy image is converted into unified, high-quality image data, laying the foundation for the defect detection accuracy of the entire system.

[0021] The void defect detection module 12 is used to identify the low-density area inside the solder joint of the standard image data, calculate the area ratio, and determine it as a void defect if it exceeds a preset area threshold.

[0022] In the solder joint defect detection scenario, low-density areas are identified as voids because: From a physical point of view, a normal solder joint is a dense connection structure formed by the melting and solidification of welding materials (such as solder). There should be no obvious defects such as gaps or bubbles inside. Voids refer to cavities that are not filled with solder and are formed inside the solder joint due to problems such as failure to discharge gas or insufficient solder during the welding process. This cavity area lacks dense welding material and is physically a low-density area relative to the high-density structure of the surrounding normal solder.

[0023] From the perspective of image features, in the solder joint image, there are significant differences in the optical properties between the void area and the normal solder area: normal solder has high density and uniform structure, and its reflection and absorption patterns of light are stable, which appears as high-density features with uniform grayscale values ​​in the image. However, the void area has air or loose structure inside, which changes the light penetration and scattering pattern. Therefore, it will show low-density visual features with low grayscale values ​​and irregular texture (such as dark areas and blurred areas) in the image.

[0024] Therefore, by identifying low-density areas in the image, the visual features in the image that conform to the physical characteristics of the void can be directly associated with the void defect, thereby achieving accurate detection of the void defect.

[0025] To address the above problem, the present application identifies the low-density area inside the solder joint in the standard image data and calculates the area ratio. If it exceeds a preset area threshold, it is determined to be a void defect.

[0026] Specifically, if Figure 2 As shown, the cavity defect detection module 12 includes: a low-density area recognition unit, configured to input the standard image data into a first defect detection model, output a pixel-level segmentation mask image, and mark the low-density area inside the solder joint; An area calculation unit, configured to calculate the ratio of the number of pixels in the low-density area to the total number of pixels in the solder joint according to the segmentation mask image, and obtain a void area ratio; A threshold determination unit, configured to query a corresponding area threshold from a preset area threshold table according to the solder joint type as a preset area threshold, wherein the solder joint type includes a single solder joint, an array solder joint, and a BGA solder joint; The void defect determination unit is configured to compare the void area ratio with a preset area threshold corresponding to the solder joint type, and determine that the void defect occurs if the void area ratio exceeds the preset area threshold.

[0027] In the embodiment of this application, Figure 2As shown, the standard image data is first input into the first defect detection model, which outputs a pixel-level segmentation mask that marks low-density areas within the solder joint. For example, the standard image data is input into the pre-trained first defect detection model, which outputs a pixel-level segmentation mask. This is an image of the same size as the original image, in which only the low-density areas within the solder joint (i.e., the void areas) are annotated with a specific marker (e.g., a red mask), achieving pixel-level precision in void location. For example, if a 1000×1000 pixel standard single solder joint image is input into the first defect detection model, the low-density areas within the image will be marked in red in the output segmentation mask. These low-density areas are the void areas.

[0028] Secondly, the ratio of the number of pixels in the low-density area to the total number of pixels in the solder joint is calculated based on the segmentation mask image to obtain the void area ratio, where void area ratio = number of pixels in the low-density area / total number of pixels in the solder joint. For example, the void area ratio is obtained by counting the number of pixels marked by the mask (i.e., the total number of pixels in the void area) and calculating it with the total number of pixels in the entire solder joint area. For example, for a standard single solder joint image of 1000×1000 pixels, the segmentation mask image output by the first defect detection model marks a low-density area of ​​200,000 pixels in red. In this case, the void area ratio = 200,000 / (1000×1000) * 100% = 20%. In this way, the visual void size is converted into a quantifiable value, providing a reliable data basis for subsequent judgment.

[0029] Again, according to the type of solder joint, the corresponding area threshold is queried from the preset area threshold table as the preset area threshold, wherein the solder joint types include single solder joints, array solder joints and BGA solder joints. Specifically, different types of solder joints have different tolerances to voids. For example, BGA solder joints have a relatively high tolerance to voids due to their structural characteristics. Therefore, different types of solder joints should be equipped with corresponding area thresholds. Based on theoretical data and a large number of experiments, this application recommends setting the area threshold of a single solder joint to 15%, the area threshold of an array solder joint to 10%, and the area threshold of a BGA solder joint to 25%, thereby constructing a preset area threshold table. Those skilled in the art can dynamically adjust the area threshold according to the actual material characteristics to construct an area threshold table with higher adaptability. For example, based on the area threshold of a single solder joint of 15%, the area threshold of an array solder joint of 10%, and the area threshold of a BGA solder joint of 25%, a preset area threshold table is constructed, from which the area threshold of a single solder joint is obtained to be 15%.

[0030] Finally, the void area percentage is compared with the preset area threshold for the corresponding solder joint type. If it exceeds the preset area threshold, it is determined to be a void defect. For example, if the calculated void area percentage of a single solder joint is 20%, and the area threshold for a single solder joint is 15% as found in the preset area threshold table, since the void area percentage of 20% exceeds the area threshold of 15% for a single solder joint, it is determined to be a void defect.

[0031] Furthermore, the training steps of the "first defect detection model" include: A data preparation unit is used to collect a sample solder joint image data set, and perform pixel-level annotation on the void area within each sample solder joint image data, and use a red mask to mark the low-density area as a sample low-density area annotation set; A model construction unit is configured to construct a first defect detection model based on a U-Net architecture, wherein the first defect detection model includes an encoder and a decoder, wherein the encoder is configured to extract image features, and the decoder implements feature mapping through a deconvolution layer; The model training unit is used to use the sample solder joint image data set and the sample low-density area annotation set to iteratively supervise the training of the first defect detection model, and complete the training after the test accuracy converges.

[0032] In an embodiment of the present application, a sample weld image data set is first collected, and the void regions within each sample weld image data are annotated at the pixel level. Low-density regions are marked with a red mask to form a sample low-density region annotation set. For example, a large number of sample weld images containing various void conditions are collected. To improve the generalization ability and prediction accuracy of the model, image data with different lighting, angles, and weld types can be collected as a sample weld image data set. The void regions in each image are then annotated at the pixel level. For example, the void boundaries and internal regions are manually or semi-automatically marked pixel by pixel with a red mask. This serves as a sample low-density region annotation set to provide input and output pairs for learning for the model.

[0033] Secondly, a first defect detection model is constructed based on the U-Net architecture. The first defect detection model includes an encoder and a decoder, wherein the encoder is used to extract image features and the decoder implements feature mapping through a deconvolution layer. For example, the model is constructed using the U-Net architecture, which is a deep learning architecture that performs well in image segmentation tasks. It mainly includes an encoder and a decoder. The encoder gradually extracts low-level features (such as edges, textures, etc.) and high-level features (such as the overall shape of solder joints, the difference between voids and normal areas, etc.) of the image through convolution layers to achieve compression of image information and capture of key features; the decoder gradually restores the image resolution through deconvolution layers (transposed convolution), maps the features extracted by the encoder back to the original image size, and finally outputs a segmentation mask map of the same size as the input image to achieve pixel-level positioning of the void area.

[0034] Finally, the first defect detection model is iteratively supervised trained using the sample solder joint image dataset and the sample low-density region annotation set, completing training after test accuracy converges. For example, the training process of the first defect detection model can be implemented through the following technical paths: 1. Data preparation: The sample solder joint image dataset and the sample low-density region annotation set are divided into a training set, a validation set, and a test set in a ratio of 7:1.5:1.5. 2. Model training: The sample solder joint image dataset is used as the model input, and the sample low-density area annotation set is used as the supervision label. The Dice loss function (or cross entropy loss function) is used to quantify the model prediction deviation. For example, by calculating the overlap between the predicted mask map and the manually annotated real mask map (Dice coefficient), the lower the overlap, the higher the loss value, which provides a clear direction for parameter optimization. The prediction result is compared with the real annotation to calculate the loss value. With the help of optimizers such as Adam, the error is transferred to each layer through the back propagation mechanism, and the convolution kernel weights, bias and other parameters are dynamically adjusted to reduce the prediction deviation of the next round. After each round of training, the performance is evaluated with the validation set. If the accuracy of the validation set decreases for multiple consecutive rounds (indicating overfitting), the early stopping mechanism is triggered to save the optimal parameters. When the accuracy of the training set and the validation set, Dice When indicators such as the coefficient stabilize at the preset threshold (e.g., the variation is less than 0.1%), the model reaches a convergence state. Finally, after passing the test set verification (e.g., the accuracy rate is ≥ 95%), the training is completed, and the first defect detection model is obtained. It can adapt to different solder joint types and image conditions, accurately identify void areas, and provide reliable basic segmentation results for the quantitative determination of void defects.

[0035] In summary, compared to existing technologies, this application uses a void defect detection module to identify low-density areas within the solder joints in the standard image data, calculate the area ratio, and determine if it exceeds a preset area threshold as a void defect. In this way, through artificial intelligence, accurate identification and determination of void defects can be achieved.

[0036] The bridge defect detection module 13 is used to calculate the minimum distance between the solder joints in the standard image data, and if it is less than a preset distance threshold, it is determined to be a bridge defect.

[0037] In the solder joint defect detection scenario, bridging defects can be determined by the minimum spacing between solder joints. This is because: From a physical perspective, normal solder joints should be independent conductive connections. During design and manufacturing, a safe spacing is reserved between adjacent solder joints based on circuit functionality, current load, and other requirements. This spacing prevents accidental conduction between solder joints, preventing circuit shorts or signal interference. However, bridging defects occur when excess solder connects adjacent solder joints during the soldering process (e.g., excessive solder, offset solder paste printing, abnormal reflow temperatures, etc.), creating a conductive path that should not exist. This connection significantly reduces the actual distance between solder joints. When the solder joints are completely bridged, the spacing may approach zero. Even with minor localized bridging, the spacing will be less than the designed safety threshold.

[0038] Therefore, the minimum distance between adjacent solder joints can be obtained and compared with the preset distance threshold to determine whether a bridging defect exists.

[0039] To address the above problem, the present application calculates the minimum spacing between solder joints in the standard image data. If the spacing is less than a preset distance threshold, it is determined to be a bridging defect.

[0040] Specifically, the bridge defect detection module 13 includes: A contour extraction unit, configured to extract the edge contour of the solder joint in the standard image data using a Canny edge detection algorithm; An effective contour screening unit, used for screening effective solder joint contours in the solder joint edge contours; A centroid calculation unit, configured to calculate the centroid coordinates of each valid solder joint based on the valid solder joint contour; a spacing prediction unit, configured to input the centroid coordinates of the valid welds into a second defect detection model constructed based on a graph neural network, and calculate and output the spacing between adjacent weld pairs as the minimum spacing between welds; The bridging defect determination unit is configured to compare the minimum spacing between the solder joints with a preset distance threshold, and if the minimum spacing between the solder joints is less than the preset distance threshold, determine that a bridging defect exists in the corresponding solder joint pair.

[0041] In the present embodiment, the Canny edge detection algorithm is first used to extract the edge contours of the solder joints in the standard image data. For example, the Canny algorithm uses a multi-stage process, such as Gaussian filtering for noise reduction, calculation of gradient strength and direction, non-maximum suppression, and dual-threshold detection, to accurately identify areas of sudden grayscale changes in the image, thereby extracting the edge contours between the solder joint and the background, and between the solder joint and other solder joints. For example, in an image containing multiple adjacent solder joints, the Canny algorithm can output the complete edge lines of each solder joint, providing basic contour information for subsequent analysis.

[0042] Next, valid solder joint contours are screened from the solder joint edge contours. For example, after contour extraction, noise contours (e.g., false edges caused by background interference or image defects) or incomplete contours (e.g., broken edges caused by partial occlusion of the solder joint) may be included. Contours that meet the characteristics of valid solder joints are screened using preset rules (e.g., contour area range, shape complexity, and closure). For example, contours with an excessively small area (likely due to noise) or severe irregularities (likely due to image defects) are excluded, and only closed contours with a reasonable area and a shape close to a circle or ellipse, that is, those that meet the typical solder joint morphology, are retained as valid solder joint contours.

[0043] Next, the centroid coordinates of each valid weld are calculated based on the valid weld contours. For example, a two-dimensional coordinate system is established on the standard image data. For each valid weld contour selected, the specific coordinate information of its geometric center in the two-dimensional coordinate system is calculated as the centroid coordinates. This simplifies the irregular weld contour into a precise coordinate point, facilitating the subsequent calculation of the distance between welds. For example, in a unified two-dimensional coordinate system, the centroid coordinates of two adjacent welds are (100, 200) and (150, 250).

[0044] Furthermore, the centroid coordinates of the valid welds are input into a second defect detection model constructed based on a graph neural network, and the spacing between adjacent weld pairs is calculated and output as the minimum spacing between welds. For example, the centroid coordinates of the valid welds are input into the second defect detection model constructed based on a graph neural network. Graph neural networks excel at processing relationships between nodes and can automatically identify which centroids belong to adjacent weld pairs and calculate the shortest distance between them as the minimum spacing between welds. For example, by analyzing the distribution of centroid coordinates, the model determines that welds (100, 200) and welds (150, 250) are adjacent pairs, and calculates the spacing between them to be 50 pixels, which is the minimum spacing between welds.

[0045] Finally, the minimum distance between the solder joints is compared with a preset distance threshold. If the minimum distance between the solder joints is less than the preset distance threshold, it is determined that the corresponding solder joint pair has a bridging defect. For example, the preset distance threshold can be set according to the solder joint type and process standard. For example, the preset distance threshold is set to 30 pixels. Those skilled in the art can dynamically adjust it according to actual conditions. If the minimum distance between the solder joints output by the model is less than the preset distance threshold, it is determined that the corresponding solder joint pair has a bridging defect. For example, if the preset distance threshold is 30 pixels and the minimum distance between the solder joints (100, 200) and the solder joint (150, 250) is 50 pixels, and the minimum distance between the solder joints is less than the preset distance threshold, it is determined that the solder joints (100, 200) and the solder joints (150, 250) have a bridging defect. In this way, accurate detection of bridging defects is achieved.

[0046] Specifically, the training steps of the "second defect detection model" include: a sample data preparation unit, configured to collect a set of sample images containing bridging defects, extract edge contours of solder joints in the sample images and screen valid solder joint contours, calculate centroid coordinates based on the screened valid solder joint contours to form a set of sample solder joint centroid coordinates, and mark actual spacing values ​​of adjacent solder joint pairs in the sample images to form a set of sample solder joint spacings; A model building unit, configured to build a second defect detection model based on a graph neural network; The model training unit is used to iteratively supervise the training of the second defect detection model using the sample solder joint centroid coordinate set as input and the sample solder joint spacing set as supervision labels, and complete the training after the test accuracy converges.

[0047] In an embodiment of the present application, a set of sample images containing bridging defects is first collected. The edge contours of the solder joints in the sample images are extracted and valid solder joint contours are screened. Based on the screened valid solder joint contours, their centroid coordinates are calculated to form a set of sample solder joint centroid coordinates. The actual spacing values ​​of adjacent solder joint pairs in the sample images are annotated to form a set of sample solder joint spacings. For example, a large number of sample images containing bridging defects and normal solder joints are collected, covering different solder joint types, arrangements, and lighting conditions to improve the generalization ability of the model. These images are used as a set of sample images. For each image, valid solder joint contours are extracted and screened, and their centroid coordinates are calculated to form a set of sample solder joint centroid coordinates, for example, including multiple centroid coordinates (x1, y1), (x2, y2), etc. At the same time, the actual physical spacing of all adjacent solder joint pairs in these samples, such as 25 pixels, 40 pixels, etc., is manually annotated or obtained using high-precision measurement tools. This set of sample solder joint spacings provides input and output pairs for the model.

[0048] Next, a second defect detection model is constructed based on a graph neural network. For example, the core of the graph neural network considers the centroids of solder joints as nodes and the potential connections between nodes as edges. The coordinates (x, y) of each centroid are used as the initial features of the node. The model automatically learns the adjacency relationships between nodes, determines which nodes belong to adjacent solder joints, and uses a multi-layer neural network to exchange information between nodes and update their feature representations, such as fusing the position information of adjacent nodes. Ultimately, it predicts the distance between each pair of adjacent nodes (solder joints).

[0049] Finally, the second defect detection model is trained through iterative supervision using the sample solder joint centroid coordinate set as input and the sample solder joint spacing set as supervision labels, and the training is completed after the test accuracy converges. Exemplarily, the corresponding data of the sample solder point centroid coordinate set and the sample solder point spacing set are divided into training set, validation set, and test set in a ratio of 7:1.5:1.5. The sample solder point centroid coordinate set is used as input and the sample solder point spacing set is used as a supervision label. The model outputs the predicted spacing value, and the deviation between the predicted value and the actual value in the sample solder point spacing set is calculated through a loss function (such as mean square error MSE). The larger the deviation, the higher the loss value. Subsequently, the optimizer (such as Adam) is used to backpropagate the error and adjust the model parameters (such as the weight of the message passing layer) to gradually reduce the prediction deviation. The generalization ability of the model is monitored through the validation set. When the prediction accuracy on the test set (such as the spacing prediction error <5%) converges stably, the training is stopped to obtain the second defect detection model. The trained second defect detection model can accurately predict the minimum spacing between adjacent solder point pairs based on the centroid coordinates of any solder point, providing reliable data for bridging defect judgment.

[0050] In summary, compared to existing technologies, this application uses a bridging defect detection module to calculate the minimum spacing between solder joints in the standard image data. If it is less than a preset distance threshold, it is determined to be a bridging defect. In this way, based on a graph neural network, accurate identification and determination of bridging defects is achieved.

[0051] The confidence analysis module 14 is used to obtain the detection confidence of the cavity defect detection module and the bridge defect detection module respectively, and for the detection results below the confidence threshold, re-collect image data from different angles for secondary analysis.

[0052] Poor image quality may occur due to problems such as light interference, angle deviation, and equipment noise during the image acquisition process, or the model may have output deviations due to insufficient feature extraction and insufficient generalization ability. Such factors can easily cause fluctuations in the reliability of single detection results, leading to errors in detection results such as misjudgment, missed defect detection, or inaccurate quantitative evaluation.

[0053] To address the above issues, the present application obtains the detection confidence of the void defect detection module and the bridge defect detection module respectively, and for detection results below the confidence threshold, re-collects image data from different angles for secondary analysis.

[0054] Specifically, the confidence analysis module 14 includes: a first confidence analysis unit, configured to obtain an output accuracy of the first defect detection model as a first model confidence; a second confidence analysis unit, configured to obtain an output accuracy of the second defect detection model as a second model confidence; a threshold comparison unit, configured to compare the first model confidence and the second model confidence with a preset confidence threshold; The secondary analysis unit is used to trigger the data acquisition module to re-collect image data from different angles when the first model confidence or the second model confidence is lower than the preset confidence threshold, and input the image data into the corresponding defect detection module for secondary analysis.

[0055] In the embodiment of the present application, the output accuracy of the first defect detection model is first obtained as the first model confidence. For example, the output accuracy of the first defect detection model is tested using a test set divided during the training of the first defect detection model using the void defect detection module. For example, 92% is used as the first model confidence. The first model confidence can reflect the detection reliability of the first defect detection model. A higher first model confidence indicates a more reliable model.

[0056] Next, the output accuracy of the second defect detection model is obtained as the second model confidence. For example, the output accuracy of the second defect detection model is tested using a test set divided during the training of the second defect detection model using the bridge defect detection module. For example, 93% is used as the second model confidence. The second model confidence can reflect the detection reliability of the second defect detection model. A higher second model confidence indicates a more reliable model.

[0057] Again, the first model confidence and the second model confidence are compared with a preset confidence threshold. For example, a preset confidence threshold can be set based on the production requirements for detection accuracy, for example, 90%, as the critical value for determining whether the model output is credible. The first model confidence and the second model confidence are numerically compared with the preset confidence threshold to determine whether the current detection result is credible. For example, if the preset confidence threshold is 90%, the first model confidence is 92%, and the second model confidence is 93%, both of which are greater than the preset confidence threshold.

[0058] Finally, when the first model confidence or the second model confidence falls below the preset confidence threshold, the data acquisition module is triggered to recapture image data from different angles and input it into the corresponding defect detection module for secondary analysis. For example, when the threshold comparison unit finds that the first model confidence (e.g., 85%) or the second model confidence (e.g., 88%) falls below the preset confidence threshold (e.g., 90%), the secondary analysis unit automatically triggers the data acquisition module to control the industrial camera to recapture image data of the solder joint area from different angles (e.g., by adjusting the shooting angle, focal length, or fill light intensity). This provides more comprehensive image information and reduces model misjudgments caused by the original image's single perspective and uneven lighting. The newly captured image data is then input into the corresponding defect detection module for secondary analysis. If the first model confidence is low, the data is re-input into the void defect detection module; if the second model confidence is low, the data is re-input into the bridge defect detection module. Ultimately, the high-confidence result from the secondary analysis serves as the final detection conclusion. For example, due to shadows in the original image, the confidence of the first model was only 85%. After re-collecting multi-angle images, the accuracy increased to 94%. At this time, the secondary results are used as the basis to ensure reliable output.

[0059] In summary, compared to existing technologies, this application uses a confidence analysis module to obtain the detection confidence of the cavity defect detection module and the bridge defect detection module respectively. For detection results below the confidence threshold, the image data is re-collected from different angles for secondary analysis. This effectively filters out low-confidence results, improves detection accuracy and stability, and avoids misjudgments caused by single-shot detection deviations.

[0060] The report generation module 15 is used to generate a structured detection report including defect type and defect severity level.

[0061] Based on the void defect detection module and the bridge defect detection module, accurate detection of void defects and bridge defects is completed, and the reliability of the core detection results is ensured through the confidence analysis module. On this basis, a structured detection report can be generated and output.

[0062] To address the above issues, this application generates a structured detection report including defect type and defect severity level.

[0063] Specifically, the report generation module 15 includes: a defect type determination unit, configured to determine whether the detected defect type is a void defect, a bridge defect, or a composite defect; A void severity calculation unit is used to calculate the actual void area ratio divided by the preset area threshold of the corresponding solder joint type as the void defect severity; a bridging severity calculation unit, configured to calculate a preset distance threshold divided by the actual bridging distance detected as the bridging defect severity; a severity grading unit, configured to grade the severity of the void defect and the severity of the bridging defect according to a preset severity grading table; The report output unit is used to generate a structured detection report including defect type and defect severity level.

[0064] In an embodiment of the present application, it is first determined whether the type of defect detected is a void defect, a bridging defect, or a composite defect. Exemplarily, the defect type is determined based on the output results of the void defect detection module and the bridging defect detection module. If only the void defect detection module determines that a defect exists, and the bridging defect detection module determines that there is no abnormality, it is determined to be a void defect; if only the bridging defect detection module determines that a defect exists, and the void defect detection module determines that there is no abnormality, it is determined to be a bridging defect; if both modules determine that a defect exists, it is determined to be a composite defect, that is, two defects exist at the same time in the same solder joint or area. For example, if a BGA solder joint is detected to have an excessive void area ratio and is found to be too small to be spaced from adjacent solder joints, it is determined to be a composite defect.

[0065] Next, the void defect severity is calculated by dividing the actual void area percentage by the preset area threshold for the corresponding solder joint type. Void defect severity = actual void area percentage / preset area threshold for the corresponding solder joint type. The void defect severity quantifies the severity of the void defect. The higher the actual void area percentage, the larger the void defect severity value, and the more severe the void defect. For example, if the preset area threshold for a single solder joint is 10%, and the actual void percentage is 20%, then the void severity = 20% / 10% = 2, indicating that the void defect severity at this time is twice the threshold standard.

[0066] Next, the bridging defect severity is calculated by dividing the preset distance threshold by the actual detected bridging distance. Here, bridging defect severity = preset distance threshold / actual detected bridging distance. The bridging defect severity quantifies the severity of the bridging defect. The smaller the actual bridging distance, the larger the bridging defect severity value, indicating a more severe bridging defect. For example, if the preset distance threshold for a solder joint is 30 pixels and the actual bridging distance is 15 pixels, the bridging severity = 30 / 5 = 2, indicating that the bridging defect severity is twice the threshold.

[0067] Furthermore, the severity of the void defect and the severity of the bridging defect are graded according to a preset severity grading table. For example, based on the preset severity grading table, the quantified severity values ​​are converted into intuitive grade labels, such as mild, moderate, severe, etc., to facilitate production personnel to quickly understand the impact of the defects. The severity grading table can be set according to industry standards or enterprise quality requirements. For example, a void severity of 1.0-1.5 is mild, 1.5-2.0 is moderate, and >2.0 is severe; a bridging severity of 1.0-1.5 is mild, 1.5-2.0 is moderate, and >2.0 is severe. For example, if the severity of a void defect is 2, it corresponds to the moderate grade, and if the severity of a bridging defect is 2, it corresponds to the moderate grade.

[0068] Finally, a structured inspection report is generated, including the defect type and severity level. For example, all of the above information is integrated to generate a structured inspection report, including: inspection time, solder joint number, defect type (void / bridging / compound), quantitative data for each defect (e.g., void area percentage 20%, actual bridge distance 15 pixels), severity value and corresponding level (e.g., moderate void, severe bridging). This structured report can identify high-frequency defect types in certain solder joints and guide welding parameter adjustments.

[0069] In summary, compared to existing technologies, this application uses a report generation module to generate structured inspection reports that include defect type and severity level. This transforms complex inspection data into clear, actionable quality information, achieving closed-loop management of solder joint defects from detection to decision-making.

[0070] In summary, the embodiments of the present application have at least the following technical effects: Compared to existing technologies, this application first uses a data acquisition module to collect image data of the solder joints to be tested, and then performs standard processing to obtain standard image data. In this way, the original, potentially noisy images are converted into unified, high-quality image data, laying the foundation for the defect detection accuracy of the entire system.

[0071] Secondly, this application uses a void defect detection module to identify low-density areas within the solder joints in the standard image data, calculate the area ratio, and determine if it exceeds a preset area threshold. In this way, through artificial intelligence, accurate identification and determination of void defects can be achieved.

[0072] Furthermore, this application uses a bridge defect detection module to calculate the minimum distance between solder joints in the standard image data. If it is less than a preset distance threshold, it is determined to be a bridge defect. In this way, based on the graph neural network, accurate identification and determination of bridge defects can be achieved.

[0073] Furthermore, this application uses a confidence analysis module to obtain the detection confidence levels of the cavity defect detection module and the bridge defect detection module. For detection results that fall below the confidence threshold, image data is recaptured from different angles for secondary analysis. This effectively filters out low-confidence results, improves detection accuracy and stability, and avoids misjudgments caused by single-shot detection errors.

[0074] Finally, this application uses a report generation module to generate a structured inspection report that includes defect type and severity level. This transforms complex inspection data into clear, actionable quality information, achieving a closed-loop management of solder joint defects from detection to decision-making.

[0075] Through the above technical solution, this application collects image data of the solder joints to be tested, performs standardized processing, and then combines it with an artificial intelligence model to detect void defects and bridging defects. Through confidence analysis, unreliable detection results are re-analyzed to reduce the false positive rate, and ultimately generate a structured inspection report that includes defect type and defect severity level. This improves the accuracy of solder joint void and bridging defect detection.

[0076] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0077] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0081] Although preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts become known.

[0082] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. Intelligent automatic testing system for solder joint voids and bridging defects, characterized by: The system comprises: The data acquisition module is used to collect image data of the solder joint to be tested, perform standardization processing, and obtain standard image data; A void defect detection module is used to identify low-density areas inside the solder joints in the standard image data, calculate the area ratio, and determine if it exceeds a preset area threshold as a void defect; A bridge defect detection module is used to calculate the minimum distance between solder joints in the standard image data, and if it is less than a preset distance threshold, it is determined to be a bridge defect; a confidence analysis module, configured to obtain the detection confidences of the cavity defect detection module and the bridge defect detection module respectively, and re-collect image data from different angles for secondary analysis of detection results below a confidence threshold; The report generation module is used to generate a structured detection report including defect type and defect severity level.

2. The intelligent automatic testing system for solder joint voids and bridging defects according to claim 1, characterized in that: The data acquisition module includes: An industrial imaging unit is used to capture images of the solder joint area to be tested using an industrial camera to obtain original image data; The image preprocessing unit is used to perform grayscale conversion, noise reduction and histogram equalization processing on the original image data to generate standard image data.

3. The intelligent automatic testing system for solder joint voids and bridging defects according to claim 1, characterized in that: The cavity defect detection module includes: a low-density area recognition unit, configured to input the standard image data into a first defect detection model, output a pixel-level segmentation mask image, and mark the low-density area inside the solder joint; An area calculation unit, configured to calculate the ratio of the number of pixels in the low-density area to the total number of pixels in the solder joint according to the segmentation mask image, and obtain a void area ratio; A threshold determination unit, configured to query a corresponding area threshold from a preset area threshold table according to the solder joint type as a preset area threshold, wherein the solder joint type includes a single solder joint, an array solder joint, and a BGA solder joint; The void defect determination unit is configured to compare the void area ratio with a preset area threshold corresponding to the solder joint type, and determine that the void defect occurs if the void area ratio exceeds the preset area threshold.

4. The intelligent automatic testing system for solder joint voids and bridging defects according to claim 3, characterized in that: The training step of the first defect detection model includes: A data preparation unit is used to collect a sample solder joint image data set, and perform pixel-level annotation on the void area within each sample solder joint image data, and use a red mask to mark the low-density area as a sample low-density area annotation set; A model construction unit is configured to construct a first defect detection model based on a U-Net architecture, wherein the first defect detection model includes an encoder and a decoder, wherein the encoder is configured to extract image features, and the decoder implements feature mapping through a deconvolution layer; The model training unit is used to use the sample solder joint image data set and the sample low-density area annotation set to iteratively supervise the training of the first defect detection model, and complete the training after the test accuracy converges.

5. The intelligent automatic testing system for solder joint voids and bridging defects according to claim 1, characterized in that: The bridge defect detection module includes: A contour extraction unit, configured to extract the edge contour of the solder joint in the standard image data using a Canny edge detection algorithm; An effective contour screening unit, used for screening effective solder joint contours in the solder joint edge contours; A centroid calculation unit, configured to calculate the centroid coordinates of each valid solder joint based on the valid solder joint contour; a spacing prediction unit, configured to input the centroid coordinates of the valid welds into a second defect detection model constructed based on a graph neural network, and calculate and output the spacing between adjacent weld pairs as the minimum spacing between welds; The bridging defect determination unit is configured to compare the minimum spacing between the solder joints with a preset distance threshold, and if the minimum spacing between the solder joints is less than the preset distance threshold, determine that a bridging defect exists in the corresponding solder joint pair.

6. The intelligent automatic testing system for solder joint voids and bridging defects according to claim 5, characterized in that: The training step of the second defect detection model includes: a sample data preparation unit, configured to collect a set of sample images containing bridging defects, extract edge contours of solder joints in the sample images and screen valid solder joint contours, calculate centroid coordinates based on the screened valid solder joint contours to form a set of sample solder joint centroid coordinates, and mark actual spacing values ​​of adjacent solder joint pairs in the sample images to form a set of sample solder joint spacings; A model building unit, configured to build a second defect detection model based on a graph neural network; The model training unit is used to iteratively supervise the training of the second defect detection model using the sample solder joint centroid coordinate set as input and the sample solder joint spacing set as supervision labels, and complete the training after the test accuracy converges.

7. The intelligent automatic testing system for solder joint voids and bridging defects according to claim 1, characterized in that: The confidence analysis module includes: a first confidence analysis unit, configured to obtain an output accuracy of the first defect detection model as a first model confidence; a second confidence analysis unit, configured to obtain an output accuracy of the second defect detection model as a second model confidence; a threshold comparison unit, configured to compare the first model confidence and the second model confidence with a preset confidence threshold; The secondary analysis unit is used to trigger the data acquisition module to re-collect image data from different angles when the first model confidence or the second model confidence is lower than the preset confidence threshold, and input the image data into the corresponding defect detection module for secondary analysis.

8. The intelligent automatic testing system for solder joint voids and bridging defects according to claim 1, characterized in that: The report generation module includes: a defect type determination unit, configured to determine whether the detected defect type is a void defect, a bridge defect, or a composite defect; A void severity calculation unit is used to calculate the actual void area ratio divided by the preset area threshold of the corresponding solder joint type as the void defect severity; a bridging severity calculation unit, configured to calculate a preset distance threshold divided by the actual bridging distance detected as the bridging defect severity; a severity grading unit, configured to grade the severity of the void defect and the severity of the bridging defect according to a preset severity grading table; The report output unit is used to generate a structured detection report including defect type and defect severity level.

Citation Information

Cited By

  • Feature classification-based welding spot defect infrared detection method and system

    CN121027222A