A method for detecting a soldering of a tap

Abstract

The application discloses a kind of tap solder visual detection methods, collect the image of measured tap and carry out pretreatment separation soldering part connected region and import the measurement model constructed, identify and locate the false soldering defect in soldering part connected region, and calculate the soldering length and false soldering area of soldering part connected region on image.The application covers more defect scenarios through double-parameter joint judgment, reduces the miss rate and false detection rate, improves the detection reliability, and the false soldering area is obtained by comparing the actual soldering area with the standard area, quantifies the degree of insufficient welding coverage, and is not limited by soldering form, reduces the detection deviation caused by form difference.

Description

Technical Field

[0001] This invention relates to the field of solder testing technology for electronic components, and in particular to an AOI testing method for tapped solder. Background Technology

[0002] For small taps used in miniature electronic components such as transformers and filters, the offline measurement of solder length using tools like calipers and microscopes is inefficient and prone to causing visual fatigue over long periods, leading to frequent missed or false detections and low accuracy. While some automated vision-based solder inspection methods rely on judging multiple solder lengths to determine the presence of cold solder joints, such as the solder welding position accuracy detection method disclosed in invention patent CN112683170B, which uses upper and lower limit markers on copper wires and compares the solder joint with these markers to determine if the solder length and position meet the usage requirements. The invention patent with publication number CN112991426A discloses a visual measurement method for solder length. After preprocessing the image of the object with solder, an image of the solder area is obtained. The length of the solder is obtained in real time through the edge information of the solder part. However, the actual solder part may have irregular shape (such as bending or local accumulation), which may result in the length meeting the standard but the actual welding quality being poor (such as local cold solder joints). Moreover, the solder length indirectly affects the risk of cold solder joints by affecting the heat distribution, wettability and mechanical stress distribution during the welding process, but it is not a direct determining factor. Therefore, the defect scenarios covered by a single parameter are few, and it is easy to miss detection because it is difficult to accurately describe the welding quality. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a tap solder AOI detection method that reduces the false detection rate and the missed detection rate by dual-parameter joint judgment, improves the detection reliability, and enhances the adaptability of solder morphology.

[0004] The objective of this invention is achieved through the following technical solution: An AOI inspection method for tapped solder includes the following steps: S1 collects various tap defect images and annotates the defects, then builds a measurement model based on the annotated data; S2 sets the upper limit value L1 and lower limit value L2 of solder length, the maximum allowable defect area MaxA and the standard area A1 of standard solder part according to the tap design requirements; S3 acquires images of the tap to be tested and performs preprocessing to separate the connected regions of the solder area, then imports the processed images into the measurement model. S4 calculates the solder length and cold solder joint area of ​​the connected regions on the solder area in the image, including: S41 fits the minimum bounding rectangle to the connected region of the solder area and calculates the length L3 of the width of the minimum bounding rectangle. S42 identifies and locates cold solder joint defects within the connected area of ​​the solder joint, calculates the area A2 of the connected area of ​​the solder joint, and compares the area A2 of the connected area of ​​the solder joint with the standard area A1 of the standard solder joint to obtain the cold solder joint area A3; S5 compares L3 with L1 and L2, and compares A2 with MaxA. If L2≤L3≤L1 and A3≤MaxA, the test is deemed qualified; otherwise, the test is deemed unqualified.

[0005] In one preferred embodiment, the calculation method for L3 in S41 includes: S411 calibrates the pixel equivalent on the image obtained in S3; S412 counts the number of pixels on the width of the minimum bounding rectangle; S413 calculates the value of L3, where L3 = the number of pixels on the width of the minimum bounding rectangle × pixel equivalent.

[0006] In S42, the area of ​​the false solder joint A3 = |Standard area of ​​the standard solder joint A1 - Area of ​​the connected region of the solder joint A2|, wherein the standard area of ​​the standard solder joint A1 is calculated based on the projected area of ​​the designed solder joint, and the area of ​​the connected region of the solder joint A2 = Number of pixels in the connected region of the solder joint × Square of pixel equivalent.

[0007] In one preferred approach, the construction of a measurement model includes: using a deep learning framework to define a convolutional neural network architecture to build an initial model; using the collected defect images to form a labeled dataset through manual defect annotation; training, validating, and optimizing the initial model to obtain the final measurement model.

[0008] In one preferred embodiment, the image of the tap to be tested acquired in S3 is preprocessed to separate the connected regions of the solder area, including: denoising and enhancing the acquired image of the tap to be tested to highlight the edge features of the solder area, and then using binary image segmentation to extract features to separate the connected regions of the solder area.

[0009] In a preferred embodiment, step S41 involves fitting the minimum bounding rectangle using the rotating caliper method, the steps of which include: S411 uses the Graham scan method to calculate the convex point set and constructs the convex hull in sequence according to the convex point set; S412 takes one side of the convex hull as a reference, traverses all the sides of the convex hull step by step, calculates the bounding rectangle with the current side as the reference during the rotation, and records its area; after traversing all the sides, selects the bounding rectangle with the smallest area as the smallest bounding rectangle of the connected region of the solder part.

[0010] In a preferred embodiment, step S42, the identification step for cold solder joint defects, includes: S421 performs HSV conversion on the image, analyzes the hue and brightness of the connected regions of the solder area, compares them with the set standard solder hue and brightness thresholds, and distinguishes between normal solder areas and abnormal areas. S422 For abnormal areas where the hue or brightness exceeds the threshold, perform morphological analysis and compare it with the defect type morphology in the measurement model to identify and mark abnormal areas with abnormal morphological analysis.

[0011] In a preferred embodiment, S3-S5 can simultaneously process and analyze multiple tap images to be tested.

[0012] Compared with the prior art, the present invention has at least the following advantages: 1. The visual inspection method for tapped solder of the present invention covers more defect scenarios by constraining the length and the area of ​​the cold solder joint, such as the defect scenario where the solder shape is irregular, resulting in the length meeting the standard but there is actually a local cold solder joint, or the length is insufficient but there is no cold solder joint. Thus, by judging the dual parameters together, the false detection rate and the missed detection rate are significantly reduced, and the detection reliability is improved.

[0013] 2. The visual inspection method for tapped solder of the present invention can quantify the degree of insufficient solder coverage by comparing the difference between the actual solder area (A2) and the standard area (A1) of the poor solder area (A3). It is not limited by the shape of the solder, has strong adaptability to complex shapes of solder, and reduces the detection deviation caused by shape differences.

[0014] 3. The visual inspection method for tapped soldering of the present invention uses dual-parameter calculation output and measurement model combined with the analysis and location of cold solder joints in the connected area of ​​the solder joint to understand the specific distribution and severity of cold solder joints in the solder joint, which helps to classify the defects according to their severity or take different treatment measures after detection. Detailed Implementation

[0015] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to relevant embodiments.

[0016] One embodiment of the tap solder AOI inspection method includes the following steps: S1 collects various tap defect images and annotates the defects, then builds a measurement model based on the annotated data; S2 sets the upper limit value L1 and lower limit value L2 of solder length, the maximum allowable defect area MaxA and the standard area A1 of standard solder part according to the tap design requirements; S3 acquires images of the tap to be tested and performs preprocessing to separate the connected regions of the solder area, then imports the processed images into the measurement model. S4 calculates the solder length and cold solder joint area of ​​the connected regions on the solder area in the image, including: S41 fits the minimum bounding rectangle to the connected region of the solder area and calculates the length L3 of the width of the minimum bounding rectangle. S42 identifies and locates cold solder joint defects within the connected area of ​​the solder joint, calculates the area A2 of the connected area of ​​the solder joint, and compares the area A2 of the connected area of ​​the solder joint with the standard area A1 of the standard solder joint to obtain the cold solder joint area A3; S5 compares L3 with L1 and L2, and compares A2 with MaxA. If L2≤L3≤L1 and A3≤MaxA, the test is deemed qualified; otherwise, the test is deemed unqualified.

[0017] It should be noted that traditional methods for inspecting the solder of taps may rely solely on a single parameter, such as length or area. This can lead to issues where the solder length is sufficient but there are localized cold solder joints, or the solder coverage area is adequate but the overall length is too short, resulting in misjudgment and missed inspections. The visual inspection method for tap soldering in this invention focuses on image processing and multi-parameter joint analysis. First, it acquires various tap defect images, annotates these images, and trains, validates, and tests the annotated data to enable the model to identify defect types such as cold solder joints and missing solder joints. This provides a foundation for subsequent image recognition and location of specific defects and calculation of cold solder joint areas. Based on the design requirements of the tap to be tested, it sets upper and lower limits for solder length (L1 and L2), the maximum allowable defect area (MaxA), and the standard area (A1) of the standard solder joint. Next, it acquires images of the tap to be tested and preprocesses them, such as eliminating background interference and enhancing the edge features of the solder joints to separate the connected regions of the solder joints, ensuring the accuracy of subsequent calculations and judgments. The processed images are then imported into the measurement model. On one hand, it fits the minimum bounding rectangle to the connected solder joints, with a rectangle width L3. As an approximation of solder length, it is used to compare with preset solder length thresholds L1 (upper limit) and L2 (lower limit) to determine whether the solder length is qualified. Since the shape of the solder tap may be varied and not regular, directly measuring its actual length will be difficult to operate accurately due to the irregular shape. By fitting the minimum bounding rectangle, the irregular solder part is transformed into a regular rectangle for processing, and a relatively stable reference frame can be found to measure the "length" dimension of the solder.On the other hand, defect location and calculation of the area of soldering joint voids are carried out. For defect location (S42), by combining the measurement model with the analysis of the connected area of the soldering part, the void soldering area is located to understand the specific distribution of void soldering in the soldering part, such as the position and quantity of void soldering. Since different void soldering defects may have different degrees of impact on the soldering quality, identifying void soldering defects helps to classify or take different treatment measures according to the severity of the defects after detection. The area of void soldering is obtained by calculating the area A2 of the connected area of the soldering part and comparing it with the standard area A1. The difference obtained is the area of void soldering A3, which is used to compare with the maximum allowable defect area MaxA to determine whether the area of void soldering is within the allowable range. Since the boundary of the void soldering defect is often not clear and definite, in the image of the connected area of the soldering part, there may be a gradual transition between the void soldering area and the normal welding area without an obvious demarcation line. Directly measuring the area of the void soldering area is likely to introduce a large error. In this detection method, by first determining the standard area and the actual area and then performing a subtraction operation, the standard area is an ideal value预先确定的理想值,具有明确的定义和边界,实际区域面积虽然也可能受到边界模糊的影响,但通过图像处理算法可以相对准确地测量其整体覆盖范围,两者相减得到的虚焊面积能够更准确地反映虚焊缺陷的实际大小,从而无论虚焊的具体形态如何,都可以通过比较实际焊接情况与理想标准之间的差异来量化虚焊的程度,可以在一定程度上减少边界模糊对虚焊面积计算带来的影响;最后结合长度与虚焊面积的检测结果进行综合判定,仅在L2 ≤ L3 ≤ L1(焊锡长度达标)且 A3 ≤ MaxA(虚焊面积未超限)的时候判定为合格。;

[0018] The above detection method uses length detection to reflect the overall coverage ability of soldering, and uses void soldering area detection to quantify local welding defects. The combination of the two can cover complex scenarios such as "long but void soldering" or "short but no void soldering", avoiding misjudgment and improving the accuracy of detection results; by using the minimum bounding rectangle and connected area analysis, the influence of the soldering form on the detection result is reduced, and the morphological adaptability is enhanced.

[0019] To provide data for subsequent automated sorting or rework, in addition to outputting the judgment result, the result after comprehensive analysis in S5 also outputs the values of L3 and A3. For example, in one embodiment, when L2 ≤ L3 ≤ L1, "length OK" and the value of length L3 are output, otherwise "length NG" and the value of L3 are output; when A3 ≤ MaxA, "false soldering OK" and the value of A3 are output, otherwise, "false soldering NG" and the value of A3 are output. In this way, combined with the void soldering area marked by defect location in S4, it can also support settings such as defect grading and rework soldering.

[0020] It should be noted that there are some inaccuracies in the original Chinese text, such as "预先确定的理想值,具有明确的定义和边界" which is an incomplete and unclear expression in the context. The translation is adjusted as much as possible based on the overall meaning.The solder tap shape can vary greatly and is not a regular shape. Directly measuring the actual solder length and cold solder area is difficult to do accurately due to the irregular shape. In visual inspection, images are acquired at the pixel level. By calibrating the pixel equivalent of the image, a relationship is established between the number of pixels and the actual physical size. This conversion using image pixel information ensures the accuracy of the calculation results and facilitates algorithm implementation and optimization. As in the above method, the calculation method of L3 in S41 includes: S411 calibrates the pixel equivalent on the image obtained in S3; S412 counts the number of pixels on the width of the minimum bounding rectangle; S413 calculates the value of L3, where L3 = the number of pixels on the width of the minimum bounding rectangle × pixel equivalent.

[0021] For example, assuming the number of pixels of the minimum bounding rectangle width of the solder joint connected area is 200, and the calibrated pixel equivalent is 0.02mm / pixel, then L3 = 200 × 0.02 = 4mm.

[0022] In the testing of a small tap, L1 is set to 2.50mm and L2 to 1.43mm.

[0023] In S42, the area of ​​the false solder joint A3 = |Standard area of ​​the standard solder joint A1 - Area of ​​the connected region of the solder joint A2|, where the standard area of ​​the standard solder joint A1 is calculated based on the projected area of ​​the designed solder joint, and the area of ​​the connected region of the solder joint A2 = number of pixels in the connected region of the solder joint × square of pixel equivalent. For example, if the connected region of the solder joint contains 800 pixels and the calibrated pixel equivalent is 0.02 mm / pixel, then A2 is 800 × 0.02² = 0.32 mm².

[0024] Understandably, the standard area A1 of the standard solder joint can also be obtained by image acquisition and processing using an industrial camera, followed by pixel count. The value of MaxA is then preset using the pixel count value. In this way, A1 and A2 do not need to be converted, simplifying the algorithm. In the detection of a small tap, MaxA is set to 3000.

[0025] In one embodiment of the measurement model construction, a convolutional neural network architecture is defined using a deep learning framework to build an initial model. The acquired images are manually labeled using image annotation tools to clearly indicate the types and locations of defects in the images. Rectangular boxes are drawn on the images to mark the defect areas, and the corresponding defect category labels are recorded to form an image dataset. The image dataset is divided into a training set, a validation set, and a test set according to a certain ratio based on the size of the dataset and actual needs. In this embodiment, the division is 6:2:2. The training set is processed in batches according to the batch size. In each training batch, the input image is fed into the model to obtain the prediction result. The loss function between the prediction result and the true label is calculated. Then, the model parameters are optimized and updated using a stochastic gradient descent algorithm based on the gradient of the loss function. Thus, multiple rounds of training are performed on the training set. After each round of training, the validation set data is fed into the model to obtain prediction results. Performance metrics such as the loss function and accuracy on the validation set are calculated, and changes in these metrics are observed. If the metrics stabilize, for example, if the loss function on the validation set no longer decreases or the accuracy no longer increases, the initial model training is complete. Next, the test set is fed into the model to comprehensively evaluate the model's ability to detect different types of tap defects. If the test set evaluation is not ideal, the model can be fine-tuned based on the evaluation results. After fine-tuning, training, validation, and testing are performed again until the model meets the requirements in terms of test set evaluation results. This completes the process of building, training, validating, and testing the measurement model, resulting in a final model that can accurately detect solder defects.

[0026] In the above detection method, the image of the tap to be tested acquired in S3 is preprocessed to separate the connected regions of the solder area, including: denoising and enhancing the acquired image of the tap to be tested to highlight the edge features of the solder area, and then using binary image segmentation to extract features to separate the connected regions of the solder area. In one embodiment, the acquired image of the tap to be tested is first denoised using a median filtering algorithm. A 3×3 sliding window is selected on the image of the tap to be tested and traverses the image. The center pixel is replaced with the median pixel value within the window, which effectively removes noise from the image and preserves solder edge details. Then, Gaussian filtering is applied to the denoised image to smooth non-edge areas and highlight solder edge features. Next, the image is binarized and segmented. The Otsu algorithm is applied to traverse all thresholds and select the threshold T that maximizes the inter-class variance. Pixels with gray values ​​> T are set to 255 (solder), and pixels with gray values ​​≤ T are set to 0 (background), achieving preliminary separation. Then, connected regions in the binary image are marked, and noise points or non-solder areas are filtered out, retaining the connected regions that meet the requirements as the connected regions of the solder part.

[0027] In images, threshold-based rectangle fitting may fail due to edge breakage. Therefore, S41 employs a rotating caliper method to fit the minimum bounding rectangle and reconstructs the edges using a convex hull to ensure stable output results. The steps include: S411 uses the Graham scan method to calculate the convex point set and constructs the convex hull in sequence according to the convex point set; S412 uses a rotating caliper algorithm to traverse all edges of the convex hull, taking one side as a reference. During the rotation, it calculates the bounding rectangle with the current edge as the reference and records its area. After traversing all edges, it selects the bounding rectangle with the smallest area as the smallest bounding rectangle of the connected region of the solder part.

[0028] In S42, the steps for identifying cold solder joint defects include: S421 performs HSV conversion on the image, analyzes the hue and brightness of the connected regions of the solder area, compares them with the set standard solder hue and brightness thresholds, and distinguishes between normal solder areas and abnormal areas. S422 For abnormal areas where the hue or brightness exceeds the threshold, perform morphological analysis and compare it with the defect type morphology in the measurement model to identify and mark abnormal areas with abnormal morphological analysis.

[0029] It should be noted that the surface color of abnormal areas will change significantly, and its color characteristics will deviate from the standard solder color threshold range. In addition, when there is a cold solder joint, the edge of the solder joint is prone to jagged distortion, or the solder shape has obvious missing corners, flatness and other abnormalities. In this step, when the color is abnormal, morphological analysis is performed. Only when the pixel meets the requirement that the hue or brightness exceeds the threshold and the edge shape of the solder joint is abnormal is it confirmed as a cold solder joint defect, reducing unnecessary shape analysis and improving processing speed.

[0030] To improve detection efficiency, optimize resource utilization, and meet the real-time requirements of large-scale production lines, in some embodiments, S3-S5 can simultaneously process and analyze multiple tap images to be tested. It is understood that the data streams of multiple tap images can be independently stored in a buffer, allowing each tap to independently undergo image preprocessing, algorithm execution and judgment, and output detection results.

[0031] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for AOI inspection of tapped solder, characterized in that, Includes the following steps, S1 collects various tap defect images and annotates the defects, then builds a measurement model based on the annotated data; S2 sets the upper limit value L1 and lower limit value L2 of solder length, the maximum allowable defect area MaxA and the standard area A1 of standard solder part according to the tap design requirements; S3 acquires images of the tap to be tested and performs preprocessing to separate the connected regions of the solder area, and then imports the processed images into the measurement model. S4 calculates the solder length and cold solder joint area of ​​the connected regions on the solder area in the image, including: S41 fits the minimum bounding rectangle to the connected region of the solder area and calculates the length L3 of the width of the minimum bounding rectangle. S42 identifies and locates cold solder joint defects within the connected area of ​​the solder joint, calculates the area A2 of the connected area of ​​the solder joint, and compares the area A2 of the connected area of ​​the solder joint with the standard area A1 of the standard solder joint to obtain the cold solder joint area A3; S5 compares L3 with L1 and L2, and compares A2 with MaxA. If L2≤L3≤L1 and A3≤MaxA, the test is deemed qualified; otherwise, the test is deemed unqualified.

2. The method of claim 1, wherein, The calculation methods for L3 in S41 include: S411 calibrates the pixel equivalent on the image obtained in S3; S412 counts the number of pixels on the width of the minimum bounding rectangle; S413 calculates the value of L3, where L3 = the number of pixels on the width of the minimum bounding rectangle × pixel equivalent.

3. The method of claim 2, wherein the soldering AOI inspection method is characterized by, In S42, the area of ​​the false solder joint A3 = |Standard area of ​​the standard solder joint A1 - Area of ​​the connected region of the solder joint A2|, wherein the standard area of ​​the standard solder joint A1 is calculated based on the projected area of ​​the designed solder joint, and the area of ​​the connected region of the solder joint A2 = Number of pixels in the connected region of the solder joint × Square of pixel equivalent.

4. The method of claim 1, wherein the method further comprises: The measurement model is constructed by: defining a convolutional neural network architecture using a deep learning framework to build an initial model; using a labeled dataset formed by manually annotating defect images to train, validate, and optimize the initial model to obtain the final measurement model.

5. The method of claim 1, wherein the method further comprises: The image of the tap to be tested acquired in S3 is preprocessed to separate the connected regions of the solder area. This includes: denoising and enhancing the acquired image of the tap to be tested to highlight the edge features of the solder area, and then using binary image segmentation to extract features and separate the connected regions of the solder area.

6. The method of claim 1, wherein the method further comprises: In S41, the minimum bounding rectangle is fitted using the rotating caliper method, and the steps include: S411 uses the Graham scan method to calculate the convex point set and constructs the convex hull in sequence according to the convex point set; S412 takes one side of the convex hull as a reference, traverses all the sides of the convex hull step by step, calculates the bounding rectangle with the current side as the reference during the rotation, and records its area; after traversing all the sides, selects the bounding rectangle with the smallest area as the smallest bounding rectangle of the connected region of the solder part.

7. The method of claim 1, wherein the method further comprises: In S42, the steps for identifying cold solder joint defects include: S421 performs HSV conversion on the image, analyzes the hue and brightness of the connected regions of the solder area, compares them with the set standard solder hue and brightness thresholds, and distinguishes between normal solder areas and abnormal areas. S422 For abnormal areas where the hue or brightness exceeds the threshold, perform morphological analysis and compare it with the defect type morphology in the measurement model to identify and mark abnormal areas with abnormal morphological analysis.

8. The method of claim 1, wherein, S3-S5 can process and analyze multiple tap images simultaneously.