A PCB solder joint defect detection method based on improved interpolation farthest point sampling

Abstract

The application discloses a PCB welding point defect detection method based on improved interpolation farthest point sampling, specifically comprising the following steps: a 3D camera scans a printed circuit board to obtain PCB point cloud data; the PCB point cloud is preprocessed and then divided into a training set and a test set; a PCB welding point detection model based on PointNet++ two stages is built; the intersection line L of the substrate and the welding point is determined; the nearest neighbor interpolation algorithm NN is used to obtain the PCB point cloud with dense welding points; the farthest point sampling algorithm is combined with the intersection line L to sample the PCB; the welding points are segmented by a segmentation model, the detection result is recorded, and the segmentation performance of the model is evaluated based on evaluation indexes; the farthest point sampling algorithm and the density farthest point sampling algorithm are fused to sample the welding point cloud, the sampled points are sent into a classification model to classify each welding point, and the category of each welding point is output; the detection result is recorded, and the segmentation performance of the model is evaluated based on evaluation indexes. The application improves the PCB welding point detection precision and accuracy.

Description

Technical Field

[0001] This invention belongs to the field of quality inspection in the electronic information industry, and in particular relates to a method for detecting PCB solder joint defects based on improved interpolation farthest point sampling. Background Technology

[0002] Printed circuit boards (PCBs) are an indispensable component of modern electronic devices, playing a crucial role in the development of electronic technology and the manufacturing of various electronic products. PCB manufacturing is a complex, multi-stage process involving various materials and processing methods. The main steps include solder paste printing, component mounting, reflow soldering, through-hole component mounting, and wave soldering. Defects such as insufficient solder joints, excessive solder joints, solder spikes, and bridging are prone to occur during the soldering process. The intricate processes, high density, and high integration requirements of PCB manufacturing lead to various surface defects, significantly impacting product quality. Therefore, solder joint quality inspection plays a vital role in this process.

[0003] With the increasing density of PCBs, manual inspection can no longer meet current production requirements, leading to the emergence of Automated Optical Inspection (AOI) methods. AOI performs inspection and classification by matching with standard images, which improves production efficiency, but is not suitable for inspecting PCB products with complex surface features. Currently, image-based inspection methods, such as the YOLO framework, can reliably identify objects in images, and many image-based inspection methods have achieved good results in PCB defect detection. Although 2D image inspection is widely used, for solder joint inspection, 2D inspection can only provide the projection information of the solder joint on a plane, and cannot accurately represent the three-dimensional shape of the solder joint. 3D inspection, on the other hand, can accurately obtain the three-dimensional shape and position information of the solder joint, thus having higher precision and accuracy in detecting the position and shape of the solder joint. In addition, 3D inspection is more robust to factors such as lighting and viewing angle than 2D inspection, and can maintain more stable inspection performance in complex environments. Therefore, point cloud-based PCB solder joint inspection has become a research hotspot. Summary of the Invention

[0004] To achieve higher detection accuracy, this invention provides a PCB solder joint defect detection method based on improved interpolation farthest point sampling.

[0005] The present invention provides a PCB solder joint defect detection method based on improved interpolation farthest point sampling, comprising the following steps:

[0006] Step 1: Set up a 3D camera and transmission device to scan the PCB and obtain PCB point cloud data;

[0007] Step 2: Preprocess the PCB point cloud data, and then divide the point cloud data into training and testing sets;

[0008] Step 3: Build a two-stage PCB solder joint detection model based on PointNet++;

[0009] Step 4: Determine the boundary line L between the substrate and the solder joint based on the distribution pattern of the Z coordinate values ​​of the PCB point cloud;

[0010] Step 5: Interpolate the PCB point cloud using the Natural Neighbor (NN) method, extract the interpolated solder joint point cloud through the boundary line L, and then merge it with the original PCB point cloud to obtain a PCB point cloud with dense solder joints. Based on this, the Solder Nearest Neighbor Interpolation (S-NNI) method is proposed.

[0011] Step 6: Combine the Farthest Point Sampling (FPS) algorithm with the boundary line L to sample the PCB, so as to sample more points at the junction of the substrate and the solder joint, and propose a new sampling algorithm (Z-frequency farthest point sample, Z-FPS).

[0012] Step 7: Segment the solder joints using the segmentation model, record the detection results, and evaluate the model's segmentation performance based on evaluation metrics;

[0013] Step 8: A fusion DR-FPS and FPS (DF-FPS) sampling algorithm is proposed to sample the point cloud of the solder joints. The sampled points are then fed into a classification model to classify each solder joint.

[0014] Step 9: Output the category of each solder joint, record the detection results, and evaluate the model segmentation performance based on the evaluation metrics.

[0015] Furthermore, step 1 specifically involves completing the setup and environment configuration of the Hikrobot 3DMVS and conveyor belt in the early stages, and then using the Hikrobot 3DMVS and conveyor belt together to complete the data collection.

[0016] Further, step 2 specifically involves: obtaining PCB point clouds by scanning with a 3D camera, removing noise using CloudCompare, and rotating the PCB plane to be parallel to the ground using the RANSAC algorithm; due to limited computer space, CloudCompare is also needed to cut the PCB point cloud into small pieces as a segmentation dataset, and then extracting solder joints from the segmentation dataset as a classification dataset. Both the segmentation dataset and the classification dataset are divided into training and test sets in an 8:2 ratio.

[0017] Furthermore, step 3 specifically consists of: the first stage, using the point cloud model PointNet++ to extract features from the PCB point cloud and segmenting solder joints from the PCB point cloud, which is the segmentation stage; the second stage, using PointNet++ to learn the features of the solder joints and classify them, which is the classification stage.

[0018] Furthermore, step 4 specifically involves:

[0019] (1) Histograms are introduced to analyze the distribution pattern of Z coordinates of PCB point clouds, with Z coordinates as the horizontal axis and point cloud distribution frequency as the vertical axis;

[0020] (2) Find the peak value of the histogram. The horizontal axis position that drops to the right from the peak value to 0.2 times the peak value is the boundary line L between the substrate and the solder joint.

[0021] Furthermore, step 5 specifically involves:

[0022] (1) Using natural neighbor interpolation to interpolate PCB point clouds, jagged point clouds are easily generated at locations with drastic changes, such as the junction of the substrate and solder joints.

[0023] (2) Extract the interpolated solder joints (including jagged point clouds) based on the boundary line L;

[0024] (3) Calculate the rate of change of the normal and height values ​​of each point in the weld joint; if the rate of change of the height of the point is ≤0.001mm and the minimum angle formed by the normal of the point and the z-axis is ≤10°, then the point is determined to be a jagged point cloud and is removed.

[0025] (4) Merge the solder point cloud with the serrations removed and the original PCB point cloud.

[0026] Furthermore, step 6 specifically involves:

[0027] (1) Obtain the Z coordinate values ​​of all points in the PCB point cloud, use the Z value as the horizontal axis and the vertical axis as the frequency of Z value distribution in different ranges to obtain a histogram, and divide the histogram into 50 bins.

[0028] (2) Based on the Z coordinate value of the point cloud, divide each point into different bins in the histogram; and create a dictionary where the key in the dictionary is the bin number in the histogram and the value in the dictionary is a numerical value that stores the index of the points contained in the bin.

[0029] (3) Starting from the peak of the histogram, find the Z value to the right where it drops to 0.2 of the peak, and obtain the boundary line L;

[0030] (4) Obtain the index number of the box where the boundary line L is located, and divide the points corresponding to the boxes with index numbers from index to index+2 into points at the boundary;

[0031] (5) Let the total number of sampling points be S. First, calculate the proportion of the point cloud at the boundary to the total number of point clouds on the PCB. ,by To determine the sampling ratio at the boundary, calculate the sampling points at the boundary. , Round down, the number of sampling points in other parts is ;

[0032] (6) The boundary and other parts are sampled separately using the FPS algorithm;

[0033] (7) The sampling points at the junction and other parts are merged as the returned result.

[0034] Furthermore, step 8 specifically involves:

[0035] (1) Let the total number of sampling points be S, which is the number of solder paste sampling points. , Round down to the nearest integer, number of pin sampling points ;

[0036] (2) First, use DR-FPS sampling. These sampling points are mostly located within the solder paste area because the solder paste has a high density.

[0037] (3) Assume the total number of PCB point clouds is N, and calculate the distance matrix from unsampled points to sampled points. (batchsize is the number of training batches);

[0038] (4) Use the distance matrix from step 3 Initialize the distance matrix in the FPS algorithm and sample unsampled points;

[0039] (5) Combine the sampling points from steps 2 and 4 as the returned result.

[0040] Furthermore, steps 7 and 9 are as follows:

[0041] A two-stage network was built based on PointNet++, and an improved interpolation sampling algorithm was added to or replaced in the original algorithm to train the network model. In the segmentation stage, npoint was set to 20480, batch size to 4, the optimizer Adam was selected, the initial learning rate was 0.001, and the epoch was set to 300. In the classification stage, npoint was set to 3096, batch size to 4, the optimizer Adam was selected, the initial learning rate was 0.001, and the epoch was set to 300.

[0042] Commonly used metrics in the field of visual inspection are employed to evaluate the model's performance, including overall accuracy, mean intersection-overall ratio (mIoU), instance accuracy, and class accuracy. Overall accuracy and mIoU measure the model's segmentation performance, while instance accuracy and class accuracy measure its performance in the classification stage. The formulas for each metric are as follows:

[0043] ;

[0044] Wherein, TP represents a true positive; TN represents a true negative; P represents a positive true value; and N represents a negative true value.

[0045] ;

[0046] Where k represents the number of categories to be segmented;

[0047] ;

[0048] Where M represents the number of categories.

[0049] The beneficial technical effects of this invention are as follows:

[0050] This invention analyzes the characteristics of sparse solder joints and dense substrate in PCB point clouds. It introduces a solder joint interpolation algorithm (S-NNI) by incorporating the substrate solder joint boundary line into the natural neighbor interpolation method to improve the uneven distribution of PCB point clouds, preparing for subsequent solder joint detection. In the solder joint segmentation stage, the point cloud at the PCB boundary is divided by the boundary line. A Z-FPS sampling algorithm is proposed, allocating more sampling points to the boundary line during downsampling, allowing the model to fully learn the information at the boundary and achieve accurate solder joint segmentation. In the solder joint classification stage, considering that most solder joint defects are caused by improper solder paste usage, the characteristics of the solder paste portion should be fully learned. Analysis reveals that the solder paste portion of the solder joint point cloud is dense while the pin portion is relatively sparse. Therefore, this invention proposes a DF-FPS sampling algorithm by fusing density-based farthest point sampling (DR-FPS) and farthest point sampling (FPS) in a matrix manner. This achieves focused sampling in the solder paste portion and uniform sampling in the pin portion, enabling the model to comprehensively learn the characteristics of the solder joints and reliably classify them. Attached Figure Description

[0051] Figure 1 This is a diagram of the data acquisition device of the present invention.

[0052] Figure 2 This is a diagram of the overall architecture of the model.

[0053] Figure 3 This is a histogram showing the distribution of Z-coordinate values ​​in the PCB point cloud.

[0054] Figure 4 This is a schematic diagram of a point cloud.

[0055] Figure 5 This is a schematic diagram of the S-NNI algorithm.

[0056] Figure 6 This is an image showing the interpolation effect of PCB solder joints based on S-NNI.

[0057] Figure 7 This is a schematic diagram of the Z-FPS algorithm.

[0058] Figure 8 A comparison chart of FPS and Z-FPS sampling.

[0059] Figure 9 This is a schematic diagram of the DF-FPS algorithm.

[0060] Figure 10 Comparison chart of FPS, DF-FPS and DR-FPS sampling.

[0061] Figure 11 This is a segmented data graph.

[0062] Figure 12 This is a comparison chart showing the accuracy of different segmentation models during training.

[0063] Figure 13 This is a categorized data chart.

[0064] Figure 14 This chart compares the accuracy of different classification models during training. Detailed Implementation

[0065] The present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0066] The present invention provides a PCB solder joint defect detection method based on improved interpolation farthest point sampling, comprising the following steps:

[0067] Step 1: Set up a 3D camera and a transmission device to scan the PCB and acquire point cloud data.

[0068] The initial setup and environment configuration of the Hikrobot 3DMVS and conveyor belt were completed. The Hikrobot 3DMVS and conveyor belt were then used together to collect datasets. The PCB point cloud acquisition device, such as... Figure 1 As shown.

[0069] Step 2: Preprocess the PCB point cloud, and divide the processed data into training set and test set.

[0070] PCB point clouds are obtained by scanning with a 3D camera. Noise is removed using CloudCompare. The RANSAC algorithm is used to rotate the PCB plane to be parallel to the ground. Then, CloudCompare is used to cut the PCB point cloud into small pieces as a segmentation dataset. Solder joints are extracted from the segmentation dataset as a classification dataset. The dataset is divided into training and test sets in an 8:2 ratio.

[0071] Step 3: Build a two-stage PCB solder joint detection model based on PointNet++.

[0072] PointNet++ is used to extract features from the PCB point cloud, and solder joints are segmented from the PCB point cloud as the segmentation stage. PointNet++ is then used to learn the features of the solder joints and classify them as the classification stage. The specific framework is as follows: Figure 2 As shown.

[0073] Step 4: Determine the boundary line L between the substrate and the solder joint based on the distribution pattern of the Z coordinate values ​​of the PCB point cloud.

[0074] (1) Histograms are introduced to analyze the Z-coordinate distribution pattern of PCB point clouds. The Z-coordinate value is used as the horizontal axis and the point cloud distribution frequency is used as the vertical axis, such as... Figure 3 As shown.

[0075] (2) Find the peak value of the histogram. The horizontal axis position that drops to the right from the peak value to 0.2 times the peak value is the boundary line L between the substrate and the solder joint.

[0076] Step 5: Interpolate the PCB point cloud using NN, extract the interpolated solder joint point cloud through the boundary line L, and then merge it with the original PCB point cloud to obtain a PCB point cloud with dense solder joints, which is used to prepare for subsequent steps.

[0077] To address the issue of sparse solder joint point clouds in PCB point clouds, an interpolation algorithm, S-NNI, is proposed to improve the uneven distribution of PCB point clouds and enable the model to fully learn the features of the solder joints. The details are as follows:

[0078] (1) When using natural neighbor interpolation to interpolate the PCB point cloud, jagged edges are easily generated at locations with drastic changes, such as the junction of the substrate and the solder joint. A schematic diagram of jagged edges is shown below. Figure 4 As shown.

[0079] (2) Extract the interpolated weld points (including jagged point clouds) based on the boundary line L.

[0080] (3) Calculate the rate of change of the normal and height values ​​of each point in the weld joint; if the rate of change of the height of the point is ≤0.001mm and the minimum angle between the normal of the point and the z-axis is ≤10°, then the point is determined to be a jagged point cloud and is removed.

[0081] (4) Merge the solder point cloud with the serrations removed and the original PCB point cloud.

[0082] A schematic diagram of the S-NNI algorithm is shown below. Figure 5 As shown. To verify the effectiveness of the S-NNI algorithm, three point cloud files were randomly selected from the PCB point cloud data, namely PCB_1, PCB_2, and PCB_3. The S-NNI algorithm was used to interpolate the solder joint point clouds, and the interpolation results are shown in the figure. Figure 6 As shown, interpolation of solder joint point clouds is achieved, resulting in a more balanced PCB distribution.

[0083] Step 6: Combine the Farthest Point Sample (FPS) algorithm with the boundary line L to sample the PCB, thereby sampling more points at the junction of the substrate and the solder joint.

[0084] Based on the distribution pattern of Z-coordinate values ​​in the PCB point cloud, the boundary line between the substrate and solder joints can be found, thereby determining the boundary of the point cloud. This allows for separate sampling at the boundary of the PCB point cloud, allocating more sampling points to this boundary to propose the (Z-frequency Farthest Point Sample, Z-FPS) algorithm. This enables the model to learn the information at the boundary, achieving accurate segmentation. Specifically:

[0085] (1) Obtain the Z coordinate values ​​of all points in the PCB point cloud. Use the Z value as the horizontal axis and the vertical axis as the frequency of Z value distribution in different ranges to obtain a histogram. Divide the histogram into 50 bins.

[0086] (2) Based on the Z coordinate values ​​of all points, divide all points into different bins of the histogram; create a dictionary where the key is the bin number in the histogram and the value is the index of the point contained in that bin.

[0087] (3) Starting from the peak of the histogram, find the Z value that drops to the peak value of 0.2 to the right to obtain the boundary line L.

[0088] (4) Obtain the index number of the box where the boundary line L is located, and divide the points corresponding to the boxes with index number in index~index+2 into points at the boundary.

[0089] (5) Let the total number of sampling points be S. First, calculate the proportion of the point cloud at the boundary to the total number of point clouds on the PCB. The sampling ratio at the boundary was obtained as follows: Calculate the sampling points at the boundary. , Round down, the number of sampling points in other parts is .

[0090] (6) The boundary and other parts are sampled separately using the FPS algorithm.

[0091] (7) The sampling points at the junction and other parts are merged as the returned result.

[0092] A schematic diagram of the Z-FPS algorithm is shown below. Figure 7 As shown, to verify the effectiveness of the Z-FPS algorithm, four point cloud files were randomly selected: PCB_01, PCB_02, PCB_03, and PCB_04. 500 points were sampled using both FPS and Z-FPS methods. The front and top views of the four different PCB point cloud sampling points are shown below. Figure 8 As shown in the image, it can be seen that the Z-FPS algorithm proposed in this invention achieves dense sampling at the junction of the substrate and the solder joint.

[0093] Step 7: Segment the solder joints using the segmentation model, record the detection results, and evaluate the model's segmentation performance based on evaluation metrics.

[0094] Set the relevant hyperparameters for training the segmentation model (as shown in Table 1), import the data to start the segmentation model training process, and obtain the optimal result parameters.

[0095] Table 1 Segmentation Parameter Settings

[0096]

[0097] Step 8: Combine the farthest point sampling algorithm (FPS) and the density-related farthest point sampling algorithm (DR-FPS) to sample the point cloud of the weld joint, and then feed the sampled points into the classification model to classify each weld joint.

[0098] Considering that most solder joint defects are caused by improper solder paste application, a fusion DR-FPS and FPS sampling algorithm is proposed, which integrates DR-FPS and FPS. The DF-FPS algorithm is used for solder joint sampling, achieving focused sampling on the solder paste area and uniform sampling on the pin area. Specifically:

[0099] (1) Let the total number of sampling points be . Solder paste sampling points , Round down to the nearest integer, number of pin sampling points .

[0100] (2) First, use DR-FPS sampling. These sampling points are mostly located within the solder paste area because the solder paste has a high density.

[0101] (3) Let the total number of PCB point clouds be N, and calculate the distance matrix from unsampled points to sampled points. batchsize is the training batch size.

[0102] (4) Use the distance matrix from step 3 Initialize the distance matrix of FPS and sample the unsampled points.

[0103] (5) Combine the sampling points from steps 2 and 4 as the returned result.

[0104] A schematic diagram of the DF-FPS algorithm is shown below. Figure 9 As shown. To verify the effectiveness of the DF-FPS algorithm, four solder joint point clouds were randomly selected from the solder joint classification data: Solder joint_01, Solder joint_02, Solder joint_03, and Solder joint_04. The number of sampling points was set to 300, and three different sampling algorithms—FPS, DF-FPS, and DR-FPS—were used to sample different solder joints. The sampling results are shown in the figure. Figure 10 As shown, DF_FPS can perform dense sampling in the solder paste area and uniform sampling in the pin area.

[0105] Step 9: Output the category of each solder joint, record the detection results, and evaluate the model segmentation performance based on the evaluation metrics.

[0106] Set the relevant hyperparameters for training the classification model (as shown in Table 2), import the data to start the classification model training process, and obtain the optimal result parameters.

[0107] Table 2 Parameter settings for training classification models

[0108]

[0109] A two-stage network was built based on PointNet++, and an improved interpolation sampling algorithm was added to or replaced in the original algorithm to train the network model. In the segmentation stage, npoint was set to 20480, batch size to 4, the optimizer Adam was selected, the initial learning rate was 0.001, and the epoch was set to 300. In the classification stage, npoint was set to 3096, batch size to 4, the optimizer Adam was selected, the initial learning rate was 0.001, and the epoch was set to 300.

[0110] Commonly used metrics in the field of visual inspection are employed to evaluate the model's performance, including overall accuracy, mean intersection-overall ratio (mIoU), instance accuracy, and class accuracy. Overall accuracy and mIoU measure the model's segmentation performance, while instance accuracy and class accuracy measure its performance in the classification stage. The formulas for each metric are as follows:

[0111] ;

[0112] Wherein, TP represents a true positive; TN represents a true negative; P represents a positive true value; and N represents a negative true value.

[0113] ;

[0114] Where k represents the number of categories to be segmented.

[0115] ;

[0116] Where M represents the number of categories.

[0117] Tested on a PCB point cloud segmentation dataset (see schematic diagram of the segmentation dataset). Figure 11The experimental equipment and environment (shown) consisted of an Intel(R) Core(TM) i9-7920X CPU @ 2.90GHz, 16GB memory, and an NVIDIA TITAN V. The deep learning environment was PyTorch 1.2.0 + cu10.0 with Python 3.7.12. Experiments showed that compared to existing methods, this method achieves higher accuracy in PCB solder joint segmentation and converges fastest during model training. The relevant training process is as follows: Figure 12 As shown.

[0118] Additionally, testing was conducted on a classification dataset (see diagram of the classification dataset). Figure 13 The experimental equipment and environment (as shown) consisted of an Intel(R) Core(TM) i9-7920X CPU @ 2.90GHz, 16GB memory, and an NVIDIA TITAN V. The deep learning environment was PyTorch 1.2.0 + cu10.0 with Python 3.7.12. Experiments revealed that compared to existing methods, this method achieves higher accuracy in classifying solder joints, and the model converges fastest during training. The relevant training process is as follows: Figure 14 As shown.

[0119] To verify that the proposed algorithms S-NNI and Z-FPS can improve the accuracy of solder joint segmentation, four ablation experiments were conducted on each model, using PointNet++ and PointConv as benchmark models. The results are shown in Table 3.

[0120] Table 3 Comparison of Solder Joint Segmentation Results

[0121]

[0122] To verify the effectiveness of the DF-FPS algorithm proposed in the solder joint classification stage, four ablation experiments were conducted on each of the PointNet++ and PointConv benchmark models, and the results are shown in Table 4.

[0123] Table 4 Comparison of Solder Joint Classification Results

[0124]

[0125] To address the issue of low accuracy in 3D inspection of PCB solder joints, this invention proposes S-NNI, Z-FPS, and DF-FPS algorithms to improve the model's accuracy in detecting PCB solder joints. Due to the sparse solder joints and dense substrate in the PCB point cloud, the interpolation algorithm S-NNI is proposed to improve the uneven distribution of the PCB point cloud. In addition, two algorithms more suitable for PCB sampling, Z-FPS and DF-FPS, are implemented and applied to PCB solder joint segmentation and classification tasks, respectively. In PCB solder joint segmentation, the Z-FPS sampling algorithm retains more points at the boundaries during downsampling, improving the accuracy of model segmentation, ultimately achieving an accuracy of 97.20%. In solder joint classification, to focus sampling on the solder paste area while uniformly sampling the pins outside the solder paste, a distance matrix is ​​used to fuse DR-FPS and FPS, proposing the DF-FPS sampling strategy. This achieves highly reliable solder joint classification with an accuracy of 97.78%.

Claims

1. A method for detecting PCB solder joint defects based on improved interpolation farthest point sampling, characterized in that, Includes the following steps: Step 1: Set up a 3D camera and transmission device to scan the PCB and obtain PCB point cloud data; Step 2: Preprocess the PCB point cloud data, and then divide the point cloud data into training and testing sets; Step 3: Build a two-stage PCB solder joint detection model based on PointNet++; In the first stage, the PointNet++ point cloud model is used to extract the features of the PCB point cloud and segment the solder joints from the PCB point cloud as the segmentation stage. The second stage involves using PointNet++ to learn the features of the solder joints and classify them as the classification stage. Step 4: Determine the boundary line L between the substrate and the solder joint based on the distribution pattern of the Z coordinate values ​​of the PCB point cloud; Step 5: Interpolate the PCB point cloud using the natural neighbor interpolation method, extract the interpolated solder joint point cloud through the boundary line L, and then merge it with the original PCB point cloud to obtain the PCB point cloud S-NNI with dense solder joints. Step 6: Combine the farthest point sampling algorithm with the boundary line L to sample the PCB, so as to achieve sampling at more points at the junction of the substrate and the solder joint, and propose a new sampling algorithm Z-FPS; (1) Obtain the Z coordinate values ​​of all points in the PCB point cloud, use the Z value as the horizontal axis and the vertical axis as the frequency of Z value distribution in different ranges to obtain a histogram, and divide the histogram into 50 bins. (2) Based on the Z coordinate value of the point cloud, divide each point into different bins in the histogram; and create a dictionary where the key in the dictionary is the bin number in the histogram and the value in the dictionary is a numerical value that stores the index of the points contained in the bin. (3) Starting from the peak of the histogram, find the Z value to the right where it drops to 0.2 of the peak, and obtain the boundary line L; (4) Obtain the index number of the box where the boundary line L is located, and divide the points corresponding to the boxes with index numbers from index to index+2 into points at the boundary; (5) Let the total number of sampling points be S. First, calculate the proportion of the point cloud at the boundary to the total number of point clouds on the PCB. ,by To determine the sampling ratio at the boundary, calculate the sampling points at the boundary. , Round down, the number of sampling points in other parts is ; (6) The boundary and other parts are sampled separately using the FPS algorithm; (7) The sampling points at the boundary and other parts are merged as the returned result; Step 7: Segment the solder joints using the segmentation model, record the detection results, and evaluate the model's segmentation performance based on evaluation metrics; Step 8: Sample the point cloud of the weld joints and input the sampled points into the classification model to classify each weld joint; Step 9: Output the category of each solder joint, record the detection results, and evaluate the model segmentation performance based on the evaluation metrics.

2. The PCB solder joint defect detection method based on improved interpolation farthest point sampling according to claim 1, characterized in that, Step 1 specifically involves first setting up and configuring the environment for the Hikrobot 3DMVS and conveyor belt, then using the Hikrobot 3DMVS and conveyor belt together to collect the dataset.

3. The PCB solder joint defect detection method based on improved interpolation farthest point sampling according to claim 1, characterized in that, Step 2 specifically involves: obtaining PCB point clouds by scanning with a 3D camera, removing noise using CloudCompare, rotating the PCB plane to be parallel to the ground using the RANSAC algorithm, then using CloudCompare to cut the PCB point cloud into small pieces as a segmentation dataset, and then extracting solder joints from the segmentation dataset as a classification dataset. Both the segmentation dataset and the classification dataset are divided into training and test sets in an 8:2 ratio.

4. The PCB solder joint defect detection method based on improved interpolation farthest point sampling according to claim 1, characterized in that, Step 4 specifically involves: (1) Histograms are introduced to analyze the distribution pattern of Z coordinates of PCB point clouds, with Z coordinates as the horizontal axis and point cloud distribution frequency as the vertical axis; (2) Find the peak value of the histogram. The horizontal axis position that drops to the right from the peak value to 0.2 times the peak value is the boundary line L between the substrate and the solder joint.

5. The PCB solder joint defect detection method based on improved interpolation farthest point sampling according to claim 1, characterized in that, Step 5 specifically involves: (1) Using NN to interpolate the PCB point cloud, jagged point clouds are easily generated at locations with drastic changes, such as the junction of the substrate and the solder joint. (2) Extract the interpolated weld points based on the boundary line L, including the jagged point cloud; (3) Calculate the rate of change of the normal and height values ​​of each point in the weld joint; if the rate of change of the height of the point is ≤0.001mm and the minimum angle formed by the normal of the point and the z-axis is ≤10°, then the point is determined to be a jagged point cloud and is removed. (4) Merge the solder point cloud with the serrations removed and the original PCB point cloud.

6. The PCB solder joint defect detection method based on improved interpolation farthest point sampling according to claim 1, characterized in that, Step 8 specifically involves: (1) Let the total number of sampling points be S, which is the number of solder paste sampling points. , Round down to the nearest integer, number of pin sampling points ; (2) First, use DR-FPS sampling. These sampling points are mostly located within the solder paste area because the solder paste has a high density. (3) Assume the total number of PCB point clouds is N, and calculate the distance matrix from unsampled points to sampled points. batchsize is the training batch size; (4) Use the distance matrix from step 3 Initialize the distance matrix in the FPS algorithm and sample unsampled points; (5) Combine the sampling points from steps 2 and 4 as the returned result.

7. The PCB solder joint defect detection method based on improved interpolation farthest point sampling according to claim 1, characterized in that, Steps 7 and 9 are specifically as follows: A two-stage network was built based on PointNet++, and an improved interpolation sampling algorithm was added to or replaced in the original algorithm to train the network model. In the segmentation stage, npoint was set to 20480, batch size to 4, the optimizer Adam was selected, the initial learning rate was 0.001, and the epoch was set to 300. In the classification stage, npoint was set to 3096, batch size to 4, the optimizer Adam was selected, the initial learning rate was 0.001, and the epoch was set to 300. Commonly used metrics in the field of visual inspection are employed to evaluate model performance, including Overall Accuracy, Mean Intersection over Union (mIoU), Instance Accuracy, and Class Accuracy. Overall Accuracy and mIoU measure the model's segmentation performance, while Instance Accuracy and Class Accuracy measure its performance in the classification stage. The formulas for each metric are as follows: ； Wherein, TP represents a true positive; TN represents a true negative; P represents a positive true value; and N represents a negative true value. ； Where k represents the number of categories to be segmented; ； Where M represents the number of categories.

Images