PCB welding spot cloud registration method based on binocular 3D vision multi-scale feature fusion

Abstract

The invention discloses a PCB welding spot point cloud registration method based on binocular 3D vision multi-scale feature fusion. The method specifically comprises the steps that a binocular 3D vision collection platform is built to scan a PCB to obtain high-precision point cloud data; the PCB point cloud data is preprocessed, and the point cloud quality is improved; partitioning the point cloud data by using Cloud complex software, and constructing a binocular welding spot point cloud registration data set; designing and building an iteration nearest point PCB welding spot point cloud registration model based on multi-scale feature extraction; sampling a PointNet + + model to carry out multi-scale feature extraction; determining a boundary line of the substrate and the welding spot, performing efficient sampling on the welding spot and the substrate by adopting a w-FPS method, and optimizing a feature extraction result; adopting a dynamic threshold screening strategy based on a KD-tree to quickly reject point pairs with low signal-to-noise ratios, and performing ICP (Inductively Coupled Plasma) registration; registration is completed, a registration result is recorded, and the registration performance of the model is evaluated based on an evaluation index; and the Open3d carries out visualization on the registration result. According to the method, the registration precision and efficiency of the point cloud of the PCB welding spots are remarkably improved, and the method has high practical application value.

Description

Technical Field

[0001] The present invention belongs to the field of quality inspection of electronic information industry, and in particular relates to a PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion. Background Art

[0002] Wave soldering technology is the mainstream soldering method in the current printed circuit board assembly (PCBA) manufacturing process, and its soldering quality directly affects the reliability and performance of electronic products. Therefore, it is crucial to detect solder joint appearance defects in a timely and efficient manner. At present, PCB solder joint detection methods mainly include manual visual inspection, automatic optical inspection (AOI), automatic X-ray inspection (AXI) and detection technology based on 3D point cloud data. Since the detection technology based on 3D point cloud data can provide 3D information such as the height, shape, and volume of the solder joint, it has gradually been widely used in high-precision and high-reliability PCB solder joint defect detection.

[0003] Detection technology based on 3D point cloud data can achieve high-precision object detection and defect identification by collecting spatial coordinate information of the target object and combining computer vision and machine learning algorithms. However, in practical applications, due to the complex and changeable characteristics of the solder joint surface, point cloud data often has voids, which will lead to the loss of some solder joint features, thus affecting the detection accuracy. The void phenomenon may lead to missed detection or misjudgment of solder joint defects, seriously affecting the accuracy of the detection results and product quality. Especially in automated quality inspection and structural integrity analysis, the missing point cloud data may greatly reduce the reliability of the detection system.

[0004] In order to solve this problem, point cloud registration technology has received extensive attention in recent years. By accurately aligning the source point cloud and the target point cloud, the registration method can ensure the integrity and accuracy of the solder joint data. Common point cloud registration methods include registration methods based on the iterative closest point (ICP) algorithm, registration algorithms based on feature matching, and registration methods based on global optimization. These methods perform well on certain objects in nature with relatively smooth surfaces, clear features, and dense point cloud data, and can complete the registration task well. However, in the case of PCB solder joints with complex structures, irregular surfaces, and missing point cloud data, these traditional registration methods often perform poorly. The point cloud of the welding area is usually sparse and incomplete, which makes it difficult for existing registration technologies to achieve ideal results under these complex conditions, which in turn affects the reliability and accuracy of subsequent solder joint defect detection and quality analysis. Summary of the invention

[0005] In order to achieve fast and accurate registration of solder joint point clouds, the present invention provides a PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion.

[0006] A PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion of the present invention comprises the following steps:

[0007] Step 1: Build a binocular 3D vision acquisition platform and scan the printed circuit board (PCB) to obtain high-precision point cloud data.

[0008] Step 2: Segment, filter, and level the PCB point cloud data to improve the point cloud quality.

[0009] Step 3: Use Cloudcompare software to divide the point cloud data into blocks and construct a binocular weld point cloud registration dataset.

[0010] Step 4: Design and build an iterative closest point (ICP) PCB solder point cloud registration model based on multi-scale feature extraction.

[0011] Step 5: Use the PointNet++ model to extract local and global multi-scale features of the weld point cloud.

[0012] Step 6: Determine the boundary line L between the substrate and the solder joint according to the distribution law of the height information (i.e., z value) of the PCB point cloud.

[0013] Step 7: Use the improved weighted farthest point sampling method w-FPS to efficiently sample the solder joints and substrate to optimize the feature extraction results.

[0014] Step 8: Use a dynamic threshold screening strategy based on KD-tree to quickly remove low signal-to-noise ratio point pairs and perform ICP registration.

[0015] Step 9: Complete the registration, record the registration results and evaluate the model registration performance based on the evaluation indicators.

[0016] Step 10: Use Open3d to visualize the registration results.

[0017] Specifically, the draw_geometries() method of Open3D is used to visualize the registration results, and different colors are used to distinguish the source point cloud and the target point cloud.

[0018] Furthermore, step 1 is to first complete the opposite installation of two Hikvision robot 3DMVS, the construction of the conveyor belt and the environmental configuration, and then use the Hikvision robot 3DMVS and the conveyor belt together, adjust the acquisition parameters, and complete the high-precision acquisition of the welding point cloud data.

[0019] Furthermore, step 2 is specifically as follows: first, the point cloud data obtained by 3DMVS scanning includes PCB point cloud and conveyor belt point cloud; the conveyor belt point cloud is removed by semantic segmentation technology, and the PCB point cloud is retained; then, the filtering algorithm is used to remove noise points in the point cloud to improve data quality; finally, the RANSAC algorithm is used to perform plane fitting on the PCB point cloud, and the PCB plane is rotated to a posture parallel to the ground.

[0020] Furthermore, step 3 is specifically as follows: using CloudCompare software to cut the processed PCB point cloud into multiple small pieces to form a registration data set; the data set includes a source point cloud set and a target point cloud set to provide data support for the subsequent registration process.

[0021] Furthermore, step 4 is specifically as follows: first, use a deep learning algorithm to extract multi-scale features of the source point cloud and the target point cloud; then, use them as input for the ICP registration algorithm; and finally, visualize the registration results.

[0022] Furthermore, step 5 is specifically as follows: using the multi-scale neighborhood aggregation technology of PointNet++ to perform multi-scale extraction of local and global features of point cloud data.

[0023] Furthermore, step 6 is specifically as follows:

[0024] (1) Analyze the height distribution law of the PCB point cloud through a histogram, and draw a height distribution histogram of the point cloud with the height value (z value) as the horizontal axis and the number of point clouds as the vertical axis.

[0025] (2) Determine the peak value and its RMS error in the histogram, and then shift it outward along the right side of the peak value by 0.1 times the RMS error. The corresponding horizontal coordinate is the boundary line L between the substrate and the solder joint.

[0026] Furthermore, step 7 is specifically as follows:

[0027] (1) According to the dividing line L, the PCB point cloud is divided into two parts: substrate and solder joints.

[0028] (2) Set the total number of sampling points N and the sampling weight parameter w.

[0029] (3) The number of sampling points for the solder joint part is calculated as N×w, and the number of sampling points for the substrate part is calculated as: N×(1-w).

[0030] (4) Sampling of substrates and solder joints using the improved w-FPS.

[0031] (5) Return the sampling results.

[0032] Furthermore, step 8 is specifically as follows:

[0033] (1) Construct a KD-Tree data structure for the input source point cloud and target point cloud.

[0034] (2) According to the KD-Tree data structure, search for corresponding points and record corresponding point pairs (p i ,q j ) and calculate its Euclidean distance d ij .

[0035] (3) Find the maximum value d max and the minimum value d min ; According to the distribution of distance, it is divided into 50 groups, and the group spacing m=(d max -d min ) / 50.

[0036] (4) Set the dynamic threshold to Z = d max -(2+0.5i)×m, i is the number of iterations.

[0037] (5) The screening distance is [d min ,Z], these point pairs are marked as low signal-to-noise ratio point pairs and excluded from the registration calculation.

[0038] (6) Set the minimum number of point clouds to be screened N 0 , if [d min ,Z] the corresponding points outside the range are less than N 0 , reset the filter condition to Z = d max -2×m.

[0039] (7) Calculate the root mean square error f and determine whether it is less than the threshold.

[0040] (8) Repeat steps (2) to (6) until convergence or the set number of iterations is reached and the results are output.

[0041] Furthermore, step 9 is specifically as follows:

[0042] The iterative closest point (ICP) PCB solder point cloud registration model based on multi-scale feature extraction introduces PointNet++ for multi-scale feature extraction, and the improved weighted farthest point sampling algorithm is used to optimize the feature extraction results. During registration, the dynamic threshold screening condition is set to filter out the low signal-to-noise ratio point pairs; in the feature extraction stage, N is set to 8096, batch_size is 1, the optimizer Adam is selected, and the initial learning rate is 0.001; in the registration stage, the maximum number of iterations epoch is set to 100, and the threshold is set to 1×10 -5 , the minimum number of screened point clouds N 0 It is 7072.

[0043] The performance of the model is evaluated by using common indicators in the field of point cloud registration, including root mean square error (RSME), rotation errors (Re), translation errors (Te) and registration time (Time) to evaluate the performance and registration quality of the algorithm. The formulas of various indicators are:

[0044]

[0045] Among them, p i and q j represents the corresponding point pair, R rest To test the rotation matrix, t rest To test the translation matrix, N is the total number of points.

[0046]

[0047] Among them, R true is the true rotation matrix, and tr(R) represents the trace of the matrix R.

[0048] Te=||t true -t rest || 2

[0049] Among them, t true is the real translation matrix.

[0050] Registration time: The total time from the start of the algorithm to the completion of point cloud registration.

[0051] The beneficial technical effects of the present invention are:

[0052] This paper combines the advantages of the iterative closest point (ICP) algorithm and deep learning methods, aiming to improve the accuracy and computational efficiency of point cloud registration. First, this paper uses the multi-scale feature extraction technology of PointNet++ to accurately capture the local and global features of the solder point cloud at different scales, and adopts an improved weighted farthest point sampling (FPS) strategy to optimize the segmentation and key point extraction of the solder point cloud using height information, significantly improving the accuracy of feature extraction. Then, combined with the dynamic guidance strategy, during the registration process, the low signal-to-noise ratio point pairs are intelligently screened to reduce the impact of mismatched point pairs, thereby improving the registration accuracy and effectively compressing the number of iterations. Finally, a series of comparative experiments were conducted on a self-made binocular solder point cloud dataset to verify the significant advantages of this method in accuracy and efficiency, indicating that this method can not only effectively solve the local optimal solution and low signal-to-noise ratio problems faced by the traditional ICP algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The binocular 3D vision acquisition platform built by the present invention.

[0054] Figure 2 An example of a dataset for solder joint defect registration.

[0055] Figure 3 This is the overall architecture diagram of the model.

[0056] Figure 4 Multi-scale feature extraction structure for PointNet++

[0057] Figure 5 It is the height distribution histogram of PCB point cloud.

[0058] Figure 6 This is the PCB point cloud segmentation effect.

[0059] Figure 7 It is an improved weighted farthest point sampling method (w-FPS).

[0060] Figure 8 Schematic diagram of sampling with different weights w.

[0061] Fig. 9 Schematic diagram of the dynamic threshold screening strategy based on KD-tree.

[0062] Fig.10 The result of registration using Open3D. DETAILED DESCRIPTION

[0063] The present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0064] A PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion of the present invention comprises the following steps:

[0065] Step 1: Build a binocular 3D vision acquisition platform and scan the printed circuit board (PCB) to obtain high-precision point cloud data.

[0066] In the early stage, the two Hikvision robot 3DMVS were installed opposite each other, the conveyor belt was set up, and the environment was configured. Then the Hikvision robot 3DMVS and the conveyor belt were used together, and the acquisition parameters were adjusted to complete the high-precision acquisition of the welding point cloud data. The binocular 3D vision acquisition platform is as follows: Figure 1 shown.

[0067] Step 2: Perform pre-processing such as segmentation, filtering, and leveling on the PCB point cloud data to improve the point cloud quality.

[0068] The point cloud data obtained by 3DMVS scanning includes PCB point cloud and conveyor belt point cloud. The conveyor belt point cloud is removed by semantic segmentation technology, and the PCB point cloud is retained; then, the filtering algorithm is used to remove the noise points in the point cloud to improve the data quality; finally, the RANSAC algorithm is used to perform plane fitting on the PCB point cloud and rotate the PCB plane to a posture parallel to the ground. Through the above preprocessing methods, the data quality is further improved.

[0069] Step 3: Use Cloudcompare software to divide the point cloud data into blocks and construct a binocular weld point cloud registration dataset.

[0070] Use CloudCompare software to cut the processed PCB point cloud into multiple small pieces to form a registration data set, such as Figure 2 The dataset consists of two parts: source point cloud set and target point cloud set, which provide data support for the subsequent registration process.

[0071] Step 4: Design and build an iterative closest point (ICP) PCB solder point cloud registration model based on multi-scale feature extraction.

[0072] Use deep learning algorithms to extract multi-scale features of the source point cloud and the target point cloud; then use them as input for the ICP registration algorithm; finally, visualize the registration results. The specific framework is as follows Figure 3 shown.

[0073] Step 5: Use the PointNet++ model to extract local and global multi-scale features of the weld point cloud.

[0074] PointNet++ is used to combine multi-scale neighborhood aggregation technology to extract local and global features of point cloud data at multiple scales, such as Figure 4 shown.

[0075] Step 6: Determine the boundary line L between the substrate and the solder joint according to the distribution law of the height information (i.e., z value) of the PCB point cloud.

[0076] (1) Analyze the height distribution law of the PCB point cloud through the histogram, use the height value (i.e., z value) as the horizontal coordinate and the number of point clouds as the vertical coordinate to draw the height distribution histogram of the point cloud, such as Figure 5 shown.

[0077] (2) Determine the peak value in the histogram and its RMS error, and then calculate the value along the right edge of the peak value Z 2 The horizontal coordinate corresponding to the outward offset of 0.1 times the root mean square error resm is the dividing line L between the substrate and the solder joint. The segmentation effect is as follows: Figure 6 shown.

[0078] In order to determine the value of the dividing line L, this embodiment designs a set of experiments, where L is respectively 2 +0resm, Z 2 +0.1resm, Z 2 +0.2resm, Z 2 +0.3resm and Z 2 +0.4resm, divide the PCBA point cloud into substrates or solder joints along the z coordinate value according to the L value, and calculate the accuracy of its segmentation P recision , the formula is as follows:

[0079]

[0080] Among them, J P Indicates the number of actual solder joints in the solder joint segment, B P represents the number of actual substrates in the segmented substrate part, J T Indicates the number of welds divided into, B T Indicates the number of welds that are divided.

[0081] The segmentation results under different L values ​​are shown in Table 1. The experimental results show that when L = Z 2 When +0.1×resm, the segmentation effect is the best.

[0082] Table 1 Segmentation accuracy under different L values

[0083]

[0084] Step 7: An improved weighted farthest point sampling method (w-FPS) is used to efficiently sample solder joints and substrates to optimize feature extraction results.

[0085] Considering the problem of unbalanced solder joint point cloud in PCB point cloud, an improved weighted farthest point sampling method (w-FPS) for PCB solder joint point cloud is proposed. By setting the weight w, the extraction of some feature points of solder joints is improved and the feature extraction results are optimized. The details are as follows:

[0086] (1) According to the dividing line L, the PCB point cloud is divided into two parts: substrate and solder joints.

[0087] (2) Set the total number of sampling points N and the sampling weight parameter w.

[0088] (3) The number of sampling points for the solder joint part is calculated as N×w, and the number of sampling points for the substrate part is calculated as: N×(1-w).

[0089] (4) Sampling of substrates and solder joints using the improved w-FPS.

[0090] (5) Return the sampling results.

[0091] The schematic diagram of w-FPS algorithm is as follows Figure 7 To prove the effectiveness of the w-FPS algorithm, 10 PCB point clouds were randomly selected and 1024 points were sampled using w-FPS with weights w of 0.5, 0.6, 0.7, and 0.8. Figure 8 As shown in Figure 2, the w-FPS algorithm implements weighted sampling of solder joints and substrates, improving the integrity of some features of solder joints.

[0092] Step 8: Use a dynamic threshold screening strategy based on KD-tree to quickly remove low signal-to-noise ratio point pairs and perform ICP registration.

[0093] In the ICP (Iterative Closest Point) registration process, due to the existence of low signal-to-noise ratio point pairs, the PCB solder joint point cloud registration accuracy may decrease and the number of iterations may increase. Therefore, the dynamic threshold screening strategy of KD-tree can be combined to effectively remove low signal-to-noise ratio point pairs, thereby reducing the impact of noise on the registration results, improving the registration accuracy and reducing the number of iterations, as follows:

[0094] (1) Construct a KD-Tree data structure for the input source point cloud and target point cloud.

[0095] (2) According to the KD-Tree data structure, search for corresponding points and record corresponding point pairs (p i ,q j ) and calculate its Euclidean distance d ij .

[0096] (3) Find the maximum value d max and the minimum value d min ; According to the distribution of distance, it is divided into 50 groups, and the group spacing m=(d max -d min ) / 50.

[0097] (4) Set the dynamic threshold to Z = d max -(2+0.5i)×m, i is the number of iterations.

[0098] (5) The screening distance is [d min ,Z], these point pairs are marked as low signal-to-noise ratio point pairs and excluded from the registration calculation.

[0099] (6) Set the minimum number of point clouds to be screened N 0 , if [d min ,Z] the corresponding points outside the range are less than N 0 , reset the filter condition to Z = d max -2×m.

[0100] (7) Calculate the root mean square error f and determine whether it is less than the threshold.

[0101] (8) Repeat steps (2) to (6) until convergence or the set number of iterations is reached and the results are output.

[0102] The schematic diagram of the dynamic threshold screening strategy based on KD-tree is as follows Fig. 9 As shown, in order to determine the size of the initial screening value, the initial value is d max -1m,d max -1.5m,d max -2m,d max -2.5m and d max -3m, conduct multiple experiments, when the initial value is Z=d max When the filter thickness is -2m, the screening effect is the best, and the results are shown in Table 2.

[0103] Table 2 Different threshold screening conditions

[0104]

[0105] Step 9: Complete the registration, record the registration results and evaluate the model registration performance based on the evaluation indicators.

[0106] Set the relevant hyperparameters of the registration model (as shown in Table 3), import the data to start the binocular weld point cloud registration and obtain the optimal result parameters.

[0107] Table 3 Registration parameter settings

[0108]

[0109] The performance of the model is evaluated by using common indicators in the field of point cloud registration, including root mean square error (RSME), rotation errors (Re), translation errors (Te) and registration time (Time) to evaluate the performance and registration quality of the algorithm. The formulas of various indicators are:

[0110]

[0111] Among them, p i and q j represents the corresponding point pair, R rest To test the rotation matrix, t rest To test the translation matrix, N is the total number of points.

[0112]

[0113] Among them, R trueis the true rotation matrix, and tr(R) represents the trace of the matrix R.

[0114] Te=||t true -t rest || 2

[0115] Among them, t true is the real translation matrix.

[0116] Registration time: The total time from the start of the algorithm to the completion of point cloud registration.

[0117] To verify the effectiveness of the algorithm, the proposed algorithm is experimented with ICP, PointLK and other algorithms on a self-constructed data set. The results are shown in Table 4. The hardware equipment used in the experiment of the present invention is Intel(R) Core(TM) i5-10200HCPU@2.40GHz, memory 16GB, NVIDIAGeForce GTX 1650Ti, video memory 4GB, and Pytorch is selected as the deep learning framework.

[0118] Table 4 Performance comparison of detection results of each method

[0119]

[0120] In addition, in order to verify the registration performance of the model for solder joints of various defect types, experiments were conducted under the same environment, and the results are shown in Table 5. For larger defect types such as bridging and excessive tin, the root mean square error of the registration can be as low as 0.0059 or less, and for smaller defect types such as insufficient tin, the root mean square error of the registration can be reduced to about 0.0031, and the time is reduced to about 1 s.

[0121] Table 5 Comparison of registration performance of each solder joint in the model

[0122]

[0123] At the same time, in order to verify the improvement of the model performance by the method proposed in the present invention, an ablation experiment was carried out under the same experimental environment, and the results are shown in Table 6.

[0124] Table 6 Ablation experiment results

[0125]

[0126]

[0127] In the study of PCB solder joint point cloud registration, in view of the shortcomings of the traditional ICP algorithm in feature extraction, the present invention introduces multiple improved modules to improve the registration accuracy and efficiency. First, the MSHF module is used to extract and fuse high-dimensional features of the source point cloud and the target point cloud, which significantly reduces the RSME to 0.0238 and reduces the registration time by 160 seconds. The experimental results show that the feature information of the solder joint after MSHF processing has an important influence on the registration result, and effectively improves the model performance. Then, on this basis, the w-FPS module is added to increase the extraction ratio of some feature points of the solder joint by setting different sampling weights. Compared with the original FPS, the RSME is further reduced to 0.0181. Finally, the KD-DTS module is introduced to filter the wrong point pairs in the iterative process, which further improves the registration accuracy and shortens the registration time. The experiment shows that the various improved modules work together to significantly enhance the registration effect, and the overall accuracy and efficiency of PCB solder joint point cloud registration are improved, thereby improving the data quality.

[0128] Step 10: Use Open3d to visualize the registration results.

[0129] Use Open3D's draw_geometries() method to visualize the registration results. Usually, different colors are used to distinguish the source point cloud and the target point cloud. Red represents the source point cloud and green represents the target point cloud. Fig.10 As shown in the figure, five kinds of registration results are represented, namely, defect-free solder joints, bridging, excessive tin, insufficient tin, and pulled tips.

Claims

1. A PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion, characterized in that: The following steps are involved: Step 1: Build a binocular 3D vision acquisition platform to scan the printed circuit board PCB to obtain high-precision point cloud data; Step 2: Segment, filter, and level the PCB point cloud data to improve the point cloud quality; Step 3: Use Cloudcompare software to divide the point cloud data into blocks and construct a binocular welding point cloud registration dataset; Step 4: Design and build an iterative closest point PCB solder point cloud registration model based on multi-scale feature extraction; Step 5: Use the PointNet++ model to extract local and global multi-scale features of the weld point cloud; Step 6: Determine the boundary line L between the substrate and the solder joint according to the height information distribution rule of the PCB point cloud; Step 7: Use the improved weighted farthest point sampling method w-FPS to efficiently sample the solder joints and substrates to optimize the feature extraction results; Step 8: Use a dynamic threshold screening strategy based on KD-tree to quickly remove low signal-to-noise ratio point pairs and perform iterative closest point ICP registration; Step 9: Complete the registration, record the registration results and evaluate the model registration performance based on the evaluation indicators; Step 10: Use Open3d to visualize the registration results; Specifically, the draw_geometries() method of Open3D is used to visualize the registration results, and different colors are used to distinguish the source point cloud and the target point cloud.

2. According to claim 1, a PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion is characterized in that: Specifically, step 1 includes first completing the opposite installation of two Hikvision robots 3DMVS, the construction of the conveyor belt, and the configuration of the environment, and then using the Hikvision robot 3DMVS and the conveyor belt together, adjusting the acquisition parameters, and completing the high-precision acquisition of the welding point cloud data.

3. According to claim 2, a PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion is characterized in that: The step 2 is specifically as follows: first, the point cloud data including the PCB point cloud and the conveyor belt point cloud are obtained by scanning with 3DMVS; the conveyor belt point cloud is removed by semantic segmentation technology, and the PCB point cloud is retained; Subsequently, a filtering algorithm is used to remove noise points in the point cloud to improve data quality. Finally, the RANSAC algorithm is used to perform plane fitting on the PCB point cloud and rotate the PCB plane to a posture parallel to the ground.

4. According to claim 1, a PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion is characterized in that: The step 3 is specifically as follows: using CloudCompare software to cut the processed PCB point cloud into multiple small pieces to form a registration data set; the data set includes a source point cloud set and a target point cloud set, providing data support for the subsequent registration process.

5. According to claim 1, a PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion is characterized in that: The step 4 is specifically as follows: first, use a deep learning algorithm to extract multi-scale features of the source point cloud and the target point cloud; then, use them as input for the ICP registration algorithm; and finally, visualize the registration results.

6. The PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion according to claim 1 is characterized in that: The step 5 specifically includes: using the multi-scale neighborhood aggregation technology of PointNet++ to perform multi-scale extraction of local and global features of point cloud data.

7. The PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion according to claim 1 is characterized in that: The step 6 is specifically as follows: (1) Analyze the height distribution law of the PCB point cloud through a histogram, and draw a height distribution histogram of the point cloud with the height value, i.e., the z value, as the horizontal coordinate and the number of point clouds as the vertical coordinate; (2) Determine the peak value and its RMS error in the histogram, and then shift it outward along the right side of the peak value by 0.1 times the RMS error. The corresponding horizontal coordinate is the boundary line L between the substrate and the solder joint.

8. The PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion according to claim 1 is characterized in that: The step 7 is specifically as follows: (1) According to the dividing line L, the PCB point cloud is divided into two parts: the substrate and the solder joint; (2) Set the total number of sampling points N and the sampling weight parameter w; (3) The number of sampling points for the solder joint part is calculated as N×w, and the number of sampling points for the substrate part is calculated as: N×(1-w); (4) Sampling of substrates and solder joints using the improved w-FPS; (5) Return the sampling results.

9. The PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion according to claim 1 is characterized in that: The step 8 is specifically as follows: (1) Construct a KD-Tree data structure for the input source point cloud and target point cloud; (2) According to the KD-Tree data structure, search for corresponding points and record corresponding point pairs (p i ,q j ) and calculate its Euclidean distance d ij ; (3) Find the maximum value d max and the minimum value d min ; According to the distribution of distance, it is divided into 50 groups, and the group spacing m=(d max -d min ) / 50; (4) Set the dynamic threshold to Z = d max -(2+0.5i)×m, i is the number of iterations; (5) The screening distance is [d min ,Z], these point pairs are marked as low signal-to-noise ratio point pairs and excluded from the registration calculation; (6) Set the minimum number of screened point clouds N0, if [d min ,Z], the corresponding points outside the range are less than N0, and the filter condition is reset to Z = d max -2×m; (7) Calculate the root mean square error f and determine whether it is less than a threshold; (8) Repeat steps (2) to (6) until convergence or the set number of iterations is reached and the results are output.

10. The PCB solder joint point cloud registration method based on binocular 3D vision multi-scale feature fusion according to claim 1, characterized in that: The step 9 is specifically as follows: An iterative closest point PCB solder point cloud registration model based on multi-scale feature extraction introduces PointNet++ for multi-scale feature extraction, and uses an improved weighted farthest point sampling algorithm to optimize feature extraction results. During registration, dynamic threshold screening conditions are set to filter out low signal-to-noise ratio point pairs. In the feature extraction stage, N is set to 8096, batch_size is set to 1, the optimizer Adam is selected, and the initial learning rate is 0.001; in the registration stage, the maximum number of iterations epoch is set to 100, and the threshold threshold is set to 1×10 -5 , the minimum number of screened point clouds N0 is 7072; Common indicators in the field of point cloud registration are used to evaluate the performance of the model, including root mean square error RSME, rotation error Re, translation error Te and registration time Time to evaluate the performance and registration quality of the algorithm. The formulas of various indicators are: Among them, p i and q j represents the corresponding point pair, R rest To test the rotation matrix, t rest To test the translation matrix, N is the total number of points; Among them, R true is the true rotation matrix, tr(R) represents the trace of the matrix R; Te=||t true -t rest ||2 Among them, t true is the real translation matrix; Registration time: The total time from the start of the algorithm to the completion of point cloud registration.

Images