A metal solder joint morphology detection method and a solder joint post-weld quality detection and identification system

Abstract

The application discloses a kind of metal weld point shape detection methods, it is related to the field of welding shape identification, including the following steps: the sampling system of multi-angle progression and multi-angle collection weld point point cloud of weld point is constructed;First, establish main point cloud in main visual angle, and design algorithm to judge the quality of main point cloud, if main point cloud data is insufficient, continue to collect auxiliary point cloud from other angles, and then carry out secondary discrimination;According to the set characteristics of the sample to be detected, design a fast 3D point cloud fusion scheme;Finally, the morphology of the weld point is output by machine learning description.The application also discloses a kind of weld point post-weld quality detection and identification system, based on the above method, and the output weld point shape information is matched with standard qualified weld point, and the quality evaluation result of weld point is given in real time.The collection efficiency of the application is high, the interference of environmental influence and sensor noise can be effectively overcome, the accuracy and integrity of point cloud file are improved, and then the accuracy of neural network identification data is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of welding form recognition, and particularly relates to a metal welding point form detection method and a welding point post-weld quality detection and recognition system. BACKGROUND

[0002] With the popularization of intelligent welding technology in modern industry, new requirements are put forward for the form recognition and quality evaluation of welding points after welding. In recent years, with the rapid development of three-dimensional point cloud modeling technology and machine learning visual models, various machine learning models and algorithms, especially in the field of vision, are widely used in the field of intelligent welding. By modeling the three-dimensional sensor to collect the modeling data of the welding point, the visual characteristics of various defects are analyzed, and a suitable machine learning model is constructed for training to form a model for recognizing and evaluating the form and quality of the copper laser welding welding point after welding.

[0003] Due to the complexity of the physical and chemical processes of copper welding, the form of the welding point after welding is also complex. The traditional three-dimensional sensor constructs a point cloud model of the welding point. Due to the complex environment of the welding site, the high reflectivity and light noise of the welding point have a great influence on the effect of the point cloud modeling, thereby affecting the recognition accuracy of the entire system. Therefore, obtaining high-precision point cloud has a high influence on the system accuracy.

[0004] In the paper "Deep Residual Learning for Image Recognition. arXiv: 1512.03385 [cs] (December 2015)", Kaiming He uses a residual learning framework to train the neural network. Directly using a residual neural network to train the welding point point cloud file, the entire training process is complex and low in efficiency.

[0005] In the prior art, traditional welding point detection is often through manual recognition or computer vision recognition based on two-dimensional pictures. However, manual recognition is low in efficiency, and the shooting of two-dimensional pictures is greatly affected by the surrounding environment light, which affects the recognition accuracy. Now some researchers introduce point cloud related technology into the field of intelligent welding, but the point cloud algorithm modeling will also be affected by certain environmental light, resulting in random missing of data, and the traditional calculation speed of point cloud is too slow, which limits its use scene.

[0006] Therefore, the skilled in the art is committed to developing a metal welding point form detection method and a welding point post-weld quality detection and recognition system to improve the collection efficiency, improve the algorithm efficiency, and improve the recognition efficiency and accuracy. SUMMARY

[0007] In view of the above defects of the prior art, the technical problems to be solved by the present application are low recognition accuracy and low efficiency of traditional point cloud, slow rate and low efficiency of point cloud fusion algorithm, and low efficiency caused by the fact that point cloud files cannot be directly input into a neural network for recognition.

[0008] To achieve the above-mentioned purpose, the present application provides a metal weld point shape detection method, comprising the following steps:

[0009] Step 1, a multi-angle progressive sampling system of a weld point is constructed to support multi-angle acquisition of weld point cloud;

[0010] The sample to be detected is a weld point or other object surface of metal material with light reflection characteristics; the surface of the weld point or object is approximately rectangular in orthographic projection and has high light reflection properties;

[0011] The multi-angle progressive sampling system comprises a structured light modeling 3D sensor and a support that can move the 3D sensor in spherical space, the sample to be detected is located near the center of the spherical space, the 3D sensor is placed at a predetermined position by moving the 3D sensor, and point cloud is acquired at each angle to avoid visual occlusion;

[0012] The sampling process is as follows: first, the sensor is located directly above the sample to be detected, and the straight line connecting the sample to be detected and the lens is perpendicular to the lens, and the point cloud at this angle is taken as the main point cloud; then, it is judged according to the algorithm whether the main point cloud data is complete, if the main point cloud data cannot meet the requirements, auxiliary point cloud acquisition is performed, that is, the straight line connecting the sample to be detected and the lens is deflected by 30 degrees to any position and a circular ring perpendicular to the straight line is made on the spherical surface as the moving track of the sensor, and the sample to be detected is taken at an angle of 30 degrees on the side every 90 degrees along the circular ring, and the data density of the main point cloud is selected to execute 1-4 times, and the point cloud generated in this step is taken as the auxiliary point cloud, wherein the same angle can be taken multiple times to avoid noise interference, and the modeling area of the sensor is limited according to the target to be detected, so that the sample to be detected can be completely modeled, and non-target objects are not modeled as much as possible;

[0013] The sample to be detected is composed of a main point cloud file and an auxiliary point cloud file, and each file supports multiple repeated sampling;

[0014] Step 2, according to the collective characteristics of the sample to be detected, a fast 3D point cloud fusion scheme is designed, and the specific operation is as follows:

[0015] Step 2.1, signal preprocessing, the target of the sample to be detected is separated, and the single weld point cloud information after preprocessing is obtained;

[0016] Step 2.2, pose correction of the sample target to be detected, after this step, the same sample target to be detected taken from different angles will have similar coordinate values in the coordinate system, and the initial value with high approximation is conducive to the speed improvement of subsequent point cloud algorithm fusion;

[0017] Step 2.3, extracting specific feature points of the sample target to be detected, using fewer data sets to represent the original point cloud set, so as to perform fast point cloud fusion, wherein the point cloud before fusion is the main point cloud modeled in step (1) or the main point cloud and the auxiliary point cloud modeled under each angle, which has visual information complementarity, and the point cloud data after fusion is more rich and comprehensive to reflect the topography of the sample to be detected;

[0018] Step 2.4, fusing the point cloud images of the same target from different angles;

[0019] Step 3, training machine learning, projecting the fused point cloud data X into XY plane to generate a gray image, which is used as the input of the convolutional neural network, and the convolutional neural network finally uses a multilayer perceptron for classification and judgment, and outputs the shape description of the welding spot, and the mathematical expression is as follows:

[0020] K(X), P(X) = class(img(X));

[0021] Where img(X) is a gray image conversion function, class() is a convolutional neural network, K(X) returns the category of the welding spot shape, and P(X) is the probability of the welding spot being qualified, and if P(X) > 0.5 is considered, the system considers it as a qualified welding spot.

[0022] Further, the repeated sampling in step 1 means that in order to avoid noise of the sensor and environmental light interference, the main point cloud and the auxiliary point cloud can be sampled multiple times, and the sampling process selects to add additional point light or line light, that is, multiple point cloud files are independently acquired at the same angle.

[0023] Further, the step 2.1 specifically includes the following steps:

[0024] Step 2.1.1, removing outliers, environmental light, noise, electromagnetic wave disturbance and other interference will make the original point cloud have an outlier set, and a filtering algorithm based on local statistical information is used to remove outliers, and the calculation steps are as follows:

[0025] The distances L1, L2,..., L n of K neighbor points of a given point X n to X n are calculated, and a Gaussian distribution N(n k , m k is fitted. 2), the deviation from the mean n k P*m k 2 Points outside of the range are removed as outlier noise points, as shown below:

[0026] If L m ∈[n k -P*m k 2 , n k +P*m k 2 ](m=1,2,...n) then point X is left m Otherwise, X m Eliminate, where n k is the mean of the fitted Gaussian distribution, m k 2 is the variance of the Gaussian distribution, and P is the system preset threshold;

[0027] Step 2.1.2: Filter out the solder point cloud data. Based on the characteristics of the target to be inspected, the point cloud is mainly divided into two areas: the point cloud model data Z1 of the solder point surface and the uncovered melt zone Z2. The point cloud plane of the Z2 area is perpendicular to the marker normal vector. The point cloud model is filtered and eliminated based on the normal direction to remove non-solder point data, as shown below:

[0028] If cos<(a,b,c),L>∈[-0.1,0.1], then delete X n , where (a,b,c) is the point X n Normal vector, L is the normal vector of the metal surface preset by the system;

[0029] Step 2.1.3, segment the solder joints. For the point cloud data processed in step 2.2, unsupervised density clustering is used to effectively implement preliminary segmentation and extraction of solder joints. Point sets clustered in the same category are considered to be the same solder joint, and the minimum bounding rectangle is calculated for adjustment. After that, the segmented solder joints are further calibrated using the slicing method based on the spatial minimum bounding box of the point cloud. After this step, a separate solder joint point cloud array can be obtained.

[0030] Furthermore, the step 2.3 further includes:

[0031] Step 2.3.1. Project the pose-corrected main point cloud and auxiliary point cloud along the Z axis onto the XY plane, perform grayscale transformation on the projection surface, and calculate the two-dimensional feature points of the grayscale image.

[0032] Step 2.3.2: Based on the correspondence between the two-dimensional grayscale image and the point cloud, the first N most characteristic points of the calculated two-dimensional feature points are marked in the point cloud as the input data for point cloud fusion.

[0033] Further, in the sampling process in the step 1, when the to-be-detected sample is in a suitable scanning area, the sensor starts to establish a main point cloud directly above the sample, and then whether to continue to scan and establish an auxiliary point cloud is determined according to the judgment of the main point cloud density detection algorithm on the main point cloud, and the present collection process is completed after the modeling of the main point cloud and the auxiliary point cloud is completed.

[0034] Further, the main point cloud density detection algorithm is specifically executed as follows: the main projection plane of the main point cloud is divided into four regions, and the point cloud density of each region is calculated, that is, the ratio N / S of the point number N and the area S is solved, and if the point cloud density is lower than a preset threshold value, the angle and the number of the auxiliary point cloud are determined and the auxiliary point cloud is shot.

[0035] Further, the to-be-detected sample is a brazed welding spot, wherein the covered fusion zone is a rectangle directly opposite the sensor, the uncovered fusion zone is a strip-shaped support, the surface of the support is perpendicular to the covered fusion zone, and only the welding spot and the marking plane are in the visual angle of the sensor modeling, and no other objects are scanned into the point cloud file.

[0036] Further, the morphology description of the to-be-detected sample in the step 3 is extracted by using a convolutional neural network according to a generated point cloud grayscale image, and the model of the convolutional neural network is ResNet-152, which comprises a first convolutional layer group, a second convolutional layer group, a third convolutional layer group, one fully connected layer and one Softmax layer.

[0037] Further, the welding spot morphology description output in the step 3 is one of several categories of preset welding spot models.

[0038] The present application also provides a welding spot post-welding quality detection recognition system, which uses the metal welding spot morphology detection method as described above, and matches the size of the welding spot surface, the welding spot height difference information and the surface hole information calculated in the step 3 with the related parameters of the standard qualified welding spot, and gives the quality evaluation result of the welding spot in real time.

[0039] The present application optimizes the point cloud recognition model based on machine learning from two aspects of point cloud processing and machine learning algorithm. A suitable data set is constructed and a suitable neural network structure is built to improve the recognition accuracy and training efficiency of the trained machine learning model.

[0040] Compared with the prior art, the present application has the following beneficial technical effects:

[0041] 1. In the aspect of point cloud recognition, the application adopts a mobile single camera to take multi-view point cloud, avoiding the missing of point cloud caused by light or occlusion in a specific view. By calculating the local point cloud density, it automatically judges whether the current view point cloud is missing. The missing position is calculated, and the missing point cloud is reselected for secondary shooting collection at a suitable view angle, and multi-view point cloud fusion is performed.

[0042] Compared with the traditional point cloud recognition algorithm, the application adopts a multi-angle sampling system to extract point cloud of welding points with complementary information from multiple angles. The application greatly increases the integrity of point cloud data collection, and the dynamic detection process only collects multiple times when necessary, improving the algorithm efficiency. Only one sensor is needed, thereby reducing the cost.

[0043] 2. The feature points representing the entire point cloud are extracted by calculating the features of the point cloud. At the same time, only the feature points are used to calculate the transformation matrix. According to the calculation method of the point cloud feature points, the features of each point are calculated and sorted. The top 10% of the feature points are retained as the representative data of the entire point cloud. The speed of the entire point cloud fusion can be improved, and the accuracy of the point cloud can be improved.

[0044] The application uses the FPFH algorithm to accelerate point cloud fusion in the field of welding point cloud fusion for the first time, uses fewer point sets to replace the original point cloud to calculate the point cloud space transformation matrix, reduces the number of calculations and data size, and improves the overall point cloud fusion speed of the system.

[0045] 3. The application designs a unique point cloud-image conversion algorithm, which can convert point cloud into 2D image and retain as many features as possible. According to the spatial pose of the target object to be detected, a projection algorithm of the welding surface is automatically generated to generate a feature map with height information, successfully combines point cloud files and neural networks, effectively eliminates the influence of traditional RGB cameras on stains and environmental light noise, and the recognition efficiency can reach more than 98%, improving the recognition efficiency.

[0046] By using the filtering and point cloud fusion algorithm with the welding points, the most real height surface of the welding point surface is effectively retained, and the height gray scale image is converted into a height gray scale image. The gray scale image is used for training of the convolutional neural network model, and the trained machine learning model can effectively obtain the features of the welding points and be used for shape recognition and quality detection.

[0047] The application focuses on improving the efficiency of the residual network in the welding spot point cloud recognition. According to the pre-processing of the input data and the characteristics of the target to be detected, the number of layers and parameters of the residual neural network are designed, which not only improves the training efficiency of the whole network, but also enables the network to learn the local and global information of the welding spot point cloud more efficiently due to the comprehensive data obtained by the fusion of complementary information. The network has good performance and recognition effect in welding spot shape description, effectively filling the industry gap.

[0048] The concept, specific structure and technical effects of the present application will be further described below with reference to the accompanying drawings, so as to fully understand the purpose, features and effects of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a method execution process schematic diagram of an embodiment of the present application;

[0050] Figure 2 is a multi-angle 3D point cloud modeling scanning work flow chart of an embodiment of the present application on a welding spot;

[0051] Figure 3 is a schematic diagram of an embodiment of the present application in which a single welding spot point cloud is segmented, denoised, decentered, and adjusted in pose after a series of denoising, redundant welding spot removal, and centering;

[0052] Figure 4 is a point cloud fusion effect diagram of an embodiment of the present application in which a single welding spot is modeled at different angles;

[0053] Figure 5 is a multi-angle 3D point cloud and deep network based welding spot post-weld quality detection recognition system work flow chart. DETAILED DESCRIPTION

[0054] The present application can be embodied in many different forms, and the embodiments are implemented on the basis of the technical solutions of the present application, and detailed implementation modes and specific operation processes are given, but the protection scope of the present application is not limited to the embodiments mentioned in the text.

[0055] EMBODIMENT

[0056] The sensor used in this embodiment is a 3D point cloud sensor based on structured light modeling, and the target to be detected is a copper laser welding spot. The welding spot projection area is approximately rectangular, with a length of 4-5 mm and a width of 2-3 mm. The algorithm and model are established and tested on a computer with Core i7-10870H CPU. Different angles and different light complementary point clouds are obtained by rotating the machine within a fixed range.

[0057] AsFigure 1 As shown, it is a method execution process diagram. A laser welding post-welding spot morphology detection method based on multi-angle 3D point cloud and deep network, comprising the following steps:

[0058] Step 1. According to Figure 2 The position shown is the modeling of the target welding spot to be detected. The modeling view directly above is the main point cloud, and the rest is the auxiliary point cloud, which allows multiple point cloud collection at the same view. In the scanning modeling process, only the target object is scanned into the point cloud data, and the remaining welding spots are covered. According to the relative position, the point cloud coordinate set is used to determine whether it is a point cloud set of the same welding spot.

[0059] Step 2. Simplify the original point cloud data into an N*3 matrix, where N represents the number of point sets, and 3 is the XYZ coordinate. Denoise and separate the point cloud data, and only keep the point cloud information on the welding spot surface. Figure 3 The figure shows the schematic diagram of the single welding spot segmented from the welding spot point cloud after a series of denoising, removing redundant welding spots, decentralization, and pose adjustment.

[0060] Step 3. Perform a screening operation on the point cloud information, the main purpose is to only keep the point cloud data of the welding spot surface shape, and remove the rest. The point cloud to be removed is noise caused by noise points, outliers and support parts in the unmelted area. The specific removal method is as follows:

[0061] Step 3.1. Traverse the point cloud. For a specific point X m , calculate the distance L1, L2,..., L n of its N neighbors to this point X m , use the distance data to fit a Gaussian distribution N(n k , m k ), artificially preset a threshold P, and remove points that deviate from the mean n k by P*m k 2 , that is, if a specific point L m ∈[nk-P*mk 2 , nk+P*mk 2 ](m=1,2,...n) then keep the point X m , otherwise remove X m , where n k is the mean of the fitted Gaussian distribution, m k is the variance of the Gaussian distribution, and P is the system preset threshold.

[0062] Step 3.2. Traverse the point cloud. For a specific point X m, a plane S is fitted according to its adjacent K neighbor points, the normal vector of the plane S is considered as the normal vector of the point, the cosine value of the included angle between the normal vector of the point X m and the normal vector of the preset plane S is calculated, if cos<(a, b, c), L ∈ [-0.1, 0.1], then X is deleted m . The unmelted area of the original point cloud file is removed, and only the point cloud data of the solder surface is retained.

[0063] Step 4. For the coordinate set of the point cloud after step 3 processing, only the solder surface data is retained, a density clustering method is used to extract the point cloud data set of each individual solder to obtain P1, P2,..., P n n, and a decentralization processing is performed, that is, the following operation is performed: P i = P i -Means(P i )(i = 1, 2,..., n), and the data of the individual solder is obtained.

[0064] Step 5. Direct density clustering cannot effectively separate the point cloud of the solder which is too close, the projection area of the cut solder surface is calculated to determine whether the separation fails, if so, principal component analysis is used to make a minimum rectangular bounding box of the solder and perform pose transformation, as shown in Figure 3 , the long side of the minimum rectangular bounding box after pose transformation is parallel to the X axis and the normal vector of the solder surface is parallel to the Z axis, at this time, a slicing method is used for further separation of the solder:

[0065] If the normal solder point cloud has multiple solder adhesions, a slice with a thickness of 0.05 mm can be made every 0.05 mm along the length and width direction of the minimum bounding box, and the number of point clouds in the slice and the preset value are used to distinguish the solder separation to realize point cloud cutting, and after this step, the individual solder point cloud array can be obtained.

[0066] Step 6. The eigenvalue calculation is performed on the main point cloud and the auxiliary point cloud of the same solder without considering the spatial coordinates and the scale, and the first N points with the largest eigenvalue in the point cloud set are used to replace the point cloud, and the spatial second transformation matrix R of the auxiliary point cloud and the main point cloud is calculated.

[0067] According to the spatial second transformation matrix R, the auxiliary point cloud and the main point cloud are sequentially fused to obtain the point set coordinates after fusion of the point cloud photographed from different angles.

[0068] The fused point cloud is generated into a gray scale image, and the gray scale image generation algorithm is as follows:

[0069] The Z coordinate of the point cloud is linearly stretched to serve as a pixel gray value, the XY coordinate of the point cloud is normalized and linearly stretched to serve as a pixel position, the adjacent point average interpolation method is used to fill the spare position of the picture, and the Gaussian filter is used to filter the filled gray scale picture, and the default background color of the gray scale picture is black, that is, the pixel value is 0.

[0070] In order to facilitate training and testing, the gray scale data set is divided into a training set and a test set according to a 60% and 40% ratio. In fact, other ratios can also be used for division, or a cross-test method can be used.

[0071] In the training process of the model, the K-fold cross-validation method is used to suppress overfitting, the initial K is 5, the time consumption t of the model and the accuracy a of the model are recorded.

[0072] The gray scale picture is put into the ResNet network to extract the point cloud view feature, and a fully connected layer and a Softmax layer are used at the end of the network to output the morphology recognition result. The model of the convolutional neural network is ResNet-152, which includes a first convolutional layer group, a second convolutional layer group, a third convolutional layer group, a fully connected layer and a Softmax layer. Figure 4 is a point cloud fusion effect picture of different angles of a single welding spot.

[0073] After the model training is completed, actual application can be carried out, for a welding spot to be detected, the main point cloud and the auxiliary point cloud are obtained by using the same 3D point cloud modeling method, the morphology characteristics of the welding spot are output in real time by using the system, and the qualified probability is given, and the specific detection method is shown in Figure 5 .

[0074] The present application optimizes the point cloud imaging and the machine learning model to improve the accuracy and speed of the system in describing the shape of the welding spot. Firstly, the point cloud with complementary information is photographed from multiple angles, and more stable and anti-noise point cloud data is obtained through point cloud fusion, so that more accurate point cloud data is obtained. In addition, in order to increase the fusion speed, a lot of algorithm preprocessing design is made, and finally a residual convolutional neural network is designed according to the characteristics of the target to be detected, so as to improve the accuracy and efficiency of the data set training and prediction.

[0075] The above describes the preferred embodiments of the present application in detail. It should be understood that those skilled in the art can make many modifications and changes without creative labor according to the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment on the basis of the prior art according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A method of detecting a morphology of a metal solder joint, characterized by, The method comprises the following steps: Step 1, a multi-angle progressive sampling system of the welding spot is constructed to support multi-angle acquisition of the welding spot point cloud; The sample to be detected is a welding spot with metal material and light reflection characteristics; the welding spot has a long rectangular approximate orthographic projection and light reflection properties; The multi-angle progressive sampling system comprises a structured light modeling 3D sensor and a support capable of moving the 3D sensor in a spherical space, wherein the sample to be detected is located near the spherical center of the spherical space, the 3D sensor is placed at a preset position through movement of the 3D sensor, and point clouds at various angles are acquired to avoid visual occlusion; The sampling process is as follows: firstly, the sensor is located directly above the sample to be detected, a straight line connecting the sample to be detected and a lens is perpendicular to the lens, and a point cloud at this angle is taken as a main point cloud; then, whether the main point cloud data is complete is determined according to an algorithm, if the main point cloud data cannot meet the requirements, auxiliary point cloud acquisition is performed, that is, the straight line connecting the sample to be detected and the lens is deflected by 30 degrees to any position, a circular ring perpendicular to the straight line is formed on the spherical surface as a sensor movement track, the sample to be detected is shot at an angle of 30 degrees from the circular ring every 90 degrees, and the number of times of shooting is selected according to the data density of the main point cloud, the point cloud generated in this step is taken as an auxiliary point cloud, and the same angle can be shot multiple times to avoid noise interference, and the sampling process is limited to the modeling area of the sensor according to the target to be detected, so that the sample to be detected can be completely modeled; The sample to be detected is composed of a main point cloud file and an auxiliary point cloud file, and each file supports multiple repeated sampling; Step 2, a fast 3D point cloud fusion scheme is designed according to the set characteristics of the sample to be detected, and the specific operation is as follows: Step 2.1, signal preprocessing, the sample to be detected is separated to obtain separate welding spot point cloud information after preprocessing; Step 2.2, the sample to be detected is subjected to pose correction, and after this step, the same sample to be detected shot from different angles has similar coordinate values in the coordinate system; Step 2.3, the sample to be detected is subjected to specific feature point extraction, and the original point cloud set is represented by using a data set, so that fast point cloud fusion can be performed, wherein the point cloud before fusion is the main point cloud or the main point cloud and the auxiliary point cloud modeled under each angle, the point cloud after fusion has visual information complementarity, and point cloud data reflecting the topography of the sample to be detected is acquired; Step 2.4, the point cloud images of the same target from different angles are fused; Step 3, training of machine learning, fused point cloud data X e R 3 , projected to XY plane to generate grayscale image as input of convolutional neural network, and the convolutional neural network finally uses multilayer perceptron for classification and judgment, and outputs the morphology description of the welding spot, and the mathematical expression is as follows: K(X), P(X) = class( img(X)); Wherein img(X) is a grayscale conversion function, class() is a convolutional neural network, K(X) returns a category of the welding spot shape, and P(X) is a probability of the welding spot, if P(X)>0.5 is considered, the system considers that the welding spot is qualified.

2. The metal joint morphology detection method of claim 1, wherein, The repeated sampling in step 1 refers to: in order to avoid the noise of the sensor and the interference of the ambient light, the main point cloud and the auxiliary point cloud can be sampled multiple times, and the sampling process selects to add additional point light or line light, that is, multiple point cloud files are independently obtained at the same angle.

3. The metal joint morphology detection method of claim 1, wherein, The step 2.1 specifically includes the following steps: Step 2.1.1, remove outlier noise points, environmental light, noise, electromagnetic wave disturbance interference will make the original point cloud have an outlier point set, and a filtering algorithm based on local statistical information is used to remove the outlier noise points, and the calculation step is: Calculate the given point X m K neighbors of point X m The distances L1, L2, ..., L n And fit the Gaussian distribution N(n k , m k 2 ), the deviation from the mean n k P m k 2 Points outside of the range are removed as outlier noise points, as shown below: If L m ∈ [n k - P m k 2 , n k + P m k 2 ] (m = 1, 2,... n) then leave point X m , otherwise reject X m , where n k is the mean of the fitted Gaussian distribution, m k 2 is the variance of the Gaussian distribution, and P is a system preset threshold. Step 2.1.2, screen out the solder joint point cloud data, according to the characteristics of the to-be-detected target, the point cloud is mainly divided into two regions: the point cloud model data Z1 of the solder joint surface and the non-covered fusion zone Z2, wherein the point cloud plane of the Z2 region is perpendicular to the marker normal vector, and the non-solder joint data in the point cloud model is screened out according to the normal direction, and is represented as follows: If cos < (a, b, c), L > ∈ [-0.1, 0.1], then delete X n where (a, b, c) is the normal vector of point X n and L is the normal vector of the metal surface preset by the system; Step 2.1.3, segment the solder joint, the point cloud data processed in step 2.1.2 is segmented and extracted based on unsupervised density clustering, wherein the point set in the same category is considered to be the same solder joint, and the minimum enclosing cuboid is calculated for adjustment, and then the segmented solder joint is further calibrated using the slice method according to the spatial minimum enclosing box of the point cloud. After this step, a single solder joint point cloud array can be obtained.

4. The metal joint morphology detection method of claim 1, wherein, The step 2.3 further includes: Step 2.3.1, project the main point cloud and the auxiliary point cloud after the pose correction along the Z axis to the XY plane, convert the projection plane to a grayscale image, and calculate the two-dimensional feature points of the grayscale image; Step 2.3.2, according to the correspondence between the two-dimensional grayscale image and the point cloud, the first N most characteristic points of the calculated two-dimensional feature points are marked in the point cloud as input data for point cloud fusion.

5. The metal joint morphology detection method of claim 1, wherein, In the sampling process in step 1, when the to-be-detected sample is in a suitable scanning area, the sensor starts to establish a main point cloud directly above the sample, and then determines whether to continue to scan and establish an auxiliary point cloud according to the density detection algorithm of the main point cloud. When the main point cloud and the auxiliary point cloud modeling is completed, the present collection process is completed.

6. The metal joint morphology detection method of claim 5, wherein, The density detection algorithm of the main point cloud is specifically executed as follows: the main projection plane of the main point cloud is divided into four regions, and the point cloud density of each region is calculated, that is, the ratio N / S of the point number N and the area S is solved. If the point cloud density is lower than the preset threshold value, the angle and the number of the auxiliary point cloud need to be determined and the auxiliary point cloud is shot.

7. The metal joint morphology detection method of claim 5, wherein, The to-be-detected sample is a copper solder joint, wherein the covered fusion zone is a rectangle directly opposite the sensor, and the non-covered fusion zone is a strip-shaped support whose surface is perpendicular to the covered fusion zone, and only the solder joint and the marker plane are in the view angle of the sensor modeling, and no other objects are scanned into the point cloud file.

8. The metal joint morphology detection method of claim 1, wherein, In step 3, the morphology description of the to-be-detected sample is extracted by a convolutional neural network according to the generated point cloud grayscale image, and the model of the convolutional neural network is ResNet-152, which includes a first convolutional layer group, a second convolutional layer group, a third convolutional layer group, one fully connected layer and one Softmax layer.

9. The metal joint morphology detection method of claim 1, wherein, The weld joint shape description output in step 3 is one of several categories of pre-defined weld joint models. The weld joint shape description output in step 3 is one of several categories of pre-defined weld joint models.

Images