Metal welding spot shape detection method and welding spot post-welding quality detection and identification system

Abstract

The invention discloses a metal welding spot form detection method, and relates to the field of welding form identification, and the method comprises the following steps: constructing a sampling system for multi-angle progressive and multi-angle welding spot cloud collection of welding spots; firstly, a main point cloud is established at a main view angle, an algorithm is designed to judge the quality of the main point cloud, if data of the main point cloud is insufficient, auxiliary point clouds continue to be collected from other view angles, and then secondary judgment is carried out; designing a rapid 3D point cloud fusion scheme according to the set features of the to-be-detected sample; and finally, outputting the form description of the welding spot through machine learning. The invention further discloses a system for detecting and identifying the quality after welding spot welding, and the system is based on the method, the output welding spot form information is matched with the standard qualified welding spot, and the quality evaluation result of the welding spot is given in real time. The method is high in collection efficiency, can effectively overcome the environmental influence and the interference of sensor noise, improves the accuracy and integrity of the point cloud file, and improves the accuracy of the neural network recognition data.

Description

Technical Field

[0001] The invention relates to the field of welding morphology recognition, and in particular to a metal welding spot morphology detection method and a welding spot post-welding quality detection and recognition system. Background Art

[0002] With the popularization of intelligent welding technology in modern industry, new requirements have been put forward for the morphological recognition and quality assessment of welding points after welding. In recent years, with the rapid development of 3D point cloud modeling technology and machine learning visual models, various machine learning models and algorithms, especially in the field of vision, have been widely used in the field of intelligent welding. Through the modeling of 3D sensors, the modeling data of welding points are collected, the visual characteristics of various defects are analyzed, and a suitable machine learning model is constructed for training to form a model for the morphological and quality recognition and assessment of welding points after copper laser welding.

[0003] Since the physical and chemical processes of copper welding are relatively complex, the shape of the solder joint after welding is also relatively complex. Traditional 3D sensors construct point cloud models of solder joints. Due to the complex environment of the welding site, the high reflection and light noise of the solder joints have a great impact on the effect of point cloud modeling, thereby affecting the recognition accuracy of the entire system. Therefore, obtaining high-precision point clouds has a great impact on system accuracy.

[0004] In the paper "Deep Residual Learning for Image Recognition.arXiv:1512.03385[cs](December 2015)", Kaiming He used a residual learning framework to train the neural network. Directly using the residual neural network to train the weld point cloud file makes the entire training process complex and inefficient.

[0005] In the existing technology, traditional solder joint detection is often done through manual recognition or computer vision recognition based on two-dimensional images. However, manual recognition is inefficient and the shooting of two-dimensional images is greatly affected by the ambient light, which affects the accuracy of recognition. Now some researchers have introduced point cloud related technologies into the field of intelligent welding, but point cloud algorithm modeling will also be affected by certain ambient light, resulting in random data loss, and the traditional calculation speed of point cloud is too slow, which limits its use scenarios.

[0006] Therefore, technicians in this field are committed to developing a metal solder joint morphology detection method and a solder joint post-welding quality detection and identification system to improve acquisition efficiency, enhance algorithm efficiency, and simultaneously improve recognition efficiency and accuracy. Summary of the invention

[0007] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is the low accuracy and efficiency of traditional point cloud recognition, the slow rate and low efficiency of point cloud fusion algorithm, and the inefficiency caused by the inability to directly input point cloud files into the neural network for recognition.

[0008] To achieve the above object, the present invention provides a method for detecting the morphology of metal solder joints, comprising the following steps:

[0009] Step 1: Build a multi-angle progressive sampling system for solder joints to support multi-angle collection of solder joint point clouds;

[0010] The sample to be tested is a metal material, a solder joint or other object surface with reflective properties; the orthographic projection of the solder joint or the surface of the object is approximately rectangular and has a high reflective property;

[0011] The multi-angle progressive sampling system includes: a 3D sensor modeled by structured light and a bracket that can move the 3D sensor in a spherical space, the sample to be detected is located near the center of the spherical space, and the 3D sensor is placed in a preset position by moving the 3D sensor to obtain point clouds shot at various angles to avoid visual occlusion;

[0012] The sampling process is as follows: first, the sensor is located directly above the sample to be detected, and the straight line connecting the sample to be detected and the lens is perpendicular to the lens, and the point cloud captured at this angle is used as the main point cloud; then, the algorithm is used to determine whether the main point cloud data is complete. If the main point cloud data cannot meet the requirements, the auxiliary point cloud is collected, that is, the straight line connecting the sample to be detected and the lens is deflected 30 degrees to any position and a circular ring perpendicular to the straight line is made on the spherical surface as the sensor movement trajectory, and the sample to be detected is photographed at a 30-degree angle every 90 degrees along the circular ring. The execution is selected 1-4 times according to the data density of the main point cloud. The point cloud generated in this step is used as the auxiliary point cloud, where multiple photos can be taken at the same angle to avoid noise interference. The entire sampling process should limit the modeling area of ​​the sensor according to the target to be detected, so that the sample to be detected can be completely modeled, and non-target objects should not be modeled as much as possible;

[0013] The sample to be detected consists of a main point cloud file and an auxiliary point cloud file, and each file supports multiple repeated sampling;

[0014] Step 2: Design a fast 3D point cloud fusion solution based on the collective features of the samples to be detected. The specific operations are as follows:

[0015] Step 2.1, signal preprocessing, specifically separating the sample target to be detected to obtain preprocessed individual welding point cloud information;

[0016] Step 2.2, correct the posture of the sample target to be detected. After this step, the same sample target to be detected taken from different perspectives will have similar coordinate values ​​in the coordinate system. The high approximation of the initial value is conducive to the speed improvement of the subsequent point cloud algorithm fusion;

[0017] Step 2.3, extracting specific feature points of the sample target to be detected, using a smaller data set to represent the original point cloud set, and then quickly performing point cloud fusion, wherein the point cloud before fusion is the main point cloud modeled in step (1) or the main point cloud and the auxiliary point clouds modeled under various viewing angles, which have complementary visual information, and after fusion, more abundant and comprehensive point cloud data reflecting the morphology of the sample to be detected is obtained;

[0018] Step 2.4, fusing point cloud images of the same target object from different perspectives;

[0019] Step 3: Perform machine learning training, project the fused point cloud data X∈R3 onto the XY plane to generate a grayscale image as the input of the convolutional neural network. The convolutional neural network finally uses a multi-layer perceptron for classification and judgment, and outputs the morphological description of the solder joint. Its mathematical expression is as follows:

[0020] K(X),P(X)=class(img(X));

[0021] Among them, img(X) is the grayscale image conversion function, class() is the convolutional neural network, K(X) returns the category of the solder joint shape, P(X) is the probability of the solder joint being qualified, and if P(X)>0.5, the system considers it to be a qualified solder joint.

[0022] Furthermore, the repeated sampling described in step 1 means that in order to avoid noise and ambient light interference from the sensor, the main point cloud and the auxiliary point cloud can be sampled multiple times, and the sampling process chooses to add additional point light sources or line light sources, that is, to independently obtain multiple point cloud files at the same angle.

[0023] Furthermore, the step 2.1 specifically includes the following steps:

[0024] Step 2.1.1, remove outlier noise. Ambient light, noise, electromagnetic wave disturbance and other interferences will cause outlier points in the original point cloud. A filtering algorithm based on local statistical information is used to remove outlier noise. The calculation steps are:

[0025] Calculate the given point X n The K neighbors of X n The distances L1, L2, ..., L n And fit the Gaussian distribution N(n k ,m k 2), the deviation from the mean n k P*m k 2 Points outside of the above are removed as outlier noise points, as shown below:

[0026] If L m ∈[n k -P*m k 2 , n k +P*m k 2 ](m=1,2,...n) then point X is left m Otherwise, X m Eliminate, where n k is the mean of the fitted Gaussian distribution, m k 2 is the variance of Gaussian distribution, P is the system preset threshold;

[0027] Step 2.1.2, filter out the solder joint point cloud data. According to the characteristics of the target to be detected, the point cloud is mainly divided into two areas: the point cloud model data Z1 of the solder joint surface and the uncovered melt zone Z2. The point cloud plane of the Z2 area is perpendicular to the marker normal vector. The non-solder joint data in the point cloud model are filtered out according to the normal direction, as shown below:

[0028] If cos<(a,b,c),L>∈[-0.1,0.1], then delete X n , where (a,b,c) is the point X n Normal vector, L is the normal vector of the metal surface preset by the system;

[0029] Step 2.1.3, segment the solder joints. For the point cloud data processed in step 2.2, unsupervised density clustering is used to effectively implement preliminary segmentation and extraction of solder joints. Points in the same category are considered to be the same solder joint, and the minimum enclosing cuboid is calculated for adjustment. After that, the segmented solder joints are further calibrated using the slicing method based on the minimum spatial enclosing box of the point cloud. After this step, a separate solder joint point cloud array can be obtained.

[0030] Furthermore, the step 2.3 also includes:

[0031] Step 2.3.1, project the main point cloud and auxiliary point cloud after posture correction along the Z axis to the XY plane, transform the projection surface into a grayscale image, and calculate the two-dimensional feature points of the grayscale image;

[0032] Step 2.3.2: According to the correspondence between the two-dimensional grayscale image and the point cloud, the first N most characteristic points of the calculated two-dimensional feature points are marked in the point cloud as the input data for point cloud fusion.

[0033] Furthermore, during the sampling process in step 1, when the distance of the sample to be detected is within a suitable scanning area, the sensor starts to establish a main point cloud directly above the sample, and then determines whether to continue scanning to establish an auxiliary point cloud based on the main point cloud's density detection algorithm. When the main point cloud and the auxiliary point cloud are modeled, this acquisition process is completed.

[0034] Furthermore, the density detection algorithm of the main point cloud is specifically implemented as follows: the main projection surface of the main point cloud is divided into 4 areas to calculate the point cloud density of each area, that is, to solve the ratio N / S of the number of points N and the area S. If the point cloud density is lower than a preset threshold, it is necessary to determine the angle and number of the auxiliary point cloud and shoot the auxiliary point cloud.

[0035] Furthermore, the sample to be tested is a copper-welded solder joint, wherein the melt-coated area is a rectangle facing the sensor, and the unmelted area is a strip-shaped bracket whose surface is perpendicular to the melt-coated area, and there are only solder joints and marking planes within the viewing angle of the sensor modeling, and no other objects are scanned into the point cloud file.

[0036] Furthermore, the morphological description of the sample to be detected in step 3 is to extract features based on the generated point cloud grayscale image using a convolutional neural network, and the convolutional neural network model is ResNet-152, which includes a first convolutional layer group, a second convolutional layer group, a third convolutional layer group, 1 fully connected layer and 1 Softmax layer.

[0037] Furthermore, the solder joint morphology description output in step 3 is one of several categories of preset solder joint models.

[0038] The present invention also provides a recognition system for post-weld quality inspection of welds. Using the metal weld morphology detection method as described above, the recognition system for post-weld quality inspection of welds matches the weld surface size, weld height difference information, and surface hole information calculated in step 3 with standard qualified weld-related parameters, and gives real-time weld quality assessment results.

[0039] The present invention starts from the two perspectives of point cloud processing and machine learning algorithm, and performs specific optimization on the point cloud recognition model based on machine learning. It constructs a suitable data set and builds a suitable neural network structure to improve the recognition accuracy and training efficiency of the machine learning model.

[0040] Compared with the prior art, the present invention has the following beneficial technical effects:

[0041] 1. In terms of point cloud recognition, the present invention uses a single mobile camera to shoot multi-view point clouds to avoid point cloud loss caused by illumination or occlusion at a specific view. By calculating the local point cloud density, it automatically determines whether the point cloud at the current view is missing. The missing position is calculated, and if missing, the appropriate view will be re-selected for secondary shooting and collection, and multi-view point cloud fusion is performed.

[0042] Compared with the traditional point cloud recognition algorithm, the present invention adopts a multi-angle sampling system to extract welding point clouds with complementary information from multiple angles. The present invention greatly increases the integrity of point cloud data collection, and its dynamic detection process only performs multiple collections when necessary, which improves the efficiency of the algorithm. Only one sensor is needed, thereby reducing costs.

[0043] 2. Perform feature calculation on the point cloud and extract feature points that can represent the entire point cloud. At the same time, only calculate the transformation matrix for the feature points. According to the calculation method of the point cloud feature points, perform feature calculation and sorting for each point. Keep the top 10% of the largest feature points as representative data of the entire point cloud. This can improve the speed of the entire point cloud fusion and improve the accuracy of the point cloud.

[0044] The present invention is the first to use the FPFH algorithm to accelerate point cloud fusion in the field of weld point cloud fusion, using a smaller number of point sets to replace the original point cloud to calculate the point cloud space transformation matrix, reducing the number of calculations and data size, and improving the overall point cloud fusion speed of the system.

[0045] 3. The present invention has designed a unique point cloud-image conversion algorithm, which allows the point cloud to be converted into a 2D image and retains the required features as much as possible. According to the spatial posture of the target object to be detected, the projection algorithm of the welding surface is automatically generated, and a feature map with height information is generated. The point cloud file and the neural network are successfully combined, effectively eliminating the influence of stains and ambient light noise that traditional RGB cameras cannot eliminate. The recognition efficiency can reach more than 98%, which improves the recognition efficiency.

[0046] By using filtering and point cloud fusion algorithms with solder joints, the most realistic height curves of the solder joint surfaces are effectively retained and converted into height grayscale images. The grayscale images are used to train convolutional neural network models. The trained machine learning model can effectively obtain the characteristics of solder joints and use them for morphological recognition and quality inspection.

[0047] The present invention focuses on improving the efficiency of residual networks in solder joint point cloud recognition. According to the preprocessing of input data and the characteristics of the target to be detected, the appropriate number of layers and parameters of the residual neural network are designed, which not only improves the training efficiency of the entire network, but also because the input point cloud has more comprehensive data after the fusion of complementary information, this network can more efficiently learn the local and global information of point cloud solder joints, and has good performance and recognition effect in solder joint shape description, effectively filling the gap in the industry.

[0048] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a schematic diagram of the method execution process of an embodiment of the present invention;

[0050] Figure 2 This is a workflow diagram of modeling and scanning of welding points by multi-angle 3D point cloud according to an embodiment of the present invention;

[0051] Figure 3 It is a schematic diagram of a single solder point segmented from a solder point cloud according to an embodiment of the present invention after a series of denoising, redundant solder point removal, decentralization, and posture adjustment;

[0052] Figure 4 This is a point cloud fusion effect diagram of a single welding point modeled at different angles in an embodiment of the present invention;

[0053] Figure 5 It is a workflow diagram of the recognition system for post-weld quality inspection of weld spots based on multi-angle 3D point cloud and deep network. DETAILED DESCRIPTION

[0054] The following describes several preferred embodiments of the present invention with reference to the drawings in the specification, so that the technical content is clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, which are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes, but the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0055] Example

[0056] The sensor used in this embodiment is a 3D point cloud sensor based on structured light modeling. The target to be detected is a copper laser welding spot. The projection area of ​​the spot is approximately rectangular, with a length of 4-5mm and a width of 2-3mm. The algorithm and model establishment and test computer uses a Core i7-10870H CPU, and obtains complementary point clouds of different viewing angles and different lighting by mechanically rotating within a fixed range.

[0057] like Figure 1 A method for detecting the morphology of weld spots after laser welding based on multi-angle 3D point cloud and deep network comprises the following steps:

[0058] Step 1. Follow Figure 2 The position shown is the 3D point cloud modeling of the target weld to be detected. The modeling perspective directly above is the main point cloud, and the rest are auxiliary point clouds, allowing multiple point cloud acquisitions from the same perspective. During the scanning and modeling process, only the target object is scanned into the point cloud data, and the rest of the welds are covered. The point cloud coordinate set is determined based on the relative position to determine whether it is the point cloud set of the same weld.

[0059] Step 2. Simplify the original point cloud data into an N*3 matrix, where N represents the number of point sets and 3 is the XYZ coordinate. De-noise and separate the point cloud data, and only retain the point cloud information on the surface of the weld. Figure 3 The schematic diagram shows the segmentation of the solder point cloud into a single solder point after a series of denoising, removal of redundant solder points, decentralization, and posture adjustment.

[0060] Step 3. Filter the point cloud information. The main purpose is to retain only the point cloud data of the surface shape of the weld and remove the rest of the point cloud. The point cloud to be removed is the noise caused by noise, outliers and the bracket part without melting area. The specific removal method is as follows:

[0061] Step 3.1. Traverse the point cloud and for a specific point X m , calculate its N neighbors to this point X m The distances L1, L2, ..., L n , use the distance data to fit the Gaussian distribution N(n k ,m k ), artificially preset the threshold P in advance, and set the deviation from the mean n k P*m k 2 Points outside of the above are removed as outlier noise points, that is, if a specific point L m ∈[nk-P*mk 2 ,nk+P*mk 2 ](m=1,2,...n) then point X is left m Otherwise, X m Eliminate, where n k is the mean of the fitted Gaussian distribution, m k is the variance of the Gaussian distribution, and P is the system preset threshold.

[0062] Step 3.2. Traverse the point cloud and for a specific point X m, according to its adjacent K neighbor points, the small plane S is fitted. The normal vector of plane S is considered to be the normal of this point. m Find the cosine value of the angle between the normal phase of and the normal phase of the preset plane S. If cos<(a,b,c), L∈[-0.1,0.1], then delete X m . Remove the uncovered molten area of ​​the original point cloud file and only keep the point cloud data of the weld surface.

[0063] Step 4. For the point cloud processed in step 3, which only contains the coordinate set of the solder joint surface data, the density clustering method is used to extract the point cloud data set of each individual solder joint to obtain P1, P2, ..., P n The point set is decentralized, that is, the following operations are performed: P i =P i -Means(P i )(i=1,2,...n), obtain the data of individual solder joints.

[0064] Step 5. Direct density clustering cannot effectively segment the point cloud of solder joints that are too close together. Calculating the projection area of ​​the solder joint surface after cutting can determine whether the segmentation fails. If it fails, principal component analysis is used to make the minimum rectangular bounding box of the solder joint and perform posture transformation, such as Figure 3 As shown in the figure, the long side of the minimum rectangular bounding box after the pose transformation is parallel to the X-axis and the surface normal of the solder joint is parallel to the Z-axis. At this time, the slicing method is used to further segment the solder joint:

[0065] If there are multiple welds stuck together in the normal weld point cloud, slices with a thickness of 0.05mm can be made every 0.05mm along the length and width of the minimum enclosing box. The number of point clouds in the slice and the preset value are used to distinguish the weld boundaries to achieve point cloud cutting. After this step, a separate weld point cloud array can be obtained.

[0066] Step 6. Perform spatial coordinate-independent and scale-independent eigenvalue calculations on the main point cloud and auxiliary point cloud of the same welding point, and replace this point cloud with the first N points with the largest eigenvalues ​​in the point cloud point set, and calculate the spatial secondary transformation matrix R of the auxiliary point cloud and the main point cloud.

[0067] According to the spatial secondary transformation matrix R, the auxiliary point cloud and the main point cloud are fused in turn to obtain the point set coordinates of the fused point clouds taken from different perspectives.

[0068] The fused point cloud is generated into a grayscale image. The grayscale image generation algorithm is as follows:

[0069] The Z coordinate of the point cloud is linearly stretched as the pixel grayscale value, and the XY coordinates of the point cloud are normalized and linearly stretched as the pixel position. The empty positions in the image are filled by interpolating the average value of adjacent points, and the filled grayscale image is Gaussian filtered. The default background color of the grayscale image is black, that is, the pixel value is 0.

[0070] In order to facilitate training and testing, the grayscale image dataset is divided into a training set and a test set at a ratio of 60% and 40%. In fact, other ratios can also be used, or a cross-test method can be used.

[0071] In the model training process, the K-fold cross-validation method is used to suppress overfitting, with the initial K=5, and the model time t and model accuracy a are recorded.

[0072] The grayscale image is placed in the ResNet network to extract the point cloud view features, and a fully connected layer and a Softmax layer are used at the end of the network to output the morphological recognition results. The convolutional neural network model is the ResNet-152, which includes the first convolutional layer group, the second convolutional layer group, the third convolutional layer group, a fully connected layer, and a Softmax layer. Figure 4 It is a point cloud fusion effect diagram of a single welding point modeled at different angles.

[0073] After the model training is completed, it can be put into practical application. For the target solder joint to be detected, the same 3D point cloud modeling method is used to obtain the main point cloud and auxiliary point cloud. The system is used to output the morphological characteristics of the solder joint in real time and give the probability of passing. The specific detection method process is shown in Figure 5 shown.

[0074] The present invention makes special optimizations in two aspects: point cloud imaging and machine learning models, to improve the accuracy and speed of the system's description of the weld shape. First, point clouds with complementary information are photographed from multiple angles, and more stable and noise-resistant point cloud data are obtained through the effect of point cloud fusion, thereby obtaining more accurate point cloud data. In addition, in order to increase the fusion speed, a lot of algorithm preprocessing designs are done. Finally, a residual convolutional neural network is designed according to the characteristics of the target to be detected, to improve the training and prediction accuracy and efficiency of the data set.

[0075] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A metal solder joint morphology detection method, characterized in that: The following steps are involved: Step 1: Build a multi-angle progressive sampling system for solder joints to support multi-angle collection of solder joint point clouds; The sample to be tested is made of metal, a welding point with reflective properties, or the surface of other objects; The surface of the solder joint or object is approximately rectangular in orthographic projection and has a high reflective property; The multi-angle progressive sampling system includes: a 3D sensor modeled by structured light and a bracket that can move the 3D sensor in a spherical space, the sample to be detected is located near the center of the spherical space, and the 3D sensor is placed in a preset position by moving the 3D sensor to obtain point clouds shot at various angles to avoid visual occlusion; The sampling process is as follows: first, the sensor is located directly above the sample to be detected, and the straight line connecting the sample to be detected and the lens is perpendicular to the lens, and the point cloud captured at this angle is used as the main point cloud; then, the algorithm is used to determine whether the main point cloud data is complete. If the main point cloud data cannot meet the requirements, the auxiliary point cloud is collected, that is, the straight line connecting the sample to be detected and the lens is deflected 30 degrees to any position and a circular ring perpendicular to the straight line is made on the spherical surface as the sensor movement trajectory, and the sample to be detected is photographed at a 30-degree angle every 90 degrees along the circular ring. The execution is selected 1-4 times according to the data density of the main point cloud. The point cloud generated in this step is used as the auxiliary point cloud, where multiple photos can be taken at the same angle to avoid noise interference. The entire sampling process should limit the modeling area of ​​the sensor according to the target to be detected, so that the sample to be detected can be completely modeled, and non-target objects should not be modeled as much as possible; The sample to be detected consists of a main point cloud file and an auxiliary point cloud file, and each file supports multiple repeated sampling; Step 2: Design a fast 3D point cloud fusion solution based on the collective features of the samples to be detected. The specific operations are as follows: Step 2.1, signal preprocessing, specifically separating the sample target to be detected to obtain preprocessed individual welding point cloud information; Step 2.2, correct the posture of the sample target to be detected. After this step, the same sample target to be detected taken from different perspectives will have similar coordinate values ​​in the coordinate system. The high approximation of the initial value is conducive to the speed improvement of the subsequent point cloud algorithm fusion; Step 2.3, extracting specific feature points of the sample target to be detected, using a smaller data set to represent the original point cloud set, and then quickly performing point cloud fusion, wherein the point cloud before fusion is the main point cloud modeled in step (1) or the main point cloud and the auxiliary point clouds modeled under various viewing angles, which have complementary visual information, and after fusion, more abundant and comprehensive point cloud data reflecting the morphology of the sample to be detected is obtained; Step 2.4, fusing point cloud images of the same target object from different perspectives; Step 3: Perform machine learning training and transform the fused point cloud data X∈R 3 , projected onto the XY plane to generate a grayscale image as the input of the convolutional neural network. The convolutional neural network finally uses a multi-layer perceptron for classification and judgment, and outputs the morphological description of the solder joint. Its mathematical expression is as follows: K(X),P(X)=class(img(X)); Among them, img(X) is the grayscale image conversion function, class() is the convolutional neural network, K(X) returns the category of the solder joint shape, P(X) is the probability of the solder joint being qualified, and if P(X)>0.5, the system considers it to be a qualified solder joint.

2. The metal solder joint morphology detection method according to claim 1, characterized in that: The repeated sampling described in step 1 means that in order to avoid noise from the sensor and interference from ambient light, both the main point cloud and the auxiliary point cloud can be sampled multiple times, and the sampling process chooses to add additional point light sources or line light sources, that is, to independently obtain multiple point cloud files at the same angle.

3. The metal solder joint morphology detection method according to claim 1, characterized in that: The step 2.1 specifically includes the following steps: Step 2.1.1, remove outlier noise. Ambient light, noise, electromagnetic wave disturbance and other interferences will cause outlier points in the original point cloud. A filtering algorithm based on local statistical information is used to remove outlier noise. The calculation steps are: Calculate the given point X m The K neighbors of X m The distances L1, L2, ..., L n And fit the Gaussian distribution N(n k ,m k 2 ), the deviation from the mean n k P*m k 2 Points outside of the above are removed as outlier noise points, as shown below: If L m ∈[n k -P*m k 2 , n k +P*m k 2 ](m=1,2,...n) then point X is left m Otherwise, X m Eliminate, where n k is the mean of the fitted Gaussian distribution, m k 2 is the variance of Gaussian distribution, P is the system preset threshold; Step 2.1.2, filter out the solder joint point cloud data. According to the characteristics of the target to be detected, the point cloud is mainly divided into two areas: the point cloud model data Z1 of the solder joint surface and the uncovered melt zone Z2. The point cloud plane of the Z2 area is perpendicular to the marker normal vector. The non-solder joint data in the point cloud model are filtered out according to the normal direction, as shown below: If cos<(a,b,c),L>∈[-0.1,0.1], then delete X n , where (a,b,c) is the point X n Normal vector, L is the normal vector of the metal surface preset by the system; Step 2.1.3, segment the solder joints. For the point cloud data processed in step 2.1.2, unsupervised density clustering is used to effectively implement preliminary segmentation and extraction of solder joints. Points in the same category are considered to be the same solder joint, and the minimum enclosing cuboid is calculated for adjustment. After that, the segmented solder joints are further calibrated using the slicing method based on the minimum spatial enclosing box of the point cloud. After this step, a separate solder joint point cloud array can be obtained.

4. The metal solder joint morphology detection method according to claim 1, characterized in that: The step 2.3 also includes: Step 2.3.1, project the main point cloud and auxiliary point cloud after posture correction along the Z axis to the XY plane, transform the projection surface into a grayscale image, and calculate the two-dimensional feature points of the grayscale image; Step 2.3.2: According to the correspondence between the two-dimensional grayscale image and the point cloud, the first N most characteristic points of the calculated two-dimensional feature points are marked in the point cloud as the input data for point cloud fusion.

5. The metal solder joint morphology detection method according to claim 1, characterized in that: During the sampling process in step 1, when the distance of the sample to be detected is within the appropriate scanning area, the sensor starts to establish the main point cloud directly above the sample, and then determines whether to continue scanning to establish the auxiliary point cloud based on the main point cloud density detection algorithm. When the main point cloud and the auxiliary point cloud are modeled, the current acquisition process is completed.

6. The metal solder joint morphology detection method according to claim 5, characterized in that: The density detection algorithm of the main point cloud is specifically implemented as follows: the main projection surface of the main point cloud is divided into 4 areas to calculate the point cloud density of each area, that is, to solve the ratio N / S of the number of points N and the area S. If the point cloud density is lower than the preset threshold, it is necessary to determine the angle and number of the auxiliary point cloud and shoot the auxiliary point cloud.

7. The metal solder joint morphology detection method according to claim 5, characterized in that: The sample to be tested is a copper-welded solder joint, in which the melted area is a rectangle facing the sensor, and the unmelted area is a strip-shaped bracket whose surface is perpendicular to the melted area. There are only solder joints and marking planes within the sensor modeling perspective, and no other objects are scanned into the point cloud file.

8. The metal solder joint morphology detection method according to claim 1, characterized in that: The morphological description of the sample to be detected in step 3 is to extract features based on the generated point cloud grayscale image using a convolutional neural network. The convolutional neural network model is ResNet-152, which includes a first convolutional layer group, a second convolutional layer group, a third convolutional layer group, 1 fully connected layer and 1 Softmax layer.

9. The metal solder joint morphology detection method according to claim 1, characterized in that: The solder joint morphology description output in step 3 is one of several categories of preset solder joint models.

10. A recognition system for post-weld quality inspection of welding spots, characterized in that: Using the metal solder joint morphology detection method as described in claims 1 to 9, the identification system for post-weld quality detection of solder joints matches the solder joint surface size, solder joint height difference information, and surface hole information calculated in step 3 with standard qualified solder joint related parameters, and provides the solder joint quality assessment results in real time.

Images