A method for extracting appearance defects of a shaped weld based on three-dimensional information

Abstract

The present application relates to the technical field, especially to a kind of based on three-dimensional information's forming weld appearance defect extraction method.The method specifically includes: the three-dimensional point cloud data of weld is collected;Three-dimensional point cloud data is preprocessed, including using statistical filtering method to remove outlier, and using the point closest to the voxel grid centroid in voxel grid is selected to carry out point cloud data downsampling;After three-dimensional point cloud data is preprocessed, through LMedS adaptive threshold segmentation method, the extraction of weld is carried out, in the weld point cloud that extraction is completed, the framework of region growing method is established, seed point is selected, whether the point in the neighborhood of seed point belongs to the same category according to normal angle threshold and curvature threshold is judged, the extraction of weld surface defect point cloud is realized.The method provided by the present application can more completely obtain three-dimensional defect geometric information, improve the overall operation efficiency of system, and be more conducive to the complete extraction of weld defect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field, and particularly relates to a forming weld appearance defect extraction method based on three-dimensional information. BACKGROUND

[0002] Welding technology is an important material structure processing technology, which is very important in mechanical processing in the fields of mechanical manufacturing, avionics, shipbuilding, automobile production and manufacturing, etc. Nowadays, more and more researches are devoted to the quality detection of welded workpieces after welding. With the wide application of visual detection in the field of industrial processing, the welding defect detection technology based on vision gradually replaces manual detection and has stable defect recognition effect in actual application. The appearance defect detection technology based on vision not only solves the shortcomings of manual detection, but also avoids the complex process of ultrasonic and infrared detection. Therefore, it has very important practical value to develop a visual detection system capable of accurately extracting weld defects in the field of automatic welding technology. At present, most methods adopt the method of installing an industrial camera at the end of a welding robot to shoot pictures of defect parts or building a line structure light image acquisition system to acquire the depth features of weld defects for defect extraction. However, both of the two methods are based on local information for calculation, and such processing method is difficult to directly make a whole three-dimensional description of the weld surface defects, so the reliability and efficiency of the two visual detection methods are low. In order to solve the problem that two-dimensional images and laser stripe images are difficult to directly make a whole three-dimensional description of the weld surface defects, some researches use active three-dimensional reconstruction methods, such as coded structured light, TOF time-of-flight method and triangulation method, to reconstruct the three-dimensional morphology of the weld, and extract the weld defects through the three-dimensional point cloud information after reconstruction. When extracting the weld, the weld is closely connected with the plate surface and has similar point cloud attributes, so it is difficult to separate the weld and the plate based on the edge and the attribute, and the model-based segmentation method and the region growing algorithm have better segmentation effect on the weld. In order to accurately extract the defects on the weld, a new method needs to be designed to realize the three-dimensional information acquisition of the weld, the accurate extraction of the weld and the accurate extraction of the defects. SUMMARY

[0003] (II) Technical problems to be solved

[0004] The present application provides a forming weld appearance defect extraction method based on three-dimensional information to overcome the defect that the defects on the weld surface cannot be accurately and wholely described in three dimensions in the prior art.

[0005] (II) Technical solutions

[0006] To solve the above problems, the present application provides a forming weld appearance defect extraction method based on three-dimensional information, which specifically comprises:

[0007] Step S1, collecting three-dimensional point cloud data of the weld by using a structured light three-dimensional camera;

[0008] Step S2, preprocessing the three-dimensional point cloud data, including removing outliers by using a statistical filtering method, and selecting the point closest to the voxel center in the voxel grid for point cloud data down-sampling;

[0009] Step S3, extracting the weld by using an LMedS adaptive threshold segmentation method on the preprocessed three-dimensional point cloud data, specifically including: calculating the distance of the point to the fitting plane, using the least squares median to estimate the plane, classifying the data points into weld point cloud and non-weld point cloud, and on this basis, using the point cloud normal vector difference to calculate the secondary segmentation threshold to extract the weld again, and finally obtaining the weld point cloud;

[0010] Step S4, establishing a region growing method framework on the extracted weld point cloud, selecting a seed point, judging whether the points in the neighborhood of the seed point belong to the same category according to the normal angle threshold and the curvature threshold, and realizing the extraction of the weld surface defect point cloud.

[0011] Preferably, the structured light three-dimensional camera is installed at the end of the industrial robot.

[0012] Preferably, the preprocessing of the three-dimensional point cloud data includes: calculating the distance of all points in the voxel grid to the voxel center, and keeping the point with the smallest distance to represent all points in the voxel grid.

[0013] Preferably, in step S3, the LMedS adaptive threshold segmentation method specifically includes: determining the number of iterations K after giving the probability P and the minimum sample number n.

[0014] Randomly selecting a subset from the point cloud data set to calculate the plane model parameters;

[0015] Calculating the distance of the sample point to the plane model as the residual;

[0016] Selecting the sample set with the smallest square median residual to fit the segmentation plane and calculate the threshold.

[0017] Preferably, the step of extracting the defect point cloud by using the region growing method specifically includes: calculating the normal vector and curvature of the weld point cloud as the initial label, and selecting the point with the minimum curvature as the initial seed point.

[0018] Judging whether the points belong to the same category according to the normal angle threshold and the curvature threshold of the seed point and the points in the neighborhood; repeating the growth until all points are traversed.

[0019] Preferably, the method is realized by an industrial robot control system.

[0020] (Three) beneficial effects

[0021] The present invention provides a method for extracting appearance defects of formed welds based on three-dimensional information, which has the following beneficial effects:

[0022] (1) Collect three-dimensional point cloud data of weld seam and combine it with overall three-dimensional information to obtain more complete three-dimensional defect geometric information;

[0023] (2) Downsampling and filtering of the three-dimensional point cloud data of the weld seam improves the overall computing efficiency of the system;

[0024] (3) By combining the adaptive threshold weld seam extraction method, the segmentation error caused by threshold changes due to object changes is overcome, and the overall extraction accuracy is improved.

[0025] (4) Using the region growth algorithm based on the weld point cloud is more conducive to the complete extraction of weld defects. Attached Figure Description

[0026] Figure 1 This is a flowchart of the method for extracting appearance defects of formed welds based on three-dimensional information according to an embodiment of the present invention;

[0027] Figure 2 This is a diagram illustrating the execution effect of an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] like Figures 1-2 As shown, the present invention provides a method for extracting appearance defects of formed welds based on three-dimensional information, specifically including:

[0030] Step S1: Use a surface structured light 3D camera to acquire 3D point cloud data of the weld seam;

[0031] In this embodiment of the invention, the surface structured light 3D camera is mounted on the end effector of an industrial robot.

[0032] Step S2: Preprocess the three-dimensional point cloud data, including using statistical filtering to remove outliers and using the point closest to the centroid of the voxel grid to downsample the point cloud data.

[0033] The preprocessing of the three-dimensional point cloud data includes: calculating the distance from all points in the voxel grid to the centroid of the voxel grid, and retaining the point with the smallest distance to represent all points in that voxel grid.

[0034] Step S3, the LMedS adaptive threshold segmentation method is used to extract the weld bead from the pre-processed three-dimensional point cloud data, specifically including: calculating the distance of points to the fitting plane, using least squares to estimate the plane, classifying the data points into weld bead point cloud and non-weld bead point cloud, and on this basis, using point cloud normal vector difference to calculate the secondary segmentation threshold to extract the weld bead again, and finally obtaining the weld bead point cloud;

[0035] The LMedS adaptive threshold segmentation method specifically includes: determining the number of iterations K after giving the probability P and the minimum sample number n;

[0036] A subset is randomly selected from the point cloud data set to calculate the plane model parameters;

[0037] The distance of the sample point to the plane model is the residual;

[0038] The sample set with the smallest square median residual is selected to fit the segmentation plane and calculate the threshold.

[0039] Step S4, in the extracted weld bead point cloud, a region growing method framework is established, a seed point is selected, whether the points in the neighborhood of the seed point belong to the same category is judged according to the normal angle threshold and the curvature threshold, and the extraction of the weld bead surface defect point cloud is realized.

[0040] The steps of the region growing method for extracting defect point cloud specifically include: calculating the normal vector and curvature of the weld bead point cloud as the initial label, and selecting the point with the minimum curvature as the initial seed point;

[0041] Whether the points belong to the same category is judged according to the normal angle threshold and the curvature threshold of the seed point and the points in the neighborhood; and the growth is repeated until all points are traversed.

[0042] It should be particularly noted that the present embodiment also includes welding quality evaluation on the weld appearance defect results extracted based on the above method.

[0043] The three-dimensional information-based formed weld appearance defect extraction method will be described in detail below

[0044] The statistical filtering method is used to remove outliers in the original point cloud data, and the distance d of each point to any point is calculated t The average value μ and the standard deviation σ of the distance between each point and any point are calculated, and the standard deviation multiple is std. When the average distance of the k1 neighboring points of a point is within the standard range (μ-σ·std, μ+σ·std), the point is retained, and if it is not within the range, it is defined as an outlier and deleted.

[0045] The point cloud data of the welding seam is down-sampled to reduce the number of points and improve the operation speed. The maximum and minimum coordinate values of the point cloud data are obtained by traversing the point cloud: x max ,y max ,z max and x min ,y min ,z min The point cloud is rasterized within such a coordinate range and divided into a plurality of voxel grids of the same size. The side length of the voxel grid can be set as d, and a point cloud can be divided into m*n*l voxel grids, and the relationship is as follows:

[0046]

[0047] The index value of the point can be recorded in a specific voxel through the relationship between the voxel side length and the coordinates of the point. Therefore, for each voxel grid, the centroid C of the point set in the voxel grid is calculated, where k is the number of points in the voxel grid, and p i is the i-th point in the point set, and the following equation is obtained:

[0048]

[0049] The Euclidean distance p of the point in the point set to the centroid C is calculated, and the calculation formula is as follows:

[0050]

[0051] Where (x c ,y c ,z c ) and (x i ,y i ,z i ) represent the coordinates of the centroid point and the point in the point set, respectively.

[0052] The closest point to the centroid C is found to replace all points in the voxel, achieving the purpose of down-sampling and preserving the original three-dimensional features of the welding seam. After the above processing, the point cloud data of the welding seam is greatly simplified, and the subsequent operation is improved in efficiency and accuracy.

[0053] Step S3, using a welding seam point cloud LMedS adaptive threshold segmentation method using differential normal features to segment the welding seam point cloud, for the pre-processed welding seam three-dimensional point cloud data, given the probability P, the sample pollution rate ε and the minimum sample number n, the LMedS minimum iteration number K is determined by the following formula (where w=1-ε):

[0054]

[0055] Suppose there is a plane Ax+By+Cz=D, then the point p(xi y i , z i The distance from the plane is d. i for:

[0056]

[0057] At this point, the residual r in the LMDS algorithm i For the desired d i The median of the squared residuals between all subsets of the sample set data and other samples is Med, which is d. i The median of squared residuals is calculated by randomly selecting Q subsets of samples from the original sample set. The model parameters and squared residuals for each subset are calculated, and the median of the squared residuals is found. Finally, the smallest median of squared residuals from the Q subsets is selected, and the weight corresponding to each point is calculated using the following formula:

[0058]

[0059] Where σ is the estimated standard deviation; n is the total number of samples in the selected area; p is the minimum number of samples required to calculate the model parameters; and r is the minimum number of samples required to calculate the model parameters. i Let p(x) be the point i y i , z i The distance from the plane is d. i med indicates that the median of the sample is taken. Outside points and inside points are distinguished by the following formula:

[0060]

[0061] Among them, w i Let represent the weight of the i-th sample, where a weight of 0 is considered an outlier and a weight of 1 is considered an inlier.

[0062] Find N high points and N low points from the initial sample set, and calculate their average, where h i ,h j Let represent the height values ​​of the high point and the low point, respectively. The formula is as follows:

[0063]

[0064] With the above parameters obtained, the distance from the point in the dataset to the plane model is calculated, and the threshold r1 calculated by the above formula is used to determine whether the point belongs to the weld bead point, thereby realizing the extraction of the weld bead D1 and the plane P1.

[0065] A differential normal segmentation is introduced on a single segmentation plane. This algorithm estimates the point cloud normal vector at two different scales for the same point using different support radii, and defines the feature of the DoN by the difference of the unit normal vectors. For any point P in the point cloud... iThe k nearest neighbors are calculated by K-neighborhood algorithm, and a local plane is fitted according to these points, and the calculation formula is:

[0066]

[0067] Where n is the normal vector of the local plane P; d is the distance from the local plane P to the coordinate origin. The local plane P passes through the centroid of the k nearest neighbors, and the normal vector satisfies ||n||2=1, so the above problem can be converted into solving the eigenvalue of the covariance matrix, and the eigenvector corresponding to the smallest eigenvalue of the covariance matrix A is the normal vector of the plane P. The calculation formula of the covariance matrix A is:

[0068]

[0069] Where k is the sampling point P i The number of local neighbors, is the centroid of the k neighborhood. The eigenvalue and eigenvector of this matrix are calculated:

[0070] A·v j =λ j ·v j

[0071] Where λ j is the jth eigenvalue of the covariance matrix A, v j is the corresponding eigenvector, which is the normal vector at point P i . For any point in the point cloud, given two different support radii r i , r s , the two different unit normal vectors n(p,r s ), n(p,r l ) at the point are calculated, and the definition of DoN is as follows:

[0072]

[0073] For all points after the first segmentation, the unit normal vector difference set {N d} is calculated according to the above formula, the threshold value D of the DoN eigenvector is set, the point cloud normal vector difference set {N d} is traversed, the vector domain is filtered, and the point cloud sample set {N r} with larger curvature at the root of the weld is obtained, and the following formula is executed to obtain the secondary segmentation threshold, where M is the number of high and low points, h i , h j are the height values of the high and low points:

[0074]

[0075] The normal vector n(p,rs ), n(p,r l The angle is determined, and the LMDS plane fitting algorithm is executed again. The value r2 obtained by the above formula is used as the secondary segmentation threshold to obtain the weld bead point cloud part D2. The point clouds of the same parts of D1 and D2 are spliced ​​together to obtain the complete weld bead point cloud and the plate surface point cloud. In this way, the initial extraction and re-extraction of the weld bead improve the extraction accuracy of the weld bead.

[0076] Step S4: Segment and extract surface defects of the weld seam based on the region growing method. Specifically, construct a kd-tree from the obtained weld point cloud, calculate the point cloud normal vector and initialize the label value based on curvature. Select the point with the minimum curvature as the seed point, add it to the seed point queue, and label it. cur Set to 0. Perform a nearest neighbor search on the points in the queue. Compare the angle between the normal vectors of the point and its nearest neighbors; if the angle is less than the threshold theta... th If the nearest neighbor is assigned to the current category, then the nearest neighbor is classified into that category. The curvature of the nearest neighbors is compared; if it is less than the curvature threshold c... th If a point is found, it is used as the seed point, and the search continues until all points have been traversed.

[0077] Specifically, the process is as follows: Select a point in the point cloud as a seed point and classify it into class C1. Search for its nearest neighbor using a kd-tree. If the Euclidean distance between the searched point and the seed point is less than a preset distance threshold, then classify that point into class C1 as well and update it as the current seed point. Continue traversing until no point meets the distance threshold condition. Find unclassified points and use them as new seed points. Repeat the first two steps until all points have been classified. Perform Euclidean clustering on the grown point cloud and remove classes with fewer than a certain threshold number of points to obtain the segmented weld point cloud. Thus, using the 3D point cloud, the appearance defect point cloud on the weld point cloud is completely extracted.

[0078] After processing the raw weld point cloud data acquired by the 3D point cloud acquisition system through the above steps, the weld bead point cloud data can be accurately segmented, and the point cloud data of weld surface defects can be further precisely segmented based on the weld bead point cloud data. Taking weld bead defects as an example, the implementation effect diagram of this invention is attached. Figure 2 As shown.

[0079] The application is based on a plane structured light 3D camera and an industrial robot, and actually contains a plane structured light 3D camera, an industrial robot, a robot control cabinet and a display screen in the working process. When the system works, the 3D camera collects three-dimensional point cloud data of a weld, the three-dimensional point cloud data obtained is preprocessed to meet the three-dimensional processing requirements, the accurate segmentation of a weld bead of the weld is carried out on the basis of the preprocessed three-dimensional point cloud data, and the weld defects are extracted on the segmented weld bead point cloud. The three-dimensional point cloud data preprocessing method is used to collect the original data of the system after the system is built. First, the statistical filtering method is used to remove the extracorporeal outliers in the point cloud data. The voxel grid of the point cloud data is established, the centroid of each voxel grid is calculated, the point closest to the centroid point is used to represent the points in the whole voxel grid, and the point cloud data is simplified.

[0080] The accurate segmentation method of the weld bead adopts a LMedS adaptive threshold segmentation method of the weld point cloud using differential normal features, so that the weld bead point cloud part is adaptively segmented and extracted.

[0081] The defect extraction method on the weld bead establishes a region growing framework, and uses the normal angle difference and local curvature difference of the defect region and the weld bead region points to extract the surface defects on the weld bead.

[0082] The above embodiments are only used to illustrate the application, and are not limited to the application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the application, so all equivalent technical solutions also belong to the scope of the application, and the patent protection scope of the application should be defined by the claims.

Claims

1. A method for extracting a defect of a formed weld appearance based on three-dimensional information, characterized by, The method comprises the following steps: Step S1, acquiring three-dimensional point cloud data of a weld seam by using a structured light three-dimensional camera; Step S2, preprocessing the three-dimensional point cloud data, including removing outliers by using a statistical filtering method, and selecting points closest to the center of a voxel grid in the voxel grid to reduce the sampling of the point cloud data; Step S3, extracting a weld bead from the preprocessed three-dimensional point cloud data by using an LMedS adaptive threshold segmentation method, specifically including: calculating the distance of a point to a fitting plane, estimating a plane by using least squares median, classifying data points into weld bead point cloud and non-weld bead point cloud, and on this basis, calculating a secondary segmentation threshold by using a point cloud normal vector difference to extract a weld bead again, and finally obtaining a weld bead point cloud; wherein the LMedS adaptive threshold segmentation method specifically includes: determining the number of iterations K after a given probability P and a minimum sample number n are given; calculating a plane model parameter from a randomly selected subset of point cloud data; calculating the distance of a sample point to a plane model, which is a residual error; selecting a sample set with the smallest square median residual error to fit a segmentation plane and calculate a threshold value; Step S4, establishing a region growing method framework on the extracted weld bead point cloud, selecting a seed point, judging whether points in the neighborhood of the seed point belong to the same category according to a normal angle threshold and a curvature threshold, and realizing extraction of a weld bead surface defect point cloud.

2. The method for extracting a shaped weld appearance defect based on three-dimensional information according to claim 1, wherein The structured light three-dimensional camera is installed at the end of an industrial robot.

3. The method for extracting a shaped weld appearance defect based on three-dimensional information according to Claim 1, wherein The preprocessing of the three-dimensional point cloud data includes: calculating the distance of all points in a voxel grid to the center of the voxel grid, and retaining the point with the smallest distance to represent all points in the voxel grid.

4. The method for extracting a shaped weld appearance defect based on three-dimensional information according to Claim 1, wherein The steps of the region growing method for extracting a defect point cloud specifically include: calculating the normal vector and curvature of the weld bead point cloud as an initial label, and selecting a point with the smallest curvature as an initial seed point; judging whether the seed point and the points in the neighborhood belong to the same category according to a normal angle threshold and a curvature threshold; and repeating the growth until all points are traversed.

5. The three-dimensional information-based shaped weld appearance defect extraction method according to claim 1, wherein The method is realized by an industrial robot control system.

Images