Weld grinding feature extraction method, weld detection and grinding method

Through infrared structured light 3D camera and point cloud processing technology, the direction vector and slice characteristics of the weld are extracted, which solves the problem of low three-dimensional reconstruction accuracy in weld grinding, and realizes efficient weld feature extraction and automated grinding detection.

CN116823735BActive Publication Date: 2024-11-01SHANGHAI SELFWELD ROBOT CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310645680.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-06-01
Publication Date
2024-11-01
Estimated Expiration
2043-06-01

AI Technical Summary

Technical Problem

The three-dimensional reconstruction accuracy in existing weld grinding is low, and the characteristics cannot be accurately extracted, which affects the efficiency and accuracy of automated grinding.

Method used

The infrared structured light 3D camera is used to obtain the workpiece point cloud information. Through European clustering, RANSAC algorithm and point cloud decentralization processing, the direction vector and slice characteristics of the weld are extracted to achieve efficient weld feature extraction.

Benefits of technology

It improves the accuracy and speed of weld feature extraction, supports efficient robotic weld automatic grinding and grinding molding inspection, and shortens inspection time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116823735B_ABST
    Figure CN116823735B_ABST
Patent Text Reader

Abstract

The present invention relates to a weld grinding feature extraction method, a weld detection and grinding method, comprising: step one, obtaining point cloud information of a workpiece by shooting with a 3D camera; step two, extracting weld grinding features by a weld grinding feature extraction method; step three, generating a weld grinding path according to the weld grinding features, and a robot grinding the workpiece according to the grinding path; step four, repeating steps one and two, obtaining grinding weld excess height features, and performing weld forming detection: if the detection passes, the grinding operation process is completed, and if the detection fails, returning to step three to continue grinding. It can realize efficient and accurate weld grinding feature extraction, can greatly improve the accuracy of weld feature extraction, shorten the weld feature extraction time, and can be used to realize efficient robot weld automatic grinding and efficient grinding weld forming detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of welding technology, and in particular to a weld grinding feature extraction method, a weld detection and grinding method. Background Art

[0002] As the manufacturing industry gradually develops towards automation and intelligence, industrial robots are playing an important role in more and more fields. Industrial robots can replace manual labor in material processing projects such as welding, grinding, and casting. They have replaced traditional manual production in some high-noise and high-pollution environments, greatly improving processing efficiency and reducing labor costs. The intelligence of industrial robots has been greatly improved by adding visual sensors, sound sensors and other methods to the body of industrial robots for multi-information fusion, allowing robots to handle various complex working conditions like humans.

[0003] Traditional manual grinding not only has low grinding efficiency and unstable grinding quality, but also causes great harm to workers' physical and mental health. Therefore, the realization of automated grinding is of great significance to replace manual labor. At present, the most widely used in the field of automated grinding is still the multi-axis CNC machine tool (CNC), but the cost of multi-axis CNC machine tools is high, and the scalability and flexibility are poor, so they cannot be used in the grinding of large welds. Compared with multi-axis CNC machine tools, industrial robots have a wider and scalable operating space and a lower price. They are more promising in the field of grinding, especially in the grinding of large workpieces. The advantages of industrial robots are obvious. Robotic processing operations can optimize operating parameters in real time based on process knowledge models and multi-sensor feedback information, thereby realizing active control of equipment during the process.

[0004] The shaping of the weld seam usually has certain requirements for the weld seam excess height. The traditional method of detecting the shaping of the weld seam usually adopts the manual measurement method, which is not only time-consuming but also has low detection accuracy. There are also methods of using profilometers to detect the surface of the weld seam, but they can only be applied to smaller workpieces. As one of the important sensors for realizing robot automated grinding, visual sensing can realize three-dimensional reconstruction of the grinded workpiece through 3D cameras. The key feature points of the weld seam can be extracted not only for subsequent grinding path planning, but also for the detection of the shaping of the weld seam and to determine whether the shaping of the weld seam is qualified. The commonly used visual sensor is the line laser 3D camera. Although the line laser 3D camera has high reconstruction accuracy, its three-dimensional reconstruction efficiency is low and the three-dimensional reconstruction speed is slow, which is not conducive to the efficient implementation of the automated grinding process. Summary of the invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the technical defects of low accuracy of three-dimensional reconstruction and inability to accurately extract features in weld grinding in the prior art.

[0006] In order to solve the above technical problems, the present invention provides a weld grinding feature extraction method, comprising:

[0007] S1. Obtaining point cloud information of a workpiece to be tested, wherein the workpiece to be tested is colored by a colored penetrant flaw detection agent, and the point cloud information is collected by an infrared structured light 3D camera;

[0008] S2, processing the original point cloud information of the workpiece to be measured to obtain the weld grinding features, including the following steps:

[0009] S21. Process the point cloud dataset P by Euclidean clustering to segment the target welding plate point cloud dataset P B ;

[0010] S22, based on the target welding plate point cloud dataset P B , the welding plate plane is fitted by RANSAC algorithm, and the point cloud data set P on the welding plate plane is obtained S ;

[0011] S23, the target welding plate point cloud dataset P B The point cloud dataset P on the weld plate plane S Make a difference and obtain the weld point cloud dataset P W ;

[0012] S24, weld point cloud dataset P W Perform Euclidean clustering to segment the weld point cloud dataset P W Divide into n independent weld point cloud datasets P Wi ;

[0013] S25. Extracting weld point cloud dataset P Wi The direction vector of the midpoint cloud;

[0014] S26, setting the lower threshold D1 and the upper threshold D2 of the weld direction vector, extracting the weld point cloud data set P Wi Weld bead direction

[0015] S27, the direction of the weld Transform to Direction, get the rotation matrix R i , and multiply the weld point cloud dataset by the rotation matrix R i , obtain the rotated weld point cloud dataset;

[0016] S28, slicing the rotated weld point cloud data set along the X-axis direction to obtain multi-pass weld slice point cloud data P sliceij ;

[0017] S29, extract weld slice point cloud data Psliceij The highest point of the point cloud along the Z-axis, the height of the sliced ​​weld, and the width of the sliced ​​weld are used to complete the extraction of weld features.

[0018] Preferably, before S21, the method further includes:

[0019] The point cloud information is subjected to voxel downsampling processing to obtain a preprocessed point cloud data set P.

[0020] Preferably, the S21 includes:

[0021] Determine the query point P i , set the distance threshold r, and the calculation formula for the distance d between any two points in the point cloud data set is:

[0022]

[0023] Among them, p i With p j Represents two points in the point cloud, p ik With p jk Represents any point in the neighborhood of two points;

[0024] Find the distance from the query point P through KD-tree i The nearest n neighbor points p j , where j = 1, 2, ..., n, calculate the n nearest neighbor points to the query point P according to the above formula i The Euclidean distance d j ;

[0025] Compare distance d j And the distance threshold r, the points less than the distance threshold r are classified into class M, until the number of points in class M no longer increases, the segmentation is completed.

[0026] Preferably, the S24 includes:

[0027] Weld point cloud dataset P W Perform Euclidean clustering segmentation and set the minimum value κ of the number of point clouds contained in the cluster. After Euclidean clustering segmentation, the point cloud dataset P Wi When the number is greater than or equal to κ, the classified point cloud dataset is retained, otherwise the classified point cloud dataset is deleted, where i=1, 2, 3…n, and n is a natural number greater than 3.

[0028] Preferably, the S25 includes:

[0029] S251, weld point cloud dataset P Wi There are m point cloud data in total. Select the weld point cloud data set P Wi A point P Wijand the q point cloud data around it (j=1, 2, 3…m);

[0030] S252, calculate the point cloud centroid C of q+1 point cloud data Wij , the formula is as follows:

[0031]

[0032] S253, perform point cloud de-centroiding processing, and subtract the point cloud centroid C from the q+1 point cloud data in sequence Wij , get the point cloud data position matrix deM after removing the centroid ij ;

[0033] S254, construct the covariance matrix: Cov ij =deM ij T *deM ij ;

[0034] S255, covariance matrix Cov ij Perform singular value decomposition, the singular vector corresponding to the minimum singular value (u ij ,v ij ,w ij ) is point P Wij The normal vector of

[0035] S256, determine whether j is equal to m, if j is less than m, return to S251, j=j+1;

[0036] S257. When j is equal to m, obtain the weld point cloud data set P Wi The direction vectors of all point clouds in the cloud.

[0037] Preferably, the S26 includes:

[0038] S261, let the fitted weld plate plane expression be A1x+B1y+C1z=D1, record the weld point cloud data set P Wi There are m point cloud data in total, for the weld point cloud data set P Wi Point P in Wij The cosine value of the angle between its direction vector and the welding plate plane is calculated by the following formula: ij :

[0039] Where, j = 1, 2, 3…m;

[0040] S262. Create array D Wi To store the weld direction point cloud data of the i-th weld, when D1≤cosΦ ij ≤D2, then the point P Wij Put it into array D Wi, otherwise continue to the next step;

[0041] S263, determine whether j is equal to m, if j is less than m, return to S261, j = j + 1;

[0042] S264. When j is equal to m, obtain the weld direction point cloud data set D Wi ;

[0043] S265, calculate the weld direction point cloud dataset D Wi The point cloud centroid C Wi , the coordinates are marked as (a i ,b i ,c i );

[0044] S266, perform point cloud de-centroiding processing, and convert D Wi The point cloud data in the subtraction of the point cloud centroid C Wi , get the point cloud data position matrix deM after removing the centroid i ;

[0045] S267, position matrix deM i Perform singular value decomposition to obtain the matrix U i , S i and V i ;

[0046] S268. The eigenvector corresponding to the maximum eigenvalue is the direction vector of the fitting line, and the centroid of the point cloud is a point on the fitting line, that is:

[0047]

[0048] Where (x i ,y i ,z i ) is the point on the straight line of the fitting space, (l i , m i , n i ) represents the weld direction of the fitting line

[0049] Preferably, the S27 includes:

[0050] Direct the weld bead Convert to unit vector:

[0051]

[0052] remember The direction unit vector is

[0053] when Then q i=(0,0,0,0), otherwise

[0054]

[0055] In the formula, sum() means summing the three directions of the vector;

[0056] By quaternion q i Construct the rotation matrix R i ;

[0057] Using the rotation matrix R i The weld point cloud dataset P Wi The coordinates are transformed to the direction where the weld is parallel to the X-axis, that is, the rotated point cloud dataset P Wi =R i *P Wi T .

[0058] Preferably, the S28 includes:

[0059] S281, set the interval d and the width f of the sliced ​​point cloud, select the point cloud with a width of f for slicing at each interval d, and record the point cloud P Wi The range is X mini To X maxi , where i represents the i-th weld, j represents the j-th point cloud slice, and j starts from 0;

[0060] S282, let j = j + 1, determine X mini +j*f+(j-1)*d is greater than X maxi , when X mini +j*f+(j-1)*d is less than or equal to X maxi , then select the coordinate range X mini +(j-1)*f+(j-1)*d~X mini The points within +j*f+(j-1)*d are taken as the slice point cloud data P sliceij ;

[0061] When X mini +j*f+(j-1)*dX maxi Greater than X maxi , then end this step.

[0062] Preferably, the S29 includes:

[0063] Note: sliceij The three-dimensional coordinates of sliceij ,y sliceij , z sliceij );

[0064] The highest point H sliceij=maxz sliceij , Slice weld height h sliceij =maxz sliceij -minz sliceij , Slice weld width W sliceij =maxy sliceij -miny sliceij , the weld foot position of the sliced ​​weld is maxy sliceij and miny sliceij .

[0065] The present invention discloses a weld detection and grinding method based on a 3D vision sensor, comprising the following steps:

[0066] Step 1: Obtain the point cloud information of the workpiece by shooting with a 3D camera;

[0067] Step 2: Based on the above weld grinding feature extraction method, extract the weld grinding features;

[0068] Step 3: Generate a weld grinding path according to the weld grinding characteristics, and the robot grinds the workpiece according to the grinding path;

[0069] Step 4: Repeat steps 1 and 2 to obtain the grinding weld excess height characteristics and perform weld forming detection: if the detection passes, the grinding process is completed; if the detection fails, return to step 3 to continue grinding.

[0070] The above technical solution of the present invention has the following advantages compared with the prior art:

[0071] 1. The present invention provides a robot weld grinding feature extraction method based on 3D visual sensing, which can realize efficient and accurate weld grinding feature extraction, can greatly improve the accuracy of weld feature extraction, shorten the weld feature extraction time, and can be used to realize efficient robot weld automatic grinding and efficient grinding weld forming detection.

[0072] 2. The three-dimensional reconstruction speed of the present invention is fast, and the three-dimensional model can be quickly processed to extract key structural features, and efficient robot weld automatic grinding and efficient grinding weld forming detection can be realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is a flow chart of the weld grinding feature extraction method of the present invention;

[0074] Figure 2 The figure is a flow chart of the weld inspection and grinding method based on 3D vision sensor. DETAILED DESCRIPTION

[0075] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0076] Reference Figure 1 As shown, the present invention discloses a weld grinding feature extraction method, comprising:

[0077] S1. Obtaining point cloud information of a workpiece to be tested, wherein the workpiece to be tested is colored by a colored penetrant flaw detection agent, and the point cloud information is collected by an infrared structured light 3D camera.

[0078] Specifically, commonly used 3D cameras include line laser 3D cameras and structured light 3D cameras. Line laser 3D cameras need to follow the movement of the robot to take multiple line laser photos and complete three-dimensional reconstruction through line laser extraction algorithms, which is less efficient; structured light 3D cameras can directly obtain the point cloud information of the photographed workpiece, but the quality of the point cloud information obtained when photographing reflective workpieces is poor. The present invention adopts an infrared structured light 3D camera and proposes a metal surface treatment method, which can effectively avoid the problem of poor imaging effect of reflective metal surfaces photographed by infrared structured light 3D cameras, and realize efficient acquisition of workpiece point cloud information. The specific implementation method is to spray DPT-5 colored penetrant flaw detection agent on the surface of the workpiece, and then use an infrared structured light 3D camera to shoot, adjust a higher exposure time, and obtain high-quality workpiece point cloud information. DPT-5 colored penetrant flaw detection agent is an efficient and convenient method commonly used in non-destructive testing. It is easy to clean and can be cleaned with a cleaning agent or water (water pressure ≤1.5kg / cm 2 ) can clean the penetrant on the surface of the workpiece.

[0079] Before collecting photos, the robot TCP calibration and the 3D camera's internal reference matrix and hand-eye conversion matrix calibration are first performed. The robot TCP calibration point is selected as the center point of the EHA floating grinding head. The calibration is completed by changing the robot's six-axis posture and taking multiple photos to obtain the 3D camera's internal reference matrix. and the hand-eye transformation matrix Next, collect photos.

[0080] The following process is followed during the photo collection process:

[0081] ① Set the photo location and number of photos n in advance;

[0082] ②The robot moves to the preset point to take pictures;

[0083] ③ Check whether the number of photos is equal to n. If it is less than n, return to step ② to continue shooting;

[0084] ④ Send the collected n point cloud photos to the industrial computer;

[0085] ⑤ The industrial computer performs point cloud stitching on the collected point cloud photos to obtain the overall point cloud information of the workpiece;

[0086] ⑥ Obtain the position coordinate P in the robot coordinate system through the following formula R :

[0087]

[0088] Among them, P L is the coordinate value of the point cloud information in the image coordinate system, is the transformation matrix from the image coordinate system to the camera coordinate system, that is, the camera intrinsic parameter, is the transformation matrix from the camera coordinate system to the tool coordinate system, that is, the camera hand-eye transformation matrix, is the transformation matrix from the tool coordinate system to the robot coordinate system, P R It is the three-dimensional coordinate value of the point cloud information in the robot coordinate system.

[0089] S2, processing the original point cloud information of the workpiece to be measured to obtain the weld grinding features, including the following steps:

[0090] S21. Process the point cloud dataset P by Euclidean clustering to segment the target welding plate point cloud dataset P B .

[0091] Since the information captured by the 3D camera contains not only the target welding plate point cloud information, but also the surrounding scene point cloud information, it is necessary to remove unnecessary point clouds in order to extract the point cloud information of the welding plate itself. The essence of Euclidean clustering is to classify points with close distances into one category. For n points in the point cloud C, the Euclidean distance is defined as the degree of closeness between two points, and the distance between neighboring points is used as the standard to achieve point cloud clustering segmentation. The specific segmentation process is:

[0092] S211, for the point cloud data set P after voxel downsampling preprocessing, determine the query point P i , set the distance threshold r, and the calculation formula for the distance d between any two points in the point cloud data set is:

[0093]

[0094] Among them, p i With p j Represents two points in the point cloud, p ik With p jk Represents any point in the neighborhood of two points;

[0095] S212, find the distance from the query point P through KD-tree i The nearest n neighbor points p j(j=1, 2, ..., n), calculate the n nearest neighbor points to the query point P according to the above formula i The Euclidean distance d j ;

[0096] S213, compare distance d j And the distance threshold r, the points less than the distance threshold r are classified into class M, until the number of points in class M no longer increases, the segmentation is completed.

[0097] After completing the Euclidean clustering, due to the reasonable selection of the previous photo positions, the group with the largest number of clustered point clouds is selected as the overall point cloud P of the welded plate. B , to realize the segmentation of the overall point cloud information of the welding plate.

[0098] S22, based on the target welding plate point cloud dataset P B , the welding plate plane is fitted by RANSAC algorithm, and the point cloud data set P on the welding plate plane is obtained S .

[0099] The RANSAC algorithm, or random consistency algorithm, is an effective robust estimation method that can still obtain relatively ideal fitting results when the point cloud data has a large error value. The RANSAC algorithm process is as follows:

[0100] 1) Calculation formula P = 1-(1-(1-ε) m ) M , where P represents the probability of obtaining at least one benign subset, ε represents the data error rate, m is the minimum amount of data required to calculate the model parameters, and M is the minimum number of basic subset samples. Calculate the minimum number of samples M based on the given ε, P, and m;

[0101] 2) The minimum number of iterations M is calculated to randomly select m points from the point cloud data, and use the formula ax+by+cz=d and the parameter a 2 +b 2 +c 2 =1 to calculate the initial values ​​of the plane model parameters;

[0102] 3) Set the threshold δ0, and for all points in the data set, use the formula d according to the calculated initial values ​​of the model parameters i =|a xi +b yi +c zi -d|Calculate its tolerance value δ; if it is within the threshold range δ0, it is classified as an in-point, otherwise it is classified as an out-point;

[0103] 4) Repeat steps 2) and 3) M times, count the number of in-place points after each classification, select the group of points with the largest number of in-place points, perform eigenvalue fitting on the largest number of in-place points to obtain the final plane model parameters;

[0104] 5) Calculate the target welding plate point cloud dataset P B The point cloud dataset P on the fitting plane S .

[0105] After fitting the welding plate plane through the RANSAC algorithm, all point cloud data on the welding plate plane can be obtained.

[0106] S23, the target welding plate point cloud dataset P B The point cloud dataset P on the weld plate plane S Make a difference and obtain the weld point cloud dataset P W ;

[0107] S24, weld point cloud dataset P W Perform Euclidean clustering to segment the weld point cloud dataset P W Divide into n independent weld point cloud datasets P Wi ,include:

[0108] Weld point cloud dataset P W Perform Euclidean clustering segmentation and set the minimum value κ of the number of point clouds contained in the cluster. After Euclidean clustering segmentation, the point cloud dataset P Wi When the number is greater than or equal to κ, the classified point cloud dataset is retained, otherwise the classified point cloud dataset is deleted, where i=1, 2, 3…n, and n is a natural number greater than 3.

[0109] By selecting an appropriate parameter κ, the weld point cloud dataset P W Divide into n independent weld point cloud datasets P Wi ;

[0110] S25. Extracting weld point cloud dataset P Wi Direction vector of the midpoint cloud, including:

[0111] S251, weld point cloud dataset P Wi There are m point cloud data in total. Select the weld point cloud data set P Wi A point P Wij and the q point cloud data around it (j=1, 2, 3…m);

[0112] S252, calculate the point cloud centroid C of q+1 point cloud data Wij , the formula is as follows:

[0113]

[0114] S253, perform point cloud de-centroiding processing, and subtract the point cloud centroid C from the q+1 point cloud data in sequence Wij , get the point cloud data position matrix deM after removing the centroidij ;

[0115] S254, construct the covariance matrix: Cov ij =deM ij T *deM ij ;

[0116] S255, covariance matrix Cov ij Perform singular value decomposition, the singular vector corresponding to the minimum singular value (u ij ,v ij ,w ij ) is point P Wij The normal vector of

[0117] S256, determine whether j is equal to m, if j is less than m, return to S251, j=j+1;

[0118] S257. When j is equal to m, obtain the weld point cloud data set P Wi The direction vectors of all point clouds in the cloud.

[0119] S26, setting the lower threshold D1 and the upper threshold D2 of the weld direction vector, extracting the weld point cloud data set P Wi Weld bead direction include:

[0120] S261, let the fitted weld plate plane expression be A1x+B1y+C1z=D1, record the weld point cloud data set P Wi There are m point cloud data in total, for the weld point cloud data set P Wi Point P in Wij The cosine value of the angle between its direction vector and the welding plate plane is calculated by the following formula: ij :

[0121] Where, j = 1, 2, 3…m;

[0122] S262. Create array D Wi To store the weld direction point cloud data of the i-th weld, when D1≤cosΦ ij ≤D2, then the point P Wij Put it into array D Wi , otherwise continue to the next step;

[0123] S263, determine whether j is equal to m, if j is less than m, return to S261, j = j + 1;

[0124] S264. When j is equal to m, obtain the weld direction point cloud data set D Wi ;

[0125] S265, calculate the weld direction point cloud dataset D Wi The point cloud centroid C Wi , the coordinates are marked as (a i ,b i ,c i );

[0126] S266, perform point cloud de-centroiding processing, and convert D Wi The point cloud data in the subtraction of the point cloud centroid C Wi , get the point cloud data position matrix deM after removing the centroid i ;

[0127] S267, position matrix deM i Perform singular value decomposition to obtain the matrix U i , S i and V i ;

[0128] S268. The eigenvector corresponding to the maximum eigenvalue is the direction vector of the fitting line, and the centroid of the point cloud is a point on the fitting line, that is:

[0129]

[0130] Where (x i ,y i ,z i ) is the point on the straight line of the fitting space, (l i , m i , n i ) represents the weld direction of the fitting line

[0131] S27, the direction of the weld Transform to Direction, get the rotation matrix R i , and multiply the weld point cloud dataset by the rotation matrix R i , and obtain the rotated weld point cloud dataset.

[0132] Since the weld direction may not be parallel to the x-axis, in order to facilitate subsequent calculations, the weld direction is replaced by the weld direction vector Convert to The specific process of the direction is as follows:

[0133] Direct the weld bead Convert to unit vector:

[0134]

[0135] remember The direction unit vector is

[0136] when Then q i =(0,0,0,0), otherwise

[0137]

[0138] In the formula, sum() means summing the three directions of the vector;

[0139] By quaternion q i Construct the rotation matrix R i ;

[0140] Using the rotation matrix R i The weld point cloud dataset P Wi The coordinates are transformed to the direction where the weld is parallel to the X-axis, that is, the rotated point cloud dataset P Wi =R i *P Wi T .

[0141] S28, slicing the rotated weld point cloud data set along the X-axis direction to obtain multi-pass weld slice point cloud data P sliceij ,include:

[0142] S281, set the interval d and the width f of the sliced ​​point cloud, select the point cloud with a width of f for slicing at each interval d, and record the point cloud P Wi The range is X mini To X maxi , where i represents the i-th weld, j represents the j-th point cloud slice, and j starts from 0;

[0143] S282, let j = j + 1, determine X mini +j*f+(j-1)*d is greater than X maxi , when X mini +j*f+(j-1)*d is less than or equal to X maxi , then select the coordinate range X mini +(j-1)*f+(j-1)*d~X mini The points within +j*f+(j-1)*d are taken as the slice point cloud data P sliceij ;

[0144] When X mini +j*f+(j-1)*dX maxi Greater than X maxi , then end this step.

[0145] S29, extract weld slice point cloud data P sliceij The highest point of the point cloud along the Z axis, the height of the sliced ​​weld, and the width of the sliced ​​weld are used to extract the weld features, including:

[0146] Note: sliceij The three-dimensional coordinates of sliceij ,y sliceij , z sliceij );

[0147] The highest point H sliceij =maxz sliceij , Slice weld height h sliceij =maxz sliceij -minz sliceij , Slice weld width W sliceij =maxy sliceij -miny sliceij , the weld foot position of the sliced ​​weld is maxy sliceij and miny sliceij .

[0148] Furthermore, before S21, the step also includes: performing voxel down-sampling processing on the point cloud information to obtain a pre-processed point cloud data set P.

[0149] Specifically, point cloud downsampling is to resample the point cloud according to certain sampling rules. The purpose is to reduce the density of the point cloud while ensuring that the overall geometric features of the point cloud remain unchanged, thereby reducing the amount of data and algorithm complexity of related processing and improving the processing speed of subsequent point cloud algorithms. Voxel downsampling is a very efficient point cloud downsampling method. The sampling points are distributed more evenly and the processing effect is better. The voxel downsampling process is:

[0150] 1) The maximum values ​​Xmax, Ymax, Zmax and the minimum values ​​Xmin, Ymin, Zmin of X, Y and Z in the three-direction coordinate system can be obtained through the point cloud coordinate values. The side lengths lx, ly and lz of the bounding box can be calculated by subtracting the minimum value from the maximum value:

[0151]

[0152] 2) Set the side length of the voxel cube cell to be divided into M, N, and L parts in the three directions of X, Y, and Z coordinate axes, so that the total number of voxel cubes is SUM = M*N*L, where floor means rounding down:

[0153]

[0154] 3) Label each voxel cube as (x, y, z) to obtain the voxel cube corresponding to each data point:

[0155]

[0156] 4) Calculate the centroid of each point cloud, and only keep the centroid point cloud of each voxel cube, and remove other point clouds. The method for calculating the centroid of the point cloud is:

[0157]

[0158] Reference Figure 2 As shown, the present invention also discloses a weld detection and grinding method based on a 3D vision sensor, comprising the following steps:

[0159] Step 1: Obtain the point cloud information of the workpiece by shooting with a 3D camera;

[0160] Step 2: Based on the above weld grinding feature extraction method, extract the weld grinding features;

[0161] Step 3: Generate a weld grinding path based on the weld grinding features, and the robot grinds the workpiece based on the grinding path.

[0162] Specifically, the weld feature points extracted in step three generate a weld grinding path, and the grinding point position and posture are sent to the robot in real time through the host computer, and the robot performs the grinding work.

[0163] Step 4: Repeat steps 1 and 2 to obtain the grinding weld height feature, and slice the highest point H through the weld point cloud. sliceij Compare the weld height requirements and conduct weld formation inspection: if the inspection passes, the grinding process is completed; if the inspection fails, return to step three to continue grinding.

[0164] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0165] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0166] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0167] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0168] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A weld grinding feature extraction method, characterized in that: include: S1. Obtaining point cloud information of a workpiece to be tested, wherein the workpiece to be tested is colored by a colored penetrant flaw detection agent, and the point cloud information is collected by an infrared structured light 3D camera; S2, processing the original point cloud information of the workpiece to be measured to obtain the weld grinding features, including the following steps: S21. Process the point cloud dataset P by Euclidean clustering to segment the target welding plate point cloud dataset P B ; S22, based on the target welding plate point cloud dataset P B , the welding plate plane is fitted by RANSAC algorithm, and the point cloud dataset P on the welding plate plane is obtained S ; S23, the target welding plate point cloud dataset P B The point cloud dataset P on the weld plate plane S Make a difference and obtain the weld point cloud dataset P W ; S24, weld point cloud dataset P W Perform Euclidean clustering to segment the weld point cloud dataset P W Divide into n independent weld point cloud datasets P Wi ; S25. Extracting weld point cloud dataset P Wi The direction vector of the midpoint cloud; S26, setting the lower threshold D1 and the upper threshold D2 of the weld direction vector, extracting the weld point cloud data set P Wi Weld bead direction S27, the direction of the weld Transform to Direction, get the rotation matrix R i , and multiply the weld point cloud dataset by the rotation matrix R i , obtain the rotated weld point cloud dataset; S28, slicing the rotated weld point cloud data set along the X-axis direction to obtain multi-pass weld slice point cloud data P sliceij ; S29, extract weld slice point cloud data P sliceij The highest point of the point cloud along the Z-axis, the height of the sliced ​​weld, and the width of the sliced ​​weld are used to complete the extraction of weld features.

2. The weld grinding feature extraction method according to claim 1, characterized in that: The S21 also includes: The point cloud information is subjected to voxel downsampling processing to obtain a preprocessed point cloud data set P.

3. The weld grinding feature extraction method according to claim 1, characterized in that: The S21 includes: Determine the query point P i , set the distance threshold r, and the calculation formula for the distance d between any two points in the point cloud data set is: Among them, p i With p j Represents two points in the point cloud, p ik With p jk Represents any point in the neighborhood of two points; Find the distance from the query point P through KD-tree i The nearest n neighbor points p j , where j = 1, 2, ..., n, calculate the n nearest neighbor points to the query point P according to the above formula i The Euclidean distance d j ; Compare distance d j The points with a distance threshold r are classified into class M until the number of points in class M no longer increases, and the segmentation is completed.

4. The weld grinding feature extraction method according to claim 1, characterized in that: The S24 includes: Weld point cloud dataset P W Perform Euclidean clustering segmentation and set the minimum value κ of the number of point clouds contained in the cluster. After Euclidean clustering segmentation, the point cloud dataset P Wi When the number is greater than or equal to κ, the classified point cloud dataset is retained, otherwise the classified point cloud dataset is deleted, where i=1, 2, 3…n, and n is a natural number greater than 3.

5. The weld grinding feature extraction method according to claim 1, characterized in that: The S25 includes: S251, weld point cloud dataset P Wi There are m point cloud data in total. Select the weld point cloud data set P Wi A point P Wij and the q point cloud data around it (j=1, 2, 3…m); S252, calculate the point cloud centroid C of q+1 point cloud data Wij , the formula is as follows: S253, perform point cloud de-centroiding processing, and subtract the point cloud centroid C from the q+1 point cloud data in sequence Wij , get the point cloud data position matrix deM after removing the centroid ij ; S254, Constructing the covariance matrix: Covi j =deMi j T *deM ij ; S255, covariance matrix Cov ij Perform singular value decomposition, the singular vector corresponding to the minimum singular value (u ij , v ij , w ij ) is point P Wij The normal vector of S256, determine whether j is equal to m, if j is less than m, return to S251, j=j+1; S257. When j is equal to m, obtain the weld point cloud data set P Wi The direction vectors of all point clouds in the cloud.

6. The weld grinding feature extraction method according to claim 5, characterized in that: The S26 includes: S261, let the fitted weld plate plane expression be A1x+B1y+C1z=D1, record the weld point cloud data set P Wi There are m point cloud data in total, for the weld point cloud data set P Wi Point P in Wij The cosine value of the angle between its direction vector and the welding plate plane is calculated by the following formula: ij : Where, j = 1, 2, 3...m; S262. Create array D Wi To store the weld direction point cloud data of the i-th weld, when D1≤cosΦ ij ≤D2, then the point P Wij Put it into array D Wi , otherwise continue to the next step; S263, determine whether j is equal to m, if j is less than m, return to S261, j = j + 1; S264. When j is equal to m, obtain the weld direction point cloud data set D Wi ; S265, calculate the weld direction point cloud dataset D Wi The point cloud centroid C Wi , the coordinates are marked as (a i , b i , c i ); S266, perform point cloud de-centroiding processing, and convert D Wi The point cloud data in the subtraction of the point cloud centroid C Wi , get the point cloud data position matrix deM after removing the centroid i ; S267, position matrix deM i Perform singular value decomposition to obtain the matrix U i , S i and V i ; S268. The eigenvector corresponding to the maximum eigenvalue is the direction vector of the fitting line, and the centroid of the point cloud is a point on the fitting line, that is: Where (x i ,y i ,z i ) is the point on the straight line in the fitting space, (l i , m i , n i ) represents the weld direction of the fitting line 7. The weld grinding feature extraction method according to claim 6, characterized in that: The S27 includes: Direct the weld bead Convert to unit vector: remember The direction unit vector is when Then q i =(0,0,0,0), otherwise In the formula, sum() means summing the three directions of the vector; By quaternion q i Construct the rotation matrix R i ; Using the rotation matrix R i The weld point cloud dataset P Wi The coordinates are transformed to the direction where the weld is parallel to the X-axis, that is, the rotated point cloud dataset P Wi =R i *P Wi T .

8. The weld grinding feature extraction method according to claim 1, characterized in that: The S28 includes: S281, set the interval d and the width f of the sliced ​​point cloud, select the point cloud with a width of f for slicing at each interval d, and record the point cloud P Wi The range is X mini To X maxi , where i represents the i-th weld, j represents the j-th point cloud slice, and j starts from 0; S282, let j = j + 1, determine X mini +j*f+(j-1)*d is greater than X maxi , when X mini +j*f+(j-1)*d is less than or equal to X maxi , then select the coordinate range X mini +(j-1)*f+(j-1)*d~X mini The points within +j*f+(j-1)*d are taken as the slice point cloud data P sliceij ; When X mini +j*f+(j-1)*dX maxi Greater than X maxi , then end this step.

9. The weld grinding feature extraction method according to claim 1, characterized in that: The S29 includes: Note: sliceij The three-dimensional coordinates of sliceij ,y sliceij , z sliceij ); The highest point H sliceij =maxz sliceij , Slice weld height h sliceij =maxz sliceij -minz sliceij , Slice weld width W sliceij =maxy sliceij -miny sliceij , the weld foot position of the sliced ​​weld is maxy sliceij and miny sliceij .

10. A weld detection and grinding method based on 3D vision sensor, characterized in that: The following steps are involved: Step 1: Obtain the point cloud information of the workpiece by shooting with a 3D camera; Step 2: extracting the features of weld grinding based on the weld grinding feature extraction method according to any one of claims 1 to 9; Step 3: Generate a weld grinding path according to the weld grinding characteristics, and the robot grinds the workpiece according to the grinding path; Step 4: Repeat steps 1 and 2 to obtain the grinding weld excess height characteristics and perform weld forming detection: if the detection passes, the grinding process is completed; if the detection fails, return to step 3 to continue grinding.

Citation Information

Cited By

  • Method for generating virtual weld bead using artificial intelligence

    WO2026121408A1