A seam detection method based on surface structured light

Abstract

The present application belongs to the field of automatic welding technology, and particularly relates to a welding seam detection method based on surface structured light. The method comprises the following steps: (1) three-dimensional reconstruction of a workpiece: using a surface structured light three-dimensional scanner to perform three-dimensional scanning on the workpiece to obtain a coded image II, a phase diagram and three-dimensional data of the workpiece, and reading through a computer; (2) welding seam feature point identification: performing welding seam feature identification or manually annotating the welding seam on the coded image II to obtain a two-dimensional coordinate set of the welding seam on the coded image II. The present application simultaneously utilizes two-dimensional information and three-dimensional information of the surface structured light scanner, and realizes an efficient welding seam detection method of "two-dimensional feature identification and three-dimensional coordinate mapping" fusion through triangulation mapping, solves the problem of welding seam detection without obvious structural features, eliminates calibration errors between devices compared with a split type multi-sensor fusion method, improves welding seam detection accuracy, and simplifies the detection process.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of automatic welding, and particularly relates to a welding seam detection method based on a plane structured light. BACKGROUND

[0002] Welding seam detection is an important work of automatic welding, and there are three major types in the prior art: A. manual teaching; B. physical touch detection; and C. machine vision detection. Manual teaching is time-consuming and laborious and is difficult to achieve flexible welding, physical touch detection is a contact measurement technology as the name implies, and is usually applied to welding work of specific workpieces and does not have universality. The visual detection method has become a hot research and development direction at present due to its non-contact, high-precision and flexible characteristics.

[0003] Patent 202411112212.9 discloses a welding robot dragging teaching welding method based on a line structured light, which obtains workpiece welding seam track information by manual dragging teaching and line laser scanning. The welding seam position, topography and other characteristics are obtained by processing the data of the line laser sensor. The welding path is planned by using the position characteristics. The welding current, speed and other parameters are adaptively adjusted according to the welding seam topography characteristics. The patent needs manual dragging when reconstructing the three-dimensional data of the workpiece, which is time-consuming. The welding seam detection algorithm is performed in three-dimensional space and cannot deal with welding seams without obvious structural characteristics.

[0004] Patent 202410271448.0 discloses a robot flexible welding path planning method for non-standard workpieces. First, the three-dimensional model of the welding workpiece is obtained by acquiring and processing the surface point cloud information of the welding workpiece. Then, the welding seam extraction is completed by using a feature recognition algorithm. Then, the welding seam posture and welding gun posture are determined. Then, the welding sequence of the welding seam is determined by applying a meta-heuristic algorithm. Finally, the collision-free path is quickly found by applying an efficient path planning algorithm, and the welding path planning is completed. The welding robot is guided to weld by using the planned welding path parameters. The present application is aimed at multiple specifications and small batches of workpieces, and combines point cloud three-dimensional reconstruction, welding seam feature extraction and other data processing methods to quickly find the collision-free path of the welding robot by applying an efficient path planning method, which can effectively improve the path quality and compress the planning time, and at the same time ensure the continuity of the welding operation and the consistency of the welding quality. The patent also performs welding seam detection in three-dimensional space and cannot deal with welding seams without obvious structural characteristics.

[0005] Patent 202211489509.8 discloses a multi-source data fusion robot welding path recognition method, which uses a two-dimensional sensor (camera) to take an image of the workpiece, selects a feature positioning point in the image, then uses a three-dimensional sensor (laser sensor) to measure the accurate positioning point coordinates, and finally performs interpolation calculation to generate a complete welding path. This method uses two-dimensional and three-dimensional fusion measurement to improve the welding seam detection accuracy to some extent, but has the following three problems: 1. The robot end needs to traverse all feature points before each welding, affecting the detection efficiency; 2. It needs to use a camera and a laser sensor together, which requires additional system calibration, and the calibration parameters affect the detection accuracy; 3. The complete welding path generated by interpolation depends on the number of positioning points. More positioning points can improve the interpolation accuracy, but the robot end needs to traverse all feature points, resulting in a more complex detection process.

[0006] The current visual detection method for automatic welding has poor intelligence level, and manual dragging measurement or pre-calibration of features are required, making the welding seam detection process more complicated. Some existing technologies (such as patent 202211489509.8) propose a method of fusing two-dimensional and three-dimensional information. These methods use multiple sensors to measure, introducing additional device-to-device calibration problems, and the calibration accuracy has a significant impact on the final detection accuracy. Some methods using three-dimensional sensors (such as laser scanners) to obtain three-dimensional data and then identify welding seams in three-dimensional data usually rely on the prominent structure of the welding seam (such as the intersection line of two planes, patent CN110449783B uses this method). However, in many cases, the welding seam structure is not prominent, for example, in the automotive manufacturing industry, there are many curved surface connections. It is very difficult to automatically identify welding seams without prominent structures in three-dimensional space. SUMMARY

[0007] The purpose of the present application is to provide a welding seam detection method based on surface structured light to solve the problems in the prior art.

[0008] The technical solution adopted by the present application to solve its technical problems is:

[0009] A welding seam detection method based on surface structured light, comprising the following steps:

[0010] (1) Three-dimensional reconstruction of the workpiece: use a surface structured light three-dimensional scanner to perform three-dimensional scanning of the workpiece, obtain the coded image II, phase diagram, and three-dimensional data of the workpiece, and read them through a computer;

[0011] (2) Welding seam feature point recognition: perform welding seam feature recognition or manual welding seam annotation on the coded image II to obtain a two-dimensional coordinate set of the welding seam on the coded image II;

[0012] (3) Three-dimensional coordinate mapping: according to the calibration parameters of the surface structured light three-dimensional scanner, the phase diagram, and the three-dimensional data, the two-dimensional coordinate set of the weld is mapped to the three-dimensional coordinate set of the weld based on the coordinate system of the surface structured light scanner through the triangulation formula;

[0013] (4) Hand-eye calibration: the three-dimensional coordinate set of the weld is converted from the coordinate system of the surface structured light scanner to the coordinate system of the welding robot end through the hand-eye calibration algorithm;

[0014] (5) Welding path planning: using the RRT path planning algorithm, the welding path is planned, and the welding robot end is driven to perform welding work in the current welding area. After welding, return to step (1) to perform welding work in the next area until all welding areas are welded;

[0015] (6) Path correction: the weld path is corrected online during welding by the surface structured light three-dimensional scanner again.

[0016] Further, the specific method of step (1) three-dimensional reconstruction of the workpiece is to install the surface structured light three-dimensional scanner on the welding robot end, so that it moves synchronously with the welding robot end. Before welding, the welding robot is driven to the starting point of the workpiece to be welded, and the surface structured light three-dimensional scanner is controlled to perform three-dimensional reconstruction of the workpiece. When the surface structured light scanner is working, it projects the coded image I onto the workpiece using its built-in projector, and captures the coded image II using its built-in camera. The coded image II is decoded to obtain the phase diagram, and at the same time the surface structured light three-dimensional scanner generates three-dimensional data of the workpiece through its built-in algorithm. The coded image II, the phase diagram, the three-dimensional data, and the calibration parameters of the surface structured light three-dimensional scanner are read by the computer.

[0017] Further, the calibration parameters of the surface structured light three-dimensional scanner in step (3) are: the internal parameters K c of the camera, the internal parameters K P of the projector, and the external parameters [R, t] between the camera and the projector.

[0018] The triangulation formula is:

[0019]

[0020] Wherein, the three-dimensional coordinates {X P , Y P , Z P} in the projector coordinate system are converted to the three-dimensional coordinates {X c , Y c , Z c}, u' is the gray value of {u, v} at the corresponding position on the phase map, i.e. the phase value, v' is an unknown number, r3 is the third row data of R, t3 is the third row data of t, R is a rotation matrix, and t is a translation matrix.

[0021] Further, the equation of the hand-eye calibration algorithm in step (4) is represented as A i X = XB i , wherein A i and B i represent the relative motion between the welding robot end effector and the structured light scanner, and X is the hand-eye calibration matrix to be solved, which is solved by a two-stage closed method, and the specific steps are as follows:

[0022] 4.1, A and R X represent the rotation components of A i , B i and X, respectively, and t X represent the translation vectors of A i , B i and X, respectively, so that

[0023] 4.2, according to the matrix operation theorem, A and B

[0024] 4.3, from step 4.2, A

[0025] 4.4, the equivalent rotation axis a i of A

[0026]

[0027] 4.5, by R X , the relationship between the equivalent rotation axes a i and b i of A and B is established, i.e. a i = R X b i , wherein b i is the equivalent rotation axis of B ;

[0028] 4.6, a sufficient number of equations are generated by a two-stage closed solving method, and all items in R X are directly solved using standard linear algebra tools;

[0029] For n>2, define the over-constrained linear equation A=RX B, where A = (a1,... a n ,a 1,1 ,...a i,j ,...a n,n ), B = (b1,... b n ,b 1,1 ,...b i,j ,...b n,n ), and a i,j = a i x a j and b i,j = b i x b j , i, j e {1,..., n}, i < j.

[0030] It is derived that the least square solution of R X is where B + = B T (BB T ) -1 denotes the Moore-Penrose pseudo-inverse of matrix B, and B + = B -1 if matrix B is full rank.

[0031] Finally, the translation matrix t X is solved by using the formula , so as to complete the solution of the hand-eye calibration and obtain the three-dimensional coordinate set {X r , Y r , Z r} under the end coordinate system of the welding robot.

[0032] Further, the specific steps of the RRT path planning algorithm in step (5) are as follows:

[0033] 5.1, determine the starting position of the mechanical arm of the welding robot, the weld position and the obstacle position, wherein the obstacles are divided into environmental obstacles and the expected trajectories planned by the remaining mechanical arms, and the starting position is taken as the root node of the RRT random tree;

[0034] 5.2, randomly sample a point in the workspace as an exploration target point, find the node closest to the sampling point in the random number, and extend a distance from the nearest node to the sampling point to create a new node;

[0035] 5.3, check whether the new node collides with the obstacle, if not, add it to the random tree and connect it with the nearest node, if there is a collision, return to step 5.2;

[0036] 5.4, Find the neighbors within a defined radius around the newly created node as candidates for replacing the parent node, calculate the path cost from the start point to each neighbor and add the path cost to each neighbor, select the node with the minimum cost as the new parent node;

[0037] 5.5, Further optimize the path cost between the nodes of the random tree after reselecting the parent node;

[0038] 5.6, For the welding robot arm, if it can successfully connect to the target position, it is considered that an effective path has been found, if an effective path cannot be found within a limited number of iterations or time, resample and expand the path;

[0039] 5.7, Repeat steps 5.1-5.6 to generate the final welding path.

[0040] Further, the specific method of step (6) path correction is:

[0041] 6.1, Move the end of the welding robot to a small distance above the start of the weld, at this time the three-dimensional coordinates {X c ,Y c ,Z c} are transformed into {X' c ,Y' c ,Z' c}, and the rigid transformation matrix T m is used to move the end, then {X' c ,Y' c ,Z' c} = T m ×{X c ,Y c ,Z c};

[0042] 6.2, Call the built-in camera of the scanner to take a second shot of the workpiece to obtain the encoded image III, and map {X' c ,Y' c ,Z' c} back to two-dimensional coordinates through inverse triangulation, i.e., inverse operation of the triangulation formula, denoted as Find the pixel with the minimum gray value within a certain range around the remapped two-dimensional coordinates of the weld , and use it as the corrected two-dimensional coordinates of the weld, the search algorithm is:

[0043]

[0044] where (u,v) U represents the neighborhood coordinate set of two-dimensional coordinates u and v, I(u,v) represents the gray value at two-dimensional coordinates u and v, u f ,yf is the corrected two-dimensional coordinate of the weld seam, because the end position T m , the scanner parameter K c , K P , R, t are all known, the corrected two-dimensional coordinate of the weld seam {u f , v f} can be directly calculated to correct the weld seam three-dimensional coordinate {X rf , Y rf , Z rf} under the moving welding robot end coordinate system;

[0045] 6.3, after completing the welding of all the corrected weld seams, repeat steps 6.1-6.2 until the welding of the entire weld seam is completed.

[0046] Further, after step (3) three-dimensional coordinate mapping is completed, a set of weld seam three-dimensional coordinates in a welding area of the workpiece is obtained, and then steps (1) to (3) are repeated to perform three-dimensional coordinate mapping of the weld seam in the next welding area, and so on, until all the weld seam three-dimensional coordinates on the workpiece are mapped, and a weld seam template is obtained, or steps (1) to (3) are omitted, and the weld seam template is directly created from the CAD data;

[0047] After the three-dimensional reconstruction of the workpiece is completed by step (1) during welding, the weld seam template is matched with the three-dimensional data, and the ICP point cloud registration algorithm is used to calculate the set of weld seam three-dimensional coordinates under the plane structured light scanner coordinate system, and then return to step (4).

[0048] Further, the calculation process of the ICP point cloud registration algorithm is as follows: first, the nearest neighbor loss function is used:

[0049]

[0050] to calculate T, wherein T is a rigid transformation matrix, R is a rotation matrix, t is a translation matrix, p i is the source point cloud set, is the nearest neighbor target point cloud set;

[0051] Then, the formula {X c , Y c , Z c} = T x {X m , Y m , Z m} is used to calculate the set of weld seam three-dimensional coordinates {X c , Y c , Z c} based on the plane structured light scanner coordinate system, wherein {X m , Y m , Z m} is a weld three-dimensional coordinate set in a weld template.

[0052] The present application has the following beneficial effects:

[0053] 1. The present application simultaneously utilizes two-dimensional information and three-dimensional information of a surface structure light scanner, and realizes a high-efficiency weld detection method of "two-dimensional feature recognition and three-dimensional coordinate mapping" fusion through triangulation mapping, which solves the problem of weld detection without obvious structural features, eliminates the calibration error between devices compared with the method of multi-sensor fusion, improves the weld detection accuracy, and simplifies the detection process.

[0054] 2. Since the three-dimensional coordinates of all weld feature points are obtained automatically, high-density weld feature points can be extracted in the two-dimensional image, which improves the weld detection accuracy without increasing additional measurement work.

[0055] 3. An online weld path correction method based on weld image remapping is proposed, which further improves the weld detection accuracy while maintaining high efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is a process schematic diagram of embodiment 1 of the present application.

[0057] Figure 2 is a data conversion relationship schematic diagram of the present application.

[0058] Figure 3 is a process schematic diagram of embodiment 4 of the present application.

[0059] Figure 4 is a process schematic diagram of embodiment 5 of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical scheme and advantages of the present application clearer and more apparent, the present application will be further described in detail below in combination with specific embodiments and drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0061] Embodiment 1: The present embodiment provides a weld detection method based on surface structure light, which is a conventional weld detection method for workpieces with high accuracy requirements.

[0062] As shown in Figures 1-2 , a weld detection method based on surface structure light includes the following steps:

[0063] (1) Three-dimensional reconstruction of the workpiece: use a surface structure light three-dimensional scanner to perform three-dimensional scanning on the workpiece to obtain the coded image II, phase diagram and three-dimensional data of the workpiece, and read through a computer, the specific method being:

[0064] The face structure light three-dimensional scanner is installed on the end of the welding robot, and moves synchronously with the end of the welding robot. Before welding, the welding robot is driven to the starting point of the workpiece to be welded, and the face structure light three-dimensional scanner is controlled to perform three-dimensional reconstruction on the workpiece. When the face structure light scanner is working, the built-in projector of the face structure light scanner projects a coded image I onto the workpiece, and the built-in camera of the face structure light scanner captures a coded image II. The coded image II is decoded to obtain a phase image. At the same time, the face structure light three-dimensional scanner generates three-dimensional data of the workpiece through its built-in algorithm. The coded image II, the phase image, the three-dimensional data, and the calibration parameters of the face structure light three-dimensional scanner are read by a computer.

[0065] (2) Welding seam feature point recognition: welding seam feature recognition is performed on the coded image II to obtain a two-dimensional coordinate set of the welding seam on the coded image II.

[0066] Feature recognition mainly relies on the characteristic that the gray value of the welding seam contour is generally lower than that of the surrounding area. Because the welding seam is always a material missing area, the welding seam has a more abundant performance in the coded image II. The gray value of the welding seam is often lower than that of the surrounding area (because the welding seam position is not filled with material). Based on this characteristic, using a machine learning algorithm to automatically extract welding seam feature points has a higher extraction success rate. These feature points are a set of continuous low gray pixels, represented by a two-dimensional coordinate set {u, v} of the image. The welding seam feature recognition method is prior art, and will not be described here.

[0067] (3) Three-dimensional coordinate mapping: according to the calibration parameters of the face structure light three-dimensional scanner, the phase image, and the three-dimensional data, the two-dimensional coordinate set of the welding seam is mapped to a three-dimensional coordinate set of the welding seam based on the coordinate system of the face structure light scanner through a triangulation formula.

[0068] Unlike existing methods that use multiple devices, the two-dimensional image used for welding seam detection is intermediate data of the three-dimensional scanner (captured by the camera inside the scanner). The built-in camera and projector of the face structure light scanner have already obtained high-precision calibration parameters using a special device at the factory, which can be directly read out. The calibration parameters based on the face structure light three-dimensional scanner are: the internal parameters K c of the camera, the internal parameters K P of the projector, and the external parameters [R, t] between the camera and the projector.

[0069] The triangulation formula is:

[0070]

[0071]

[0072] wherein the three-dimensional coordinates in the projector coordinate system {X P , YP P} into three-dimensional coordinates {X c c c} in the coordinate system of the area structured light scanner, u' is the gray value of {u, v} at the corresponding position on the phase diagram, that is, the phase value, v' is an unknown number, r3 is the third row data of R, t3 is the third row data of t, R is a rotation matrix, and t is a translation matrix.

[0073] (4) Hand-eye calibration: The set of three-dimensional coordinates of the weld is converted from the coordinate system of the area structured light scanner to the coordinate system of the welding robot end through a hand-eye calibration algorithm.

[0074] The equation of the hand-eye calibration algorithm is A i X = XB i , wherein A i and B i represent the relative motion between the welding robot end effector and the area structured light scanner, X is the hand-eye calibration matrix to be solved, which is solved by a two-stage closed method, and the specific steps are as follows:

[0075] 4.1, A and R X represent the rotation components of A i , B i and X, respectively, and t X represent the translation vectors of A i , B i and X, respectively, and thus A

[0076] 4.2, according to the matrix operation theorem, A and B

[0077] 4.3, from step 4.2, A

[0078] 4.4, the equivalent rotation axis a i of A is represented as:

[0079]

[0080] 4.5, by R X , the relationship between the equivalent rotation axes a i and b i of A and B is established, that is, a i = R X b i , wherein b i is​​​ the equivalent rotation axis of R

[0081] 4.6, generate a sufficient number of equations by a two-stage closed solution method, and directly solve all items in R X using standard linear algebra tools;

[0082] For n>2, define the over-constrained linear equation A=R X B, where A=(a1,…a n ,a 1,1 ,…a i,j ,…a n,n ), B=(b1,…b n ,b 1,1 ,…b i,j ,…b n,n ), and a i,j =a i ×a j andb i,j =b i ×b j ,i,j∈{1,…,n},i<j;

[0083] It is concluded that the least square solution of R X is where B + =B T (BB T ) -1 represents the Moore-Penrose pseudo-inverse of matrix B, and if matrix B is full rank, B + =B -1 .

[0084] Finally, the translation matrix t X is solved by using the formula , so as to complete the solution of hand-eye calibration, and obtain the three-dimensional coordinate set {X r ,Y r ,Z r} under the end coordinate system of the welding robot.

[0085] (5) Welding path planning: using RRT path planning algorithm, the welding path is planned, and the welding robot end is driven to carry out welding work in the welding area, and after welding, step (1) is returned to carry out welding work in the next area, until all welding areas are welded.

[0086] The specific steps of RRT path planning algorithm are:

[0087] 5.1, determine the start position of the welding robot, the welding seam position and the obstacle position, where the obstacle is divided into environmental obstacles and the rest of the expected trajectory planned by the remaining robot arm, and the start position is taken as the root node of the RRT random tree;

[0088] 5.2, randomly sample a point in the workspace as an exploration target point, find the nearest node to the sampling point in the random number, and extend a distance from the nearest node to the sampling point to create a new node;

[0089] 5.3, check whether the new node collides with the obstacle, if not, add it to the random tree and connect it to the nearest node, if there is a collision, return to step 5.2;

[0090] 5.4, find the neighbors within a defined radius around the newly generated node as candidates for replacing the parent node, calculate the path cost from the start point to each neighbor node and the path cost to each neighbor node, and select the node with the minimum cost as the new parent node;

[0091] 5.5, after reselecting the parent node, further optimize the connection cost between the nodes of the random tree;

[0092] 5.6, for the welding robot arm, if it can successfully connect to the target position, it is considered that an effective path has been found, if an effective path cannot be found within a limited number of iterations or time, resample and expand the path;

[0093] 5.7, repeat steps 5.1-5.6 to generate the final welding path.

[0094] (6) Path correction: the welding seam path is corrected online again by the face structure light three-dimensional scanner during welding.

[0095] The specific method of path correction is:

[0096] 6.1, move the end of the welding robot to a small distance above the start point of the welding seam, at this time the three-dimensional coordinates {X c ,Y c ,Z c} are transformed into {X' c ,Y' c ,Z' c}, and the rigid transformation matrix T m is used to move the end, then {X' c ,Y' c ,Z' c} = T m ×{X c ,Y c ,Z c};

[0097] 6.2, call the scanner built-in camera to take a second shot of the workpiece to get the encoding image III, and map {X c ,Y c ,Z c} back to two-dimensional coordinates through inverse triangulation, denoted as Find the remapped two-dimensional coordinates of the weld The pixel with the smallest grayscale value in a certain range near the remapped two-dimensional coordinates of the weld is taken as the corrected two-dimensional coordinates of the weld, and the search algorithm is:

[0098]

[0099] where (u,v) U represents the neighborhood coordinate set of two-dimensional coordinates u and v, I(u,v) represents the grayscale value at two-dimensional coordinates u and v, u f ,y f is the corrected two-dimensional coordinates of the weld, because the end position T m , the scanner parameters K c , K P , R, and t are known, the corrected two-dimensional coordinates of the weld {u f ,v f} can be directly calculated to obtain the three-dimensional coordinates of the weld in the corrected end coordinate system of the welding robot {X rf ,Y rf ,Z rf};

[0100] 6.3, after completing the welding of all the corrected welds, repeat steps 6.1-6.2 until the welding of the entire weld is completed.

[0101] Example 2:

[0102] This embodiment provides a weld detection method based on planar structured light, which is aimed at workpieces with low welding precision requirements. The method steps are basically the same as those of Example 1, except that step (6) path correction is not used during operation.

[0103] Example 3:

[0104] This embodiment provides a weld detection method based on planar structured light, which is aimed at cases where the weld features are not very obvious (small grayscale difference) and cannot be automatically identified in two dimensions. The method steps are basically the same as those of Example 1, except that in step (2), the encoding image II is manually annotated to obtain the two-dimensional coordinate set of the weld on the encoding image II.

[0105] Example 4:

[0106] The embodiment provides a welding seam detection method based on a surface structure light, and is used for batch welding of workpieces. The method steps are basically the same as those of the embodiment 1, and the difference is that:

[0107] As shown in the embodiment 1, first, the three-dimensional coordinate mapping of the welding seam in a welding area of the workpiece is completed according to steps (1) to (3), then steps (1) to (3) are repeated to perform the three-dimensional coordinate mapping of the welding seam in the next welding area, and the above steps are repeated until the three-dimensional coordinate mapping of all the welding seams on the workpiece is completed, so that the welding seam template is obtained. Since the welding seams of all the workpieces in the same batch are the same, only one workpiece needs to be used to create the welding template. Figure 3

[0108] When welding, the welding seam template is matched with the three-dimensional data after the three-dimensional reconstruction of the workpiece is performed through step (1), the welding seam three-dimensional coordinate set in the surface structure light scanner coordinate system is calculated by using an ICP point cloud registration algorithm, and then step (4) is returned to continue steps (4) to (5) shown in the embodiment 1. Step (6) can be selected according to the welding accuracy requirement.

[0109] The calculation process of the ICP point cloud registration algorithm is that, first, the nearest neighbor loss function is used:

[0110]

[0111] to calculate T, wherein T is a rigid transformation matrix, R is a rotation matrix, t is a translation matrix, p i is a source point cloud set, is a nearest neighbor target point cloud set.

[0112] Then, the formula {X c ,Y c ,Z c} = T x {X m ,Y m ,Z m} is used to calculate the welding seam three-dimensional coordinate set {X c ,Y c ,Z c} in the surface structure light scanner coordinate system, wherein {X m ,Y m ,Z m} is the welding seam three-dimensional coordinate set in the welding seam template.

[0113] Embodiment 5:

[0114] The embodiment provides a welding seam detection method based on a surface structure light, and is used for batch welding of workpieces. The method steps are basically the same as those of the embodiment 4, and the difference is that:

[0115] As shown in the embodiment 4, first, the three-dimensional coordinate mapping of the welding seam in a welding area of the workpiece is completed according to steps (1) to (3), then steps (1) to (3) are repeated to perform the three-dimensional coordinate mapping of the welding seam in the next welding area, and the above steps are repeated until the three-dimensional coordinate mapping of all the welding seams on the workpiece is completed, so that the welding seam template is obtained. Since the welding seams of all the workpieces in the same batch are the same, only one workpiece needs to be used to create the welding template.​Figure 4 As shown, the weld seam template is extracted from CAD data, and the creation of the weld seam template does not need to repeat steps (1) to (3), and the CAD data of the workpiece is directly read by a computer, and the remaining steps are implemented according to example 3.

[0116] The above examples only describe the preferred embodiments of the present application, and do not limit the concept and scope of the present application. Various modifications and improvements to the technical solutions of the present application made by those skilled in the art without departing from the design concept of the present application shall fall within the protection scope of the present application.

[0117] The technologies, shapes, and structural parts not described in detail in the present application are well-known technologies.

Claims

1. A weld inspection method based on surface structured light, characterized in that, Includes the following steps: (1) Three-dimensional reconstruction of the workpiece: The workpiece is scanned in three dimensions using a surface structured light three-dimensional scanner to obtain the coded image II, phase map and three-dimensional data of the workpiece, which are then read by a computer; (2) Weld feature point recognition: Weld feature recognition or manual weld annotation is performed on the coded image II to obtain the two-dimensional coordinate set of the weld on the coded image II; (3) Three-dimensional coordinate mapping: Based on the calibration parameters, phase map, and three-dimensional data of the surface structured light three-dimensional scanner, the two-dimensional coordinate set of the weld is mapped to the three-dimensional coordinate set of the weld based on the coordinate system of the surface structured light scanner through the triangulation formula; (4) Hand-eye calibration: The three-dimensional coordinate set of the weld is converted from the coordinate system of the surface structured light scanner to the coordinate system of the welding robot end effector through the hand-eye calibration algorithm; (5) Welding path planning: The RRT path planning algorithm is used to plan the welding path and drive the welding robot end to perform welding work in the welding area. After the welding is completed, return to step (1) to perform welding work in the next area until all welding areas are completed. In step (5), the weld path is corrected online again using a surface structured light 3D scanner during the welding process; The specific methods for path correction are as follows: 6.1 The distance from the end effector of the mobile welding robot to the starting point of the weld is a short distance, at which point the three-dimensional coordinates are { X c, Y c, Z c } transform into { X c ` , Y c ` , Z c ` }, using rigid body transformation matrix T m To indicate end-effector movement, then { X c ` , Y c ` , Z c ` }= T m ×{ X c, Y c, Z c }; 6.

2. Use the scanner's built-in camera to take a second picture of the workpiece to obtain coded image III, then... X c ` , Y c ` , Z c ` The coordinates are remapped to two-dimensional coordinates through inverse triangulation, i.e., the inverse operation of the triangulation formula, and denoted as . Find the remapped 2D coordinates of the weld. The pixel with the smallest grayscale value within a certain radius is used as the two-dimensional coordinate of the weld seam to be corrected. The search algorithm is as follows: ； in( u,v ) U Representing two-dimensional coordinates u,v The set of neighborhood coordinates, I ( u,v ) represents two-dimensional coordinates u,v grayscale value at that location u f ,y f These are the two-dimensional coordinates of the corrected weld, due to the position of the end point movement. T m Scanner parameters K C , K p R and t are all known. The corrected two-dimensional coordinates of the weld are { u f ,v f The three-dimensional coordinates of the weld seam in the end-effector coordinate system of the corrected moving welding robot can be directly calculated. X rf, Y rf, Z rf }; 6.3 After completing the welding of all corrected welds, repeat steps 6.1-6.2 until the welding of an entire weld is completed; After the three-dimensional coordinate mapping in step (3) is completed, the three-dimensional coordinate set of the weld seam in a welding area of ​​the workpiece is obtained. Then, the three-dimensional coordinate mapping of the weld seam in the next welding area is repeated from step (1) to step (3). This process is repeated until the three-dimensional coordinate mapping of all weld seams on the workpiece is completed, and the weld seam template is obtained. Alternatively, the operations from step (1) to step (3) can be omitted, and the weld seam template can be created directly from the CAD data.

2. The weld inspection method based on surface structured light according to claim 1, characterized in that, The specific method for the three-dimensional reconstruction of the workpiece in step (1) is as follows: the surface structure light three-dimensional scanner is installed on the end of the welding robot and moves synchronously with the end of the welding robot. Before welding begins, the welding robot is driven to the starting point of the workpiece to be welded, and the surface structure light three-dimensional scanner is controlled to perform three-dimensional reconstruction of the workpiece. When the surface structure light scanner is working, it uses its built-in projector to project the encoded image I onto the workpiece and uses its built-in camera to capture the encoded image II. The encoded image II is decoded to obtain the phase map. At the same time, the surface structure light three-dimensional scanner generates the three-dimensional data of the workpiece through its built-in algorithm. The encoded image II, the phase map, the three-dimensional data, and the calibration parameters of the surface structure light three-dimensional scanner are read by the computer.

3. The weld inspection method based on surface structured light according to claim 2, characterized in that, The calibration parameters for the structured light-based 3D scanner in step (3) are: camera internal parameters. K C Internal parameters of the projector K p External parameters between the camera and the projector R,t ]; The trigonometric formula is: ； ； ； Among them, through [ R,t ]Transfer the three-dimensional coordinates in the projector coordinate system { X p, Y p, Z p } Transform to 3D coordinates in the surface structured light scanner coordinate system { X c, Y c, Z c }, u` is { u , v The gray value at the corresponding position on the phase map is the phase value. v` is an unknown. r3 is the third row of data in R, t3 is the third row of data in t, R is the rotation matrix, and t is the translation matrix.

4. The weld inspection method based on surface structured light according to claim 3, characterized in that, The equation for the hand-eye calibration algorithm in step (4) is expressed as follows: A i X=XB i ,in A i and B i This indicates the relative motion between the welding robot's end effector and the surface structured light scanner. X The hand-eye calibration matrix to be solved is obtained using a two-stage closed-form method. The specific steps are as follows: 4.1, will R Ai , R Bi and R X They represent A i , B i and X The rotational component, t Ai , t Bi and t X They are respectively represented as A i , B i and X The translation vector is then obtained. ; 4.

2. According to the matrix operation theorem, we get R Ai R X =R X R Bi and R Ai t X + t Ai = R X t Bi + t X ; 4.

3. Obtained from step 4.2 R Ai =R X R Bi R -1 X ; 4.4 R Ai Equivalent axis of rotation a i Represented as: ； 4.5, through R X Establish R Ai and R Bi Equivalent axis of rotation a i and b i The connection between them, namely a i = R X b i ,in b i for R Bi The equivalent axis of rotation; 4.6 Generate a sufficient number of equations using a two-stage closed-form solution method, and solve them directly using standard linear algebra tools. R X All items in; For n>2, define over-constrained linear equations A = R X B ,in A =( a 1 , … a n ，a 1,1 , … a i,j , … a n,n ), B =( b 1 , … b n ， b 1,1 , … b i,j , … b n,n ), and have a i,j =a i × a j and b i,j = b i × b j ,i,j∈{1,…,n},i <j; Conclusion: R X The least squares solution is ,in B + =B T (BB T ) -1 Representation matrix B The Moore-Penrose pseudoinverse, if the matrix B If it is full rank, then B + =B -1 ; Finally, by using the formula R Ai t X + t Ai = R X t Bi + t X Solve for the translation matrix t X This completes the solution for hand-eye calibration, obtaining the set of three-dimensional coordinates in the welding robot's end effector coordinate system. X r, Y r, Z r } 5. The weld inspection method based on surface structured light according to claim 4, characterized in that, The specific steps of the RRT path planning algorithm described in step (5) are as follows: 5.1 Determine the starting position, weld position, and obstacle position of the welding robot's robotic arm. The obstacles are divided into environmental obstacles and the planned expected trajectory of the remaining robotic arm. The starting position is used as the root node of the RRT random tree. 5.

2. Randomly sample a point in the workspace as the target point for exploration. Find the node closest to the sampled point from the random number generator. Extend a certain distance from the nearest node toward the sampled point to create a new node. 5.3 Check if the new node collides with an obstacle. If there is no collision, add it to the random tree and connect it to the nearest node. If there is a collision, return to step 5.

2. 5.

4. Find the nearest neighbor within a defined radius near the newly generated node as a candidate to replace the parent node. Calculate the path cost from the nearest neighbor node to the starting point plus the path cost to each nearest neighbor node, and select the node with the minimum cost as the new parent node. 5.5 After reselecting the parent node, further optimize the cost of connections between random tree nodes; 5.6 For the welding robot arm, if it can successfully connect to the target position, it is considered that a valid path has been found. If a valid path cannot be found within a limited number of iterations or within a time limit, the sampling and path expansion are repeated. 5.7 Repeat steps 5.1-5.6 to generate the final welding path.

6. The weld inspection method based on surface structured light according to claim 1, characterized in that, During welding, after the workpiece is reconstructed in three dimensions in step (1), the weld template is matched with the three-dimensional data, and the three-dimensional coordinate set of the weld in the coordinate system of the surface structure light scanner is calculated using the ICP point cloud registration algorithm. Then, the process returns to step (4).

7. The weld inspection method based on surface structured light according to claim 6, characterized in that, The calculation process of the ICP point cloud registration algorithm is as follows: first, it uses the nearest neighbor loss function: ； Calculated T ,in, T Let be the rigid body transformation matrix. R Let be a rotation matrix. t It is a translation matrix. p i For the source point cloud collection, The nearest neighbor set of target point clouds; Then through the formula { X c, Y c, Z c }= T ×{ X m, Y m, Z m The set of three-dimensional coordinates of the weld seam in the coordinate system of the surface structured light scanner was calculated. X c, Y c, Z c },in{ X m, Y m, Z m } represents the set of three-dimensional coordinates of the weld in the weld template.

Images