Welding seam detection method based on surface structured light

Abstract

The invention belongs to the technical field of automatic welding, and particularly relates to a welding seam detection method based on surface structured light. Comprising the following steps: (1) three-dimensional reconstruction of a workpiece: performing three-dimensional scanning on the workpiece by using a surface structured light three-dimensional scanner to obtain a coded image II, a phase diagram and three-dimensional data of the workpiece, and reading the coded image II, the phase diagram and the three-dimensional data through a computer; and (2) identifying welding seam feature points: performing welding seam feature identification or manual welding seam marking on the coded image II to obtain a two-dimensional coordinate set of a welding seam on the coded image II. Two-dimensional information and three-dimensional information of a surface structured light scanner are utilized at the same time, an efficient welding seam detection method integrating two-dimensional feature recognition and three-dimensional coordinate mapping is achieved through triangulation mapping, the problem of welding seam detection without obvious structural features is solved, meanwhile, compared with a split type multi-sensor fusion method, the welding seam detection method has the advantages that the welding seam detection efficiency is improved, and the welding seam detection cost is reduced. Calibration errors between devices are eliminated, the welding seam detection precision is improved, and the detection process is simplified.

Description

Technical Field

[0001] The present invention belongs to the technical field of automated welding, and in particular relates to a weld detection method based on surface structured light. Background Art

[0002] Weld seam inspection is an important task in automated welding. Existing technologies mainly fall into three categories: A. Manual teaching; B. Physical touch detection; C. Machine vision detection. Manual teaching is time-consuming and labor-intensive and it is difficult to achieve flexible welding. Physical touch detection, as the name implies, is a contact measurement technology that is usually used in welding work on specific workpieces and is not universal. Visual inspection methods have become a popular research and development direction with their non-contact, high-precision, and flexible characteristics.

[0003] Patent 202411112212.9 discloses a drag-and-teach welding method for a welding robot based on line structured light, which obtains the weld trajectory information of the workpiece by methods such as manual drag-and-teach and line laser scanning. By processing the data of the line laser sensor, the weld position, morphology and other features are obtained. The welding path is planned using position features. Adaptive adjustments are made to parameters such as welding current and speed based on weld morphology features. This patent requires manual dragging when reconstructing the three-dimensional data of the workpiece, which is time-consuming. The proposed weld detection algorithm is performed in three-dimensional space and cannot handle welds without obvious structural features.

[0004] Patent 202410271448.0 discloses a robot flexible welding path planning method for non-standard workpieces. First, after acquiring and processing the surface point cloud information of the welding workpiece, a three-dimensional model of the welding workpiece is obtained; then the feature recognition algorithm is used to complete the weld extraction; then the weld posture and welding gun posture are determined; the meta-heuristic algorithm is used to determine the welding sequence of the weld; finally, an efficient path planning algorithm is used to quickly find a collision-free path to complete the welding path planning; the planned welding path parameters are used to guide the welding robot to weld. The present invention is aimed at multi-specification small-batch workpieces, combined with data processing methods such as point cloud three-dimensional reconstruction and weld feature extraction, and an efficient path planning method is used to quickly find a collision-free path for the welding robot, which can effectively improve the path quality and compress the planning time, while ensuring the continuity of the welding operation and the consistency of the welding quality. This patent also performs weld detection in three-dimensional space, and is also unable to deal with welds without obvious structural features.

[0005] Patent 202211489509.8 discloses a multi-source data fusion robot welding path recognition method, which uses a two-dimensional sensor (camera) to capture the workpiece image, selects feature positioning points in the image, and then uses a three-dimensional sensor (laser sensor) to assist in measuring the precise coordinates of the positioning points, and finally performs interpolation calculations to generate a complete welding path. This method uses two-dimensional and three-dimensional fusion measurements to improve the weld detection accuracy to a certain extent, but there are three problems: 1. Before each welding, the robot end needs to be driven to traverse all feature points, which affects the detection efficiency; 2. It is necessary to use cameras and laser sensors in conjunction, and there is additional system calibration work, and the calibration parameters have an impact on the detection accuracy; 3. The final generated complete welding path is obtained by interpolation, and the interpolation accuracy depends on the number of positioning points. Although more positioning points can improve the interpolation accuracy, since the robot end needs to traverse all feature points, it will lead to a more complicated detection process.

[0006] The current visual inspection method used in automated welding has a poor level of intelligence and requires manual dragging measurement or pre-calibration of features, making the weld inspection process more cumbersome; some existing technologies (such as patent 202211489509.8) propose methods for fusing two-dimensional and three-dimensional information. These methods use multiple sensors for measurement at the same time, introducing additional calibration problems between devices, and the calibration accuracy has a greater impact on the final inspection accuracy; there are also some methods that use three-dimensional sensors (such as laser scanners) to obtain three-dimensional data, and then identify welds in three-dimensional data. Usually, the method relies on the significant structure of the weld (such as the intersection line of two planes, and patent CN110449783B uses this method). In many cases, the weld structure is not significant. For example, in the field of automobile manufacturing, there are a lot of curved surface structure connection work. These welds without significant structures are very difficult to automatically identify in three-dimensional space. Summary of the invention

[0007] The purpose of the present invention is to provide a weld detection method based on surface structured light to solve the problems existing in the prior art.

[0008] The technical solution adopted by the present invention to solve its technical problem is:

[0009] A weld detection method based on surface structured light comprises the following steps:

[0010] (1) Three-dimensional reconstruction of the workpiece: Use a surface structured light three-dimensional scanner to perform three-dimensional scanning on the workpiece to obtain the workpiece's coded image II, phase map, and three-dimensional data, which are then read by a computer;

[0011] (2) Weld feature point recognition: Perform weld feature recognition or manual weld annotation on coded image II to obtain a two-dimensional coordinate set of the weld on coded image II;

[0012] (3) Three-dimensional coordinate mapping: According to the calibration parameters, phase map, and three-dimensional data of the surface structured light three-dimensional scanner, the two-dimensional coordinate set of the weld is mapped into a three-dimensional coordinate set of the weld based on the coordinate system of the surface structured light scanner through the triangulation formula;

[0013] (4) Hand-eye calibration: The three-dimensional coordinate set of the weld is converted from the surface structured light scanner coordinate system to the welding robot end coordinate system through the hand-eye calibration algorithm;

[0014] (5) Welding path planning: Use the RRT path planning algorithm to plan the welding path and drive the welding robot end to perform welding work in the current welding area. After welding is completed, return to step (1) to perform welding work in the next area until all welding areas are welded;

[0015] (6) Path correction: During the welding process, the weld path is corrected online again using a surface structured light 3D scanner.

[0016] Furthermore, the specific method of three-dimensional reconstruction of the workpiece in step (1) is to install a surface structured light three-dimensional scanner on the end of the welding robot so that it moves synchronously with the end of the welding robot. Before welding begins, the welding robot is driven to the starting point of the workpiece to be welded, and the surface structured light three-dimensional scanner is controlled to perform three-dimensional reconstruction of the workpiece. When the surface structured light scanner is working, its built-in projector is used to project a coded image I onto the workpiece, and its built-in camera is used to capture and obtain a coded image II, and the coded image II is decoded to obtain a phase map. At the same time, the surface structured light three-dimensional scanner generates three-dimensional data of the workpiece through its built-in algorithm, and the coded image II, the phase map, the three-dimensional data, and the calibration parameters of the surface structured light three-dimensional scanner are read by a computer.

[0017] Furthermore, the calibration parameters of the surface structured light 3D scanner in step (3) are: the internal parameters K of the camera c , the internal parameters K of the projector P , the external parameters between the camera and the projector [R,t];

[0018] The triangulation formula is:

[0019]

[0020] Among them, the three-dimensional coordinates {X P ,Y P ,Z P}Convert to the three-dimensional coordinates of the surface structured light scanner coordinate system {X c ,Y c ,Z c}, u' is the gray value of the corresponding position of {u,v} on the phase map, that is, the phase value, v' is an unknown number, r3 is the third row of data in R, t 3 is the third row of data of t, R is the rotation matrix, and t is the translation matrix.

[0021] Furthermore, the equation of the hand-eye calibration algorithm in step (4) is expressed as A i X=XB i , where A i and B i represents the relative motion between the end effector of the welding robot and the surface structured light scanner, X is the hand-eye calibration matrix to be solved, and it is solved based on a two-stage closed-form method. The specific steps are:

[0022] 4.1. and R X Respectively represent A i , B i and the X rotation component, and t X Respectively represented as A i , B i and the translation vector of X, so we get

[0023] 4.2. According to the matrix operation theorem, we get and

[0024] 4.3. Transform from step 4.2 to obtain

[0025] 4.4、 The equivalent rotation axis a i It is expressed as:

[0026]

[0027] 4.5. Through R X Establish and The equivalent rotation axis a i and b i The connection between i =R X b i , where b i for The equivalent rotation axis of

[0028] 4.6. Generate a sufficient number of equations through a two-stage closed-form solution method and solve R directly using standard linear algebra tools X All items in ;

[0029] For n>2, define the overconstrained linear equation A=R X B, where A=(a1 ,…a n ,a 1,1 ,…a i,j ,…a n,n ), B=(b 1 ,…b n ,b 1,1 ,…b i,j ,…b n,n ), and there is a i,j =a i ×a j and b i,j =b i ×b j ,i,j∈{1,…,n},i <j;

[0030] Result: R X The least squares solution of Among them B + =B T (BB T ) -1 represents the Moore-Penrose pseudoinverse of matrix B. If matrix B is full rank, then B + =B -1 .

[0031] Finally, by using the formula Solve for the translation matrix t X , thus completing the hand-eye calibration solution and obtaining the three-dimensional coordinate set {X r ,Y r ,Z r}.

[0032] Furthermore, the specific steps of step (5) RRT path planning algorithm are as follows:

[0033] 5.1. Determine the starting position, weld position and obstacle position of the welding robot's manipulator arm, where the obstacles are divided into environmental obstacles and the expected trajectory planned by the remaining manipulator arms. The starting position is used as the root node of the RRT random tree.

[0034] 5.2. Randomly sample a point in the workspace as the exploration target point, find the node closest to the sampling point in the random number, extend a distance from the nearest node to the sampling point, and create a new node;

[0035] 5.3. Check whether the new node collides with the obstacle. If not, add it to the random tree and connect it to the nearest node. If there is a collision, return to step 5.2.

[0036] 5.4. Find the nearest neighbor within the defined radius near the newly generated node as a candidate to replace the parent node. Calculate the path cost from the nearest neighbor node to the starting point plus the path cost to each nearest neighbor point, and select the node with the smallest cost as the new parent node;

[0037] 5.5. After reselecting the parent node, further optimize the cost of the connection between random tree nodes;

[0038] 5.6. For the welding robot arm, if it can successfully connect to the target position, it is considered that a valid path has been found. If a valid path cannot be found within a limited number of iterations or time, resampling and path extension are performed;

[0039] 5.7. Repeat steps 5.1-5.6 to generate the final welding path.

[0040] Furthermore, the specific method of path correction in step (6) is:

[0041] 6.1. Move the end of the welding robot to a short distance above the starting point of the weld. At this time, the three-dimensional coordinates {X c ,Y c ,Z c} is transformed into {X' c ,Y' c ,Z' c}, with the rigid body transformation matrix T m Indicates end movement, then {X' c ,Y' c ,Z' c =T m ×{X c ,Y c ,Z c};

[0042] 6.2. Use the scanner's built-in camera to take a second photo of the workpiece to obtain the coded image III. c ,Y' c ,Z' c}Remap to two-dimensional coordinates through inverse triangulation, that is, the inverse operation of the triangulation formula, denoted as Finding the remapped weld 2D coordinates The pixel with the smallest gray value within a certain range nearby is used as the corrected two-dimensional coordinate of the weld. The search algorithm is:

[0043]

[0044] Where (u,v) U Represents the neighborhood coordinate set of the two-dimensional coordinates u, v, I(u,v) represents the grayscale value at the two-dimensional coordinates u, v, u f ,yf is the corrected 2D coordinate of the weld, because the end position is moved by T m , Scanner parameters K c , K P , R, t are all known, and the corrected weld two-dimensional coordinates {u f ,v f The three-dimensional coordinates of the weld in the terminal coordinate system of the mobile welding robot after correction {X rf ,Y rf ,Z rf};

[0045] 6.3. After completing the welding of all corrected welds, repeat steps 6.1-6.2 until the welding of a whole weld is completed.

[0046] Further, after the three-dimensional coordinate mapping of step (3) is completed, a set of three-dimensional coordinates of the weld in a welding area of ​​the workpiece is obtained, and steps (1) to (3) are repeated to perform three-dimensional coordinate mapping of the weld in the next welding area, and this is repeated until the three-dimensional coordinate mapping of all welds on the workpiece is completed, and a weld template is obtained, or the operations of steps (1) to (3) are omitted, and the weld template is directly created from the CAD data;

[0047] During welding, after the three-dimensional reconstruction of the workpiece is performed through step (1), the weld template is matched with the three-dimensional data, and the three-dimensional coordinate set of the weld in the surface structured light scanner coordinate system is calculated using the ICP point cloud registration algorithm, and then the process returns to step (4).

[0048] Furthermore, the calculation process of the ICP point cloud registration algorithm is to first pass the nearest neighbor loss function:

[0049]

[0050] Calculate T, where T is the rigid transformation matrix, R is the rotation matrix, t is the translation matrix, and p i is the source point cloud set, is the target point cloud set of the nearest neighbors;

[0051] Then through the formula {X c ,Y c ,Z c}=T×{X m ,Y m ,Z m}Calculate the three-dimensional coordinate set of the weld based on the surface structured light scanner coordinate system {X c ,Y c ,Z c}, where {X m ,Y m ,Z m} is the three-dimensional coordinate set of the weld in the weld template.

[0052] The present invention has the following beneficial effects:

[0053] 1. The present invention simultaneously utilizes the two-dimensional information and three-dimensional information of the surface structured light scanner, and realizes an efficient weld detection method that integrates "two-dimensional feature recognition and three-dimensional coordinate mapping" through triangulation mapping. While solving the problem of weld detection without obvious structural features, compared with the split multi-sensor fusion method, it eliminates the calibration error between devices, improves the weld detection accuracy, and simplifies the detection process.

[0054] 2. Since the three-dimensional coordinates of all weld feature points are acquired automatically, high-density weld feature points can be extracted from the two-dimensional image, which not only improves the weld detection accuracy but also does not increase additional measurement work.

[0055] 3. An online weld path correction method based on weld image remapping is proposed, which further improves the weld detection accuracy while maintaining high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of Example 1 of the present invention.

[0057] Figure 2 It is a schematic diagram of data conversion relationship of the present invention.

[0058] Figure 3 It is a schematic diagram of the flow chart of Example 4 of the present invention.

[0059] Figure 4 It is a flow chart of Example 5 of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0061] Embodiment 1: This embodiment provides a weld detection method based on surface structured light, which is aimed at conventional weld detection methods for workpieces requiring higher precision.

[0062] like Figure 1-2 As shown, a weld detection method based on surface structured light includes the following steps:

[0063] (1) Three-dimensional reconstruction of the workpiece: Use a surface structured light three-dimensional scanner to perform three-dimensional scanning on the workpiece to obtain the workpiece's coded image II, phase map, and three-dimensional data, and read them through a computer. The specific method is as follows:

[0064] The surface structured light 3D scanner is installed on the end of the welding robot so that it moves synchronously with the end of the welding robot. Before welding begins, the welding robot is driven to the starting point of the workpiece to be welded, and the surface structured light 3D scanner is controlled to perform three-dimensional reconstruction of the workpiece. When the surface structured light scanner is working, its built-in projector is used to project the coded image I onto the workpiece, and its built-in camera is used to capture and obtain the coded image II, and the coded image II is decoded to obtain a phase map. At the same time, the surface structured light 3D scanner generates three-dimensional data of the workpiece through its built-in algorithm, and the coded image II, phase map, three-dimensional data, and calibration parameters of the surface structured light 3D scanner are read through a computer.

[0065] (2) Weld feature point recognition: Weld feature recognition is performed on coded image II to obtain a two-dimensional coordinate set of the weld on coded image II.

[0066] Feature recognition mainly relies on the fact that the grayscale value of the weld contour is generally lower than that of the surrounding area. Since the weld is always an area where material is missing, the weld has a richer representation in the coded image II. The grayscale of the weld is often lower than that of the surrounding area (because there is no material filling at the weld position). Based on this feature, the use of machine learning algorithms to automatically extract weld feature points has a higher extraction success rate. These feature points are a set of continuous low-grayscale pixels, represented by a two-dimensional coordinate set {u, v} of the image. The weld feature recognition method is an existing technology and will not be described in detail here.

[0067] (3) Three-dimensional coordinate mapping: Based on the calibration parameters, phase map, and three-dimensional data of the surface structured light three-dimensional scanner, the two-dimensional coordinate set of the weld is mapped into a three-dimensional coordinate set of the weld based on the coordinate system of the surface structured light scanner through the triangulation formula.

[0068] Different from the existing method of using multiple devices, since the two-dimensional image used for weld detection is the intermediate data of the three-dimensional scanner (taken by the camera inside the scanner), the built-in camera and projector of the surface structured light scanner have obtained high-precision calibration parameters using special equipment before leaving the factory, which can be read directly. The calibration parameters based on the surface structured light three-dimensional scanner are: the internal parameters K of the camera c , the internal parameters K of the projector P , external parameters between the camera and the projector [R,t];.

[0069] The triangulation formula is:

[0070]

[0071]

[0072] Among them, the three-dimensional coordinates {X P ,YP ,Z P}Convert to the three-dimensional coordinates of the surface structured light scanner coordinate system {X c ,Y c ,Z c}, u' is the gray value of the corresponding position of {u,v} on the phase map, that is, the phase value, v' is an unknown number, r 3 is the third row of data in R, t 3 is the third row of data of t, R is the rotation matrix, and t is the translation matrix.

[0073] (4) Hand-eye calibration: The hand-eye calibration algorithm is used to convert the three-dimensional coordinate set of the weld from the surface structured light scanner coordinate system to the welding robot end coordinate system.

[0074] The equation of the hand-eye calibration algorithm is expressed as A i X=XB i , where A i and B i represents the relative motion between the end effector of the welding robot and the surface structured light scanner, X is the hand-eye calibration matrix to be solved, and it is solved based on a two-stage closed-form method. The specific steps are:

[0075] 4.1. and R X Respectively represent A i , B i and the X rotation component, and t X Respectively represented as A i , B i and the translation vector of X, so we get

[0076] 4.2. According to the matrix operation theorem, we get and

[0077] 4.3. Transform from step 4.2 to obtain

[0078] 4.4、 The equivalent rotation axis a i It is expressed as:

[0079]

[0080] 4.5. Through R X Establish and The equivalent rotation axis a i and b i The connection between i =R X bi , where b i for The equivalent rotation axis of

[0081] 4.6. Generate a sufficient number of equations through a two-stage closed-form solution method and solve R directly using standard linear algebra tools X All items in ;

[0082] For n>2, define the overconstrained linear equation A=R X B, where A=(a 1 ,…a n ,a 1,1 ,…a i,j ,…a n,n ), B=(b 1 ,…b n ,b 1,1 ,…b i,j ,…b n,n ), and there is a i,j =a i ×a j andb i,j =b i ×b j ,i,j∈{1,…,n},i <j;

[0083] Result: R X The least squares solution of Among them B + =B T (BB T ) -1 represents the Moore-Penrose pseudoinverse of matrix B. If matrix B is full rank, then B + =B -1 .

[0084] Finally, by using the formula Solve for the translation matrix t X , thus completing the hand-eye calibration solution and obtaining the three-dimensional coordinate set {X r ,Y r ,Z r}.

[0085] (5) Welding path planning: Use the RRT path planning algorithm to plan the welding path and drive the welding robot end to perform welding work in the current welding area. After welding is completed, return to step (1) to perform welding work in the next area until all welding areas are welded.

[0086] The specific steps of the RRT path planning algorithm are:

[0087] 5.1. Determine the starting position, weld position and obstacle position of the welding robot's manipulator arm, where the obstacles are divided into environmental obstacles and the expected trajectory planned by the remaining manipulator arms. The starting position is used as the root node of the RRT random tree.

[0088] 5.2. Randomly sample a point in the workspace as the exploration target point, find the node closest to the sampling point in the random number, extend a distance from the nearest node to the sampling point, and create a new node;

[0089] 5.3. Check whether the new node collides with the obstacle. If not, add it to the random tree and connect it to the nearest node. If there is a collision, return to step 5.2.

[0090] 5.4. Find the nearest neighbor within the defined radius near the newly generated node as a candidate to replace the parent node. Calculate the path cost from the nearest neighbor node to the starting point plus the path cost to each nearest neighbor point, and select the node with the smallest cost as the new parent node;

[0091] 5.5. After reselecting the parent node, further optimize the cost of the connection between random tree nodes;

[0092] 5.6. For the welding robot arm, if it can successfully connect to the target position, it is considered that a valid path has been found. If a valid path cannot be found within a limited number of iterations or time, resampling and path extension are performed;

[0093] 5.7. Repeat steps 5.1-5.6 to generate the final welding path.

[0094] (6) Path correction: During the welding process, the weld path is corrected online again using a surface structured light 3D scanner.

[0095] The specific method of path correction is:

[0096] 6.1. Move the end of the welding robot to a short distance above the starting point of the weld. At this time, the three-dimensional coordinates {X c ,Y c ,Z c} is transformed into {X' c ,Y' c ,Z' c}, with the rigid body transformation matrix T m Indicates end movement, then {X' c ,Y' c ,Z' c =T m ×{X c ,Y c ,Z c};

[0097] 6.2. Use the scanner's built-in camera to take a second photo of the workpiece to obtain the coded image III. c ,Y' c ,Z' c}Remap to two-dimensional coordinates through inverse triangulation, that is, the inverse operation of the triangulation formula, denoted as Finding the remapped weld 2D coordinates The pixel with the smallest gray value within a certain range nearby is used as the corrected two-dimensional coordinate of the weld. The search algorithm is:

[0098]

[0099] Where (u,v) U Represents the neighborhood coordinate set of the two-dimensional coordinates u, v, I(u,v) represents the grayscale value at the two-dimensional coordinates u, v, u f ,y f is the corrected 2D coordinate of the weld, because the end position is moved by T m , Scanner parameters K c , K P , R, t are all known, and the corrected weld two-dimensional coordinates {u f ,v f The three-dimensional coordinates of the weld in the terminal coordinate system of the mobile welding robot after correction {X rf ,Y rf ,Z rf};

[0100] 6.3. After completing the welding of all corrected welds, repeat steps 6.1-6.2 until the welding of a whole weld is completed.

[0101] Embodiment 2:

[0102] This embodiment provides a weld detection method based on surface structured light, which is aimed at workpieces that do not require high welding accuracy. The method steps are basically the same as those in Example 1, except that the path correction in step (6) is not used during the operation.

[0103] Embodiment 3:

[0104] This embodiment provides a weld detection method based on surface structured light, which is aimed at the situation where the weld features are very unclear (the grayscale difference is small) and it is impossible to automatically perform two-dimensional feature recognition. The method steps are basically the same as those in Example 1, except that in step (2), the welds are manually annotated on the coded image II to obtain a two-dimensional coordinate set of the weld on the coded image II.

[0105] Embodiment 4:

[0106] This embodiment provides a weld detection method based on surface structured light, which is aimed at welding of large quantities of workpieces. The method steps are basically the same as those in Embodiment 1, except that:

[0107] like Figure 3 As shown, first complete the three-dimensional coordinate mapping of the weld in one welding area of ​​the workpiece according to steps (1) to (3), then repeat steps (1) to (3) to map the three-dimensional coordinates of the weld in the next welding area, and repeat this process until the three-dimensional coordinate mapping of all welds on the workpiece is completed, and a weld template is obtained. Since the welds of all workpieces in the same batch are the same, only one workpiece needs to be used to create a welding template.

[0108] During welding, after the three-dimensional reconstruction of the workpiece is performed through step (1), the weld template is matched with the three-dimensional data, and the three-dimensional coordinate set of the weld in the surface structured light scanner coordinate system is calculated using the ICP point cloud registration algorithm. Then, the process returns to step (4) and continues with steps (4) to (5) shown in Example 1. Step (6) can be selected according to the welding accuracy requirements.

[0109] The calculation process of the ICP point cloud registration algorithm is to first pass the nearest neighbor loss function:

[0110]

[0111] Calculate T, where T is the rigid transformation matrix, R is the rotation matrix, t is the translation matrix, and p i is the source point cloud set, is the target point cloud set of the nearest neighbors;

[0112] Then through the formula {X c ,Y c ,Z c}=T×{X m ,Y m ,Z m}Calculate the three-dimensional coordinate set of the weld based on the surface structured light scanner coordinate system {X c ,Y c ,Z c}, where {X m ,Y m ,Z m} is the three-dimensional coordinate set of the weld in the weld template.

[0113] Embodiment 5:

[0114] This embodiment provides a weld detection method based on surface structured light, which is aimed at welding of large quantities of workpieces and has CAD data of the welded workpieces. The method steps are basically the same as those in Embodiment 4, except that:

[0115] like Figure 4 As shown, the weld template is extracted from the CAD data. There is no need to repeatedly cycle through steps (1) to (3) to create the weld template. The CAD data of the workpiece can be directly read using a computer, and the remaining steps are implemented according to Example 3.

[0116] The above embodiments are only for describing the preferred implementation of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solution of the present invention should fall within the protection scope of the present invention.

[0117] The techniques, shapes, and structural parts not described in detail in the present invention are all well-known techniques.

Claims

1. A weld detection method based on surface structured light, characterized in that: The following steps are involved: (1) Three-dimensional reconstruction of the workpiece: Use a surface structured light three-dimensional scanner to perform three-dimensional scanning on the workpiece to obtain the workpiece's coded image II, phase map, and three-dimensional data, which are then read by a computer; (2) Weld feature point recognition: Perform weld feature recognition or manual weld annotation on coded image II to obtain a two-dimensional coordinate set of the weld on coded image II; (3) Three-dimensional coordinate mapping: According to the calibration parameters, phase map, and three-dimensional data of the surface structured light three-dimensional scanner, the two-dimensional coordinate set of the weld is mapped into a three-dimensional coordinate set of the weld based on the coordinate system of the surface structured light scanner through the triangulation formula; (4) Hand-eye calibration: The three-dimensional coordinate set of the weld is converted from the surface structured light scanner coordinate system to the welding robot end coordinate system through the hand-eye calibration algorithm; (5) Welding path planning: Use the RRT path planning algorithm to plan the welding path and drive the welding robot end to perform welding work in the current welding area. After welding is completed, return to step (1) to perform welding work in the next area until all welding areas are welded; (6) Path correction: During the welding process, the weld path is corrected online again using a surface structured light 3D scanner.

2. The weld detection method based on surface structured light according to claim 1, characterized in that: The specific method for three-dimensional reconstruction of the workpiece in step (1) is to install a surface structured light three-dimensional scanner on the end of the welding robot so that it moves synchronously with the end of the welding robot. Before welding begins, the welding robot is driven to the starting point of the workpiece to be welded, and the surface structured light three-dimensional scanner is controlled to perform three-dimensional reconstruction of the workpiece. When the surface structured light scanner is working, its built-in projector is used to project a coded image I onto the workpiece, and its built-in camera is used to capture and obtain a coded image II, and the coded image II is decoded to obtain a phase map. At the same time, the surface structured light three-dimensional scanner generates three-dimensional data of the workpiece through its built-in algorithm, and the coded image II, the phase map, the three-dimensional data, and the calibration parameters of the surface structured light three-dimensional scanner are read by a computer.

3. The weld detection method based on surface structured light according to claim 2, characterized in that: The calibration parameters of the surface structured light 3D scanner in step (3) are: the internal parameters K of the camera c , the internal parameters K of the projector P , the external parameters between the camera and the projector [R,t]; The triangulation formula is: Among them, the three-dimensional coordinates {X P ,Y P ,Z P }Convert to the three-dimensional coordinates of the surface structured light scanner coordinate system {X c , Y c , Z c }, u′ is the gray value of the corresponding position of {u, v} on the phase map, that is, the phase value, v′ is an unknown number, r3 is the third row of data of R, t3 is the third row of data of t, R is the rotation matrix, and t is the translation matrix.

4. The weld detection method based on surface structured light according to claim 3 is characterized in that the equation of the hand-eye calibration algorithm in step (4) is expressed as A i X=XB i , where A i and B i represents the relative motion between the end effector of the welding robot and the surface structured light scanner, X is the hand-eye calibration matrix to be solved, and it is solved based on a two-stage closed-form method. The specific steps are: 4.

1. and R X Respectively represent A i , B i and the X rotation component, and t X Respectively represented as A i , B i and the translation vector of X, so we get 4.

2. According to the matrix operation theorem, we get and 4.

3. Transform from step 4.2 to obtain 4.4、 The equivalent rotation axis a i It is expressed as: 4.

5. Through R X Establish and The equivalent rotation axis a i and b i The connection between i =R X b i , where b i for The equivalent rotation axis of 4.

6. Generate a sufficient number of equations through a two-stage closed-form solution method and solve R directly using standard linear algebra tools X All items in ; For n>2, define the overconstrained linear equation A=R X B, where A=(a1,…a n , a 1，1 , …a i，j , …a n，n ), B=(b1,…b n , b 1,1 , …b i，j , …b n，n ), and there is a i，j =a i ×a j andb i，j =b i ×b j , i, j∈{1,...,n}, i<j; Result: R X The least squares solution of Among them B + =B T (BB T ) -1 represents the Moore-Penrose pseudoinverse of matrix B. If matrix B is full rank, then B + =B -1 . Finally, by using the formula Solve for the translation matrix t X , thus completing the hand-eye calibration solution and obtaining the three-dimensional coordinate set {X r , Y r , Z r }.

5. The weld detection method based on surface structured light according to claim 4, characterized in that: The specific steps of the RRT path planning algorithm in step (5) are as follows: 5.

1. Determine the starting position, weld position and obstacle position of the welding robot's manipulator arm, where the obstacles are divided into environmental obstacles and the expected trajectory planned by the remaining manipulator arms. The starting position is used as the root node of the RRT random tree. 5.

2. Randomly sample a point in the workspace as the exploration target point, find the node closest to the sampling point in the random number, extend a distance from the nearest node to the sampling point, and create a new node; 5.

3. Check whether the new node collides with the obstacle. If not, add it to the random tree and connect it to the nearest node. If there is a collision, return to step 5.

2. 5.

4. Find the nearest neighbor within the defined radius near the newly generated node as a candidate to replace the parent node. Calculate the path cost from the nearest neighbor node to the starting point plus the path cost to each nearest neighbor point, and select the node with the smallest cost as the new parent node; 5.

5. After reselecting the parent node, further optimize the cost of the connection between random tree nodes; 5.

6. For the welding robot arm, if it can successfully connect to the target position, it is considered that a valid path has been found. If a valid path cannot be found within a limited number of iterations or time, resampling and path extension are performed; 5.

7. Repeat steps 5.1-5.6 to generate the final welding path.

6. The weld detection method based on surface structured light according to claim 1, characterized in that: The specific method of path correction in step (6) is: 6.

1. Move the end of the welding robot to a short distance above the starting point of the weld. At this time, the three-dimensional coordinates {X c ,Y c ,Z c } is transformed into {X′ c ,Y′ c ,Z′ c }, with the rigid body transformation matrix T m represents the end movement, then {X′ c ,Y′ c ,Z′ c =T m ×{X c ,Y c ,Z c }; 6.

2. Use the scanner's built-in camera to take a second photo of the workpiece to obtain the coded image III. c ,Y′ c ,Z′ c }Remap to two-dimensional coordinates through inverse triangulation, that is, the inverse operation of the triangulation formula, denoted as Finding the remapped weld 2D coordinates The pixel with the smallest gray value within a certain range nearby is used as the corrected two-dimensional coordinate of the weld. The search algorithm is: Where (u,v) U Represents the neighborhood coordinate set of the two-dimensional coordinates u, v, I(u,v) represents the grayscale value at the two-dimensional coordinates u, v, u f ,y f is the corrected 2D coordinate of the weld, because the end is moved to position T m , Scanner parameters K c , K P , R, t are all known, and the corrected weld two-dimensional coordinates {u f ,v f The three-dimensional coordinates of the weld in the terminal coordinate system of the mobile welding robot after correction {X rf ,Y rf ,Z rf }; 6.

3. After completing the welding of all corrected welds, repeat steps 6.1-6.2 until the welding of a whole weld is completed.

7. The weld detection method based on surface structured light according to claim 1, characterized in that: After the three-dimensional coordinate mapping in step (3) is completed, a set of three-dimensional coordinates of the weld in a welding area of ​​the workpiece is obtained, and steps (1) to (3) are repeated to perform three-dimensional coordinate mapping of the weld in the next welding area, and this is repeated until the three-dimensional coordinate mapping of all welds on the workpiece is completed to obtain a weld template, or the operations of steps (1) to (3) are omitted, and the weld template is created directly from the CAD data; During welding, after the three-dimensional reconstruction of the workpiece is performed through step (1), the weld template is matched with the three-dimensional data, and the three-dimensional coordinate set of the weld in the surface structured light scanner coordinate system is calculated using the ICP point cloud registration algorithm, and then the process returns to step (4).

8. The weld detection method based on surface structured light according to claim 7, characterized in that: The calculation process of the ICP point cloud registration algorithm is to first use the nearest neighbor loss function: Calculate T, where T is the rigid transformation matrix, R is the rotation matrix, t is the translation matrix, and p i is the source point cloud set, is the target point cloud set of the nearest neighbors; Then through the formula {X c , Y c , Z c }=T×{X m , Y m , Z m }Calculate the three-dimensional coordinate set of the weld based on the surface structured light scanner coordinate system {X c , Y c , Z c }, where {X m , Y m , Z m } is the three-dimensional coordinate set of the weld in the weld template.

Images