Automatic welding method and system based on binocular vision

Abstract

The invention provides an automatic welding method and system based on binocular vision. The automatic welding method comprises the steps that S1, data collection is conducted, specifically, a welding seam of a welded workpiece is scanned through a binocular camera and line laser; s2, the collected original data are transmitted to an industrial computer in real time to be analyzed and processed, and welding seam point cloud data are obtained; s3, creating a target welding seam template according to the shape of the welding seam, and generating template point cloud data according to the welding seam template; s4, calculating the welding seam point cloud data to obtain the center of gravity of the welding seam point cloud and the template point cloud; and S5, template comparison is conducted, the gravity center of the template point cloud and the gravity center of the welding seam point cloud are made to coincide, registration errors between the point clouds are reduced through continuous iteration, an optimal transformation matrix is found, and finally the welding seam template is subjected to slicing operation to obtain an optimal welding track. The method has strong adaptive capacity, can flexibly cope with various complex working conditions, meets diversified welding requirements, and provides strong technical support for transformation and upgrading of the modern manufacturing industry.

Description

Technical Field

[0001] The invention belongs to the technical field of welding processing research and application development, and specifically relates to an automated welding method and system based on binocular vision. Background Art

[0002] Welding is one of the indispensable key technologies in modern manufacturing. The welding process is a process of bonding two or more adjacent metal parts into a whole by means of arc heating. Traditional welding methods have many shortcomings. The process mainly relies on skilled workers with welding machine experience to operate. On the one hand, it requires high operating level and technical proficiency of welders; on the other hand, a large amount of high-temperature splashing melt, dust and ultraviolet rays will be generated during the welding process. Therefore, the welding working environment is very harsh, which is very easy to cause harm to welders.

[0003] To solve the above problems, researchers have developed a robot automatic welding system based on two-dimensional vision, which uses industrial cameras to collect the morphology of the welding workpiece, analyze the welding trajectory and guide the industrial robot to perform welding operations. The robot automatic welding system using two-dimensional vision not only realizes the automation of the welding process, but also reduces the occurrence of worker injuries during the welding process. The automation of the welding process is reflected in the use of two-dimensional vision technology to plan the welding path, ensure that the welding head welds according to the predetermined route, and detect defects in the weld, such as pores, cracks, etc., to ensure the quality of welding.

[0004] Since two-dimensional vision can only obtain two-dimensional information on the surface of the welded workpiece, such as weld identification of a plate weld located on a plane, or the use of two-dimensional vision to track welds and other processes. The advantage of two-dimensional vision technology is that it can quickly process a large amount of image data during the welding process, is suitable for high-speed production lines, and is relatively easy to implement and maintain. However, for complex three-dimensional workpieces, such as welding of non-planar workpieces such as pipes, curved surfaces, and spherical surfaces, it is difficult to identify and track welds using two-dimensional vision, which affects the quality of the welds.

[0005] Therefore, it is an urgent problem to develop a weld recognition method for three-dimensional workpieces, which can be used for the recognition, welding tracking and optimization of welding trajectories of three-dimensional workpieces. Summary of the invention

[0006] In view of this, an object of the present invention is to provide an automated welding method and system based on binocular vision, so as to solve the problem that weld shapes vary during welding and it is difficult to accurately identify weld positions.

[0007] The technical solution of the present invention is: an automated welding method based on binocular vision, comprising:

[0008] S1: Data acquisition: Scan the weld of the welded workpiece through a binocular camera and a line laser;

[0009] S2: The collected raw data is transmitted to the industrial computer in real time for analysis and processing to obtain the weld point cloud data;

[0010] S3: creating a target weld template according to the shape of the weld, and generating template point cloud data according to the weld template;

[0011] S4: Calculate the weld point cloud data to obtain the center of gravity of the weld point cloud and the template point cloud;

[0012] S5: Perform template comparison to make the center of gravity of the template point cloud and the weld point cloud coincide, and reduce the registration error between the point clouds through continuous iteration to find the optimal transformation matrix. Finally, slice the weld template to obtain the optimal welding trajectory.

[0013] S6: Send the welding trajectory to the six-degree-of-freedom robotic arm through the computer, and control the robotic arm to perform welding.

[0014] Preferably, the data collection in S1 includes:

[0015] The weld is scanned using a line laser. The laser beam emitted by the line laser projects a bright and continuous laser line segment on the weld surface. The laser line segment is clearly visible in the camera field of view of the binocular camera. When the laser line segment scans a flat surface, the line segment captured by the camera is a straight line segment; when the laser line segment scans an uneven weld, the line segment is deformed and transformed from a straight line segment to a broken line segment.

[0016] The two industrial cameras of the binocular camera are used as image capture devices, which synchronously capture the images of the laser line segments in the weld area from two different positions on the left and right, and then transmit the collected raw data to the industrial computer in real time for analysis and processing.

[0017] Preferably, S2 includes:

[0018] S21: traverse each row of the left and right views scanned by the binocular camera, calculate the pixel position difference distance of the laser light of the left and right views in the row, and calculate the disparity value distance of each pixel of the left and right images to generate a disparity map;

[0019] S22: Calculate the depth value of the corresponding pixel point, set the distance between the center lines of the two cameras as BASE_LINE, and the depth value of each pixel represents its actual distance from the midpoint of the two cameras Z = (fx×BASE_LINE) / distance, where fx is the focal length;

[0020] S23: Analyze all pixel points corresponding to the left and right views to obtain an image containing the position (x, y) and depth value z of each pixel point; the pixel coordinates and three-dimensional coordinates are calculated through the formula: X = ((x - c x ) × Z) / fx; Y = ((y - c y ) × Z) / fx, where c x and c y are camera constants, and so on for the conversion of each pixel point and three-dimensional coordinates to obtain an image containing the spatial coordinates of each pixel point, that is, a three-dimensional point cloud;

[0021] S24: Convert the two-dimensional pixel position and depth value into actual coordinates in three-dimensional space to generate weld point cloud data; remove the redundant points and only retain the weld point cloud data.

[0022] Preferably, S3 includes: Import the STL file of the weld template, discretize it to obtain the template point cloud, and the algorithm process of discretization is as follows:

[0023] According to the three-dimensional model of the weld, create a spatial rectangular coordinate system xyz, and the x, y, and z axes respectively represent the position information of the model in three different directions;

[0024] Vertically cut the template with a plane parallel to the yz plane to obtain a cross-section ABC of the model, unfold the cross-section in the yz plane, extract the edge information of the cross-section, and obtain the outer contour of the cross-section;

[0025] Divide the cross-section with equally spaced straight lines parallel to the z axis (0 < y < y', where y' is the maximum length of the cross-section), extract the points where the straight lines intersect the cross-section contour, and obtain the point cloud data of the ABC cross-section, that is, the point cloud of one cross-section of the weld model. Intercept the model with multiple planes parallel to the yz plane and perform the same operation to finally obtain the template point cloud.

[0026] Preferably, in S4, the centroid coordinates (W x , W y , W z ) of the weld point cloud and the centroid coordinates (C x , C y , C z ) of the template point cloud, where the centroid is the weighted average of all point coordinates of the weld point cloud or the template point cloud.

[0027] Preferably, in S5, move each point of the template point cloud so that the centroid coordinates (C x , C y , C z ) of the template point cloud coincide with the centroid coordinates (W x , W y , W z)Coincide in position. The method is as follows: Calculate the deviation of the centroid coordinates of the two point clouds, Δx = W x -C x , Δy = W y -C y , Δz = W z -C z , and add the deviation of the centroid to the coordinates (x i , y i , z i ) of each point in the template point cloud. That is, the new coordinates of each point in the template point cloud are (x i +Δx, y i +Δy, z i +Δz);

[0028] Traverse all points in the weld point cloud to find the point Pw that is farthest from the centroid W; Rotate the template point cloud around the x, y, and z axes respectively. The rotation angles around the x, y, and z axes are (θ, δ, γ), and the value ranges of θ, δ, and γ are (0 - 360°). Each time θ, δ, and γ are rotated, they increase by 1° respectively;

[0029] Set a variable d0 to infinity. Traverse all the rotation angles around the x, y, and z axes to obtain the new coordinates of the template point cloud, and find the point Pc in the new template point cloud that is farthest from the centroid C. Calculate the distance d1 between Pw and Pc. If d1 < d0, record the new rotation matrix (θ, δ, γ), and assign d0 to d1. After n loops, the minimum rotation matrix (θ, δ, γ) will be obtained, where the value of n is 360 * 360 * 360;

[0030] Apply the rigid body transformation composed of the calculated translation matrix (W x -C x , W y -C y , W z -C z ) and the rotation matrix (θ, δ, γ) to the template point cloud to obtain the new position of the template point cloud and align the template point cloud with the weld point cloud.

[0031] Preferably, in S5, slicing the weld template to obtain the optimal welding trajectory includes:

[0032] Import the STL format file of the weld template, use the plane parallel to the x - y plane as the slicing plane, and the equation of the slicing plane is z = i * tk. The value of i is 1 - n, n is an integer, and n multiplied by the welding layer thickness is equal to the height of the weld template STL model; tk is the welding layer thickness;

[0033] The template is cut by the slicing plane, and all the triangular facets of the weld template STL model are traversed. If the three points of the triangular facet are not all on the same side of the tangent plane, the intersection line of the tangent plane and the triangular facet is calculated. The method is: take the three sides of the triangle as line segments, and construct the equations of the straight lines passing through the three side segments of the triangle respectively;

[0034] Combine the equation of the line and the equation of the tangent plane z = i * tk to find the intersection of the line and the plane; if the intersection is between the two endpoints of the edge of the triangle patch, then the point is the intersection of the triangle patch and the tangent plane;

[0035] The intersection points of the other two sides of the triangular face are obtained in the same way. The intersection points of all the triangular facets on the STL model and the cutting plane are obtained, and the above points are connected in sequence to obtain the contour line of the cutting plane.

[0036] The STL model is sliced ​​to obtain the contour of each slice, the contour being the edge of the welding track, and the contour area is filled according to the track to obtain the welding track of the contour;

[0037] On the plane of the obtained contour, create a series of parallel lines intersecting with the contour, each parallel line has two intersection points with the contour, take one of the intersection points as the welding start point and the other intersection point as the welding end point, and construct the welding filling trajectory within the contour.

[0038] The present invention also provides a control system for the above method, including a line laser, a binocular camera, a welding workpiece, a six-degree-of-freedom robotic arm, and an industrial computer, wherein the binocular camera is composed of two industrial cameras, and the binocular camera and the line laser are installed on the six-degree-of-freedom robotic arm.

[0039] The present invention provides a weld seam recognition method for three-dimensional workpieces, which integrates three-dimensional vision technology with welding technology to achieve accurate positioning and recognition of welded workpieces. It can be used for the recognition of three-dimensional workpiece welds, tracking welding and optimization of welding trajectories, significantly improving the accuracy and efficiency of welding and reducing the damage caused to workers during the welding process. In addition, the welding robot based on three-dimensional vision has strong adaptive capabilities, can flexibly respond to various complex working conditions, meet diverse welding needs, and provide strong technical support for the transformation and upgrading of modern manufacturing industry.

[0040] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 A schematic diagram of the process of the automated welding method based on binocular vision provided by the present invention;

[0044] Figure 2 It is a left and right view of the embodiment of the present invention containing only laser light;

[0045] Figure 3 A schematic diagram for generating weld point cloud data of the present invention;

[0046] Figure 4 This is a schematic diagram of template point cloud generation according to the present invention. DETAILED DESCRIPTION

[0047] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present invention. Instead, they are merely examples of systems consistent with some aspects of the present invention as detailed in the appended claims.

[0048] The present invention provides an automatic welding method based on binocular vision, comprising:

[0049] Through binocular vision camera and line laser scanning, the weld point cloud data is obtained;

[0050] A target weld template is created according to the shape of the weld, and template point cloud data is generated according to the weld template; the weld point cloud data is calculated to obtain the center of gravity of the weld point cloud and the template point cloud.

[0051] When performing template comparison operations, an algorithm is used to make the center of gravity of the template point cloud and the weld point cloud coincide, and the registration error between the point clouds is reduced through continuous iteration to find the optimal transformation matrix. Finally, the weld template is sliced ​​to obtain the optimal welding trajectory.

[0052] The system for implementing the above-mentioned is composed of a line laser 1, a binocular camera 2, a welding workpiece 3, a six-degree-of-freedom robot 4, and an industrial computer control system, and realizes automatic welding of welds through a three-dimensional visual recognition method based on template comparison.

[0053] During welding, a control system is used to control the six-degree-of-freedom robotic arm to move to the starting point of the workpiece 3 to be welded, and the binocular camera 2 and the line laser 1 are turned on to scan the weld of the workpiece 3 to be welded; the camera transmits the collected information data to the industrial computer, and the weld feature is extracted and the weld template is compared through the template comparison algorithm, and then the optimal welding trajectory is generated by slicing, and finally the six-degree-of-freedom robotic arm is controlled by the control system to perform welding.

[0054] Specifically, the steps of the three-dimensional visual recognition method based on template comparison are as follows:

[0055] 1. Data Collection

[0056] 1. A line laser (1) is used to scan the weld. The laser beam emitted by the line laser (1) projects a bright and continuous laser line segment on the weld surface. The line segment is clearly visible in the camera field of view. When the laser line segment scans a flat surface, the line segment captured by the camera is a straight line segment; when the laser line segment scans an uneven weld, the line segment will deform from a straight line segment to a broken line segment. At the same time, a binocular camera (2) is set as an image capture device to synchronously capture images of the laser line segment in the weld area from two different positions on the left and right, and then the collected raw data is transmitted in real time to an industrial computer (5) for analysis and processing.

[0057] 2. For an image captured by camera (2), for example, a photo with a size of 2048 in width and 1024 in height, each pixel point can be represented by the pixel coordinates (i, j) and the pixel value of the point, wherein i is the horizontal coordinate of the pixel, with a value of 0-2048, and j is the vertical coordinate of the pixel, with a value of 0-1024. The data format of a color image can be represented by the pixel coordinates (i, j) and the three pixel values ​​of red (R), green (G), and blue (B) of each pixel point. Each color pixel point of the color image can be represented by (i, j, R(i, j), G(i, j), B(i, j)); the data format of a grayscale image can be represented by the pixel coordinates (i, j) and the grayscale pixel value (the grayscale value is Pixel_Value) of each pixel point. Each grayscale pixel point of the grayscale image can be represented by (i, j, Pixel_Value(i, j)).

[0058] 3. Convert all color pixels on the color image into grayscale pixels. For example, convert the color pixel at position i, j into grayscale pixel, and calculate the grayscale value of the pixel according to the weighted formula Pixel_Value(i, j) = 0.299×R(i, j) + 0.587×GR(i, j) + 0.114×BR(i, j). Traverse the grayscale images taken by the left and right cameras in the order of rows first and columns later. For example, if the current row is i, analyze the pixel value of each pixel in the row. If the pixel value Pixel_Value(i, j) is 1, accumulate the pixel value of the pixel and the pixel column coordinate j, that is, the total pixel value: Sum_Pixel_Value = Sum_Pixel_Value + Pixel_Value(i, j); total pixel columns: Sum_Col = Sum_Col + j.

[0059] After counting the pixel coordinates of each row, the central light column coordinates of the row are obtained using the central light column coordinates: Col_Laser = Sum_Pixel_Value / Sum_Col. The coordinates of the central light can be expressed as: (i, Col_Laser, 1), and the pixel value of the row that is not equal to the Col_Laser column is 0.

[0060] like Figure 2 As shown, the images taken by the left and right cameras are processed according to the above analysis method to obtain left and right views containing only laser light.

[0061] 2. Point Cloud Generation and Processing

[0062] 1. Traverse each row in the left and right views above and calculate the pixel position difference distance (also called horizontal displacement) of the laser light of the left and right views in the row. For example, the column of the point with a pixel value of 1 in the left image is j. L , and the column for the point with pixel value 1 in the right figure is j R , distance = j R -j L . Calculate the disparity distance of each pixel in the left and right images to generate a disparity map.

[0063] 2. Calculate the depth value of the corresponding pixel point, set the distance between the center lines of the two cameras as BASE_LINE, and the depth value of each pixel represents its actual distance from the midpoint of the two cameras Z = (fx×BASE_LINE) / distance. fx is the focal length.

[0064] 3. Analyze all the pixels corresponding to the left and right views to obtain an image containing the position (x, y) and depth value z of each pixel. The pixel coordinates and three-dimensional coordinates can be calculated by the formula: X = ((xc x)×Z) / fx; Y = ((y - c y )×Z) / fx. Where cx and cy are constants of the camera. By converting each pixel point and three-dimensional coordinate according to the above method, the spatial coordinates of each pixel point can be obtained, that is, the three-dimensional point cloud.

[0065] 4. Convert the two-dimensional pixel position and depth value into the actual coordinates in three-dimensional space to generate the weld point cloud data. Remove the redundant points and only retain the weld point cloud data for subsequent template comparison. As Figure 3 shown.

[0066] III. Template Comparison

[0067] 1. Import the STL file of the weld template, discretize the template to obtain the template point cloud, and the algorithm process of discretization is as follows: As Figure 4 shown:

[0068] The weld template is as shown in (a). According to the three-dimensional model of the weld, create a spatial rectangular coordinate system xyz, and the x, y, and z axes respectively represent the position information of the model in three different directions. Vertically cut the template (b) with a plane parallel to the yz plane to obtain a cross-section ABC of the model (c), expand the cross-section in the yz plane (d), extract the edge information of the cross-section, and obtain the outer contour of the cross-section. Divide the cross-section with equally spaced straight lines parallel to the z axis (0 < y < y', where y' is the maximum length of the cross-section), extract the intersection points of the straight lines and the cross-section contour, and obtain the point cloud data of the ABC cross-section, that is, the point cloud of one cross-section of the weld model (e). Intercept the model with multiple planes parallel to the yz plane and perform the same operation to finally obtain the template point cloud (f).

[0069] 2. Find the centroid coordinates (W x , W y , W z ) of the weld point cloud and the centroid coordinates (C x , C y , C z ) of the template point cloud. The centroid is the weighted average of all point coordinates of the weld point cloud or the template point cloud. For example: there are n points in the point cloud, and the coordinates of each point are (x i , y i , z i ), where the value range of i is 1 to n. After accumulating the x coordinates of the n points in the point cloud and dividing by the total number of points n in the point cloud, the x coordinate C x of the centroid is obtained. The y coordinate C y of the centroid is calculated by the same method, and the z coordinate C z of the centroid is obtained. The centroid coordinates (W x , W y , Wz ) Move each point of the template point cloud so that the centroid coordinates (C x , C y , C z ) of the template point cloud coincide with the centroid coordinates (W x , W y , W z ) of the weld point cloud. The method is as follows: First, calculate the centroid coordinate deviation of the two point clouds, Δx = W x - C x , Δy = W y - C y , Δz = W z - C z . Add the deviation of the centroid to the coordinates (x i , y i , z i ) of each point in the template point cloud. That is, the new coordinates of each point in the template point cloud are (x i + Δx, y i + Δy, z i + Δz).

[0070] 3. Traverse all points in the weld point cloud to find the point Pw that is farthest from the centroid W. Rotate the template point cloud around the x, y, and z axes respectively. The rotation angles around the x, y, and z axes are (θ, δ, γ), and the value ranges of θ, δ, and γ are (0 - 360°). Each time θ, δ, and γ are rotated, they increase by 1° respectively. Set a variable d0 to infinity. Traverse all the rotation angles around the x, y, and z axes to obtain the new coordinates of the template point cloud, and find the point Pc in the new template point cloud that is farthest from the centroid C. Calculate the distance d1 between Pw and Pc. If d1 < d0, record the new rotation matrix (θ, δ, γ), and assign d0 to d1. After n loops, the minimum rotation matrix (θ, δ, γ) will be obtained, where the value of n is 360 * 360 * 360.

[0071] 4. Apply the rigid body transformation composed of the calculated translation matrix (W x - C x , W y - C y , W z - C z ) and the rotation matrix (θ, δ, γ) to the template point cloud to obtain the new position of the template point cloud and align the template point cloud with the weld point cloud.

[0072] IV. Slice Planning for Welding Trajectory

[0073] After the template comparison operation is performed and the comparison results meet the requirements, the weld template is sliced ​​through a specific algorithm. Slicing is to divide the 3D model into a series of continuous 2D contours, which represent the cross-sections that the welding gun needs to weld layer by layer.

[0074] 1. Import the weld template STL format file, use the plane parallel to the xy plane as the slicing plane, the equation of the slicing plane is z=i*tk, i is 1-n, n is an integer, and n multiplied by the weld layer thickness is equal to the height of the weld template STL model; tk is the weld layer thickness. Use the above-mentioned cutting plane to cut the template, traverse all the triangular facets of the weld template STL model, if the three points of the triangular facet are not all on the same side of the cutting plane, then find the intersection of the cutting plane and the triangular facet, the method is: take the three sides of the triangle as line segments, respectively construct the equations of the straight line passing through the three side segments of the triangle, for example: the equation of the straight line of one of the line segments is: x = x0 + a*t; y = y0 + b*t; z = z0 + c*t, where x0, y0, z0 are known points of the straight line, a, b, c are the direction vectors of the straight line; t is a parameter; the intersection of the straight line and the plane can be found by combining the equation of the straight line and the equation of the cutting plane z = i*tk; if the intersection is between the two endpoints of the side of the triangular facet, then the point is the intersection of the triangular facet and the cutting plane. The intersection of the other two sides of the triangular facet is obtained in the same way. According to the above method, the intersection points of all the triangular facets on the STL model and the cutting plane are obtained, and the contour line of the cutting plane can be obtained by connecting the above points in sequence.

[0075] 2. The STL model is sliced ​​using the method described in 1 to obtain the contour of each slice, which is the edge of the welding track. The contour area is filled according to a certain track to obtain the welding track of the contour.

[0076] 3. On the plane of the contour obtained by the method described in 2, create a series of parallel lines intersecting with the contour, each parallel line has two intersection points with the contour, take one of the intersection points as the welding starting point and the other intersection point as the welding end point, and construct a welding filling trajectory within the contour. The welding trajectory is sent to the six-degree-of-freedom robotic arm through a computer to control the robotic arm for welding.

[0077] Example 1

[0078] The automated welding method based on binocular vision includes:

[0079] Step 1: Use a line laser to scan the weld. At the same time, the binocular camera collects images of the laser line segment in the weld area from two different positions on the left and right. The image collected by the camera is binarized to obtain grayscale image data. Its format can be expressed as the pixel coordinates (i, j) and grayscale pixel values ​​(grayscale value is Pixel_Value) of each pixel point. Each grayscale pixel point of the grayscale image can be expressed as (i, j, Pixel_Value(i, j)).

[0080] Step 2: Traverse the grayscale images captured by the left and right cameras in the order of rows first and columns. For example, if the current row is i, analyze the pixel value of each pixel in the row. If the pixel value Pixel_Value(i, j) is greater than the set threshold threshold, which can be 0-255, then accumulate the pixel value and pixel column coordinate j of the pixel, that is, the total pixel value: Sum_Pixel_Value=Sum_Pixel_Value+Pixel_Value(i, j); total pixel columns: Sum_Col=Sum_Col+j. After counting the pixel coordinates of each row, use the center light column coordinates: Col_Laser=Sum_Pixel_Value / Sum_Col to obtain the center light column coordinates of the row. The coordinates of the center light can be expressed as: (i, Col_Laser, 1), and the pixel value of the row that is not equal to the Col_Laser column is 0. The images captured by the left and right cameras are processed according to the above analysis method to obtain the left and right views containing only laser light.

[0081] Step 3: Traverse each row of the left and right views in step 2, and calculate the pixel position difference distance (also called horizontal displacement) of the laser light of the left and right views in the row. For example, the column of the point with a pixel value of 1 in the left image is jL, and the column of the point with a pixel value of 1 in the right image is jR, and distance = jR-jL. Calculate the disparity value distance for each pixel of the left and right images to generate a disparity map.

[0082] The actual distance z of each pixel in the disparity map is calculated using the formula: Z = (fx × BASE_LINE) / distance; X = ((xc x )×Z) / fx;Y=((yc y )×Z) / fx, where fx is the focal length, BASE_LINE is the distance between the center lines of the two camera lenses, and cx and cy are camera constants. By converting each pixel point and 3D coordinates according to the above method, the spatial coordinates of all pixel points, i.e., 3D point cloud, can be obtained.

[0083] Step 4: Import the weld template file, create a spatial rectangular coordinate system xyz based on the weld template, cut the weld template with a plane parallel to the xy plane to obtain the cross-sectional profile of the model, cut the cross-sectional profile with a series of parallel lines intersecting the cross-sectional profile, extract the points where the parallel lines intersect the cross-sectional profile, and obtain the point cloud of the cross-sectional profile. Cut the model with multiple planes parallel to xy, obtain the point cloud of each cutting surface according to the above method, and summarize it to obtain the template point cloud.

[0084] Step 5: The method of cutting the weld template using a plane parallel to the xy plane is to import the weld template STL format file, use the plane parallel to the xy plane as the slicing plane, and the equation of the cutting plane is z equals i multiplied by the weld layer thickness, i is 1-n, n is an integer, and n multiplied by the weld layer thickness is equal to the height of the weld template STL model. Use the above-mentioned cutting plane to cut the template, traverse all the triangular facets of the weld template STL model, if the three points of the triangular facet are not all on the same side of the cutting plane, then find the intersection line of the cutting plane and the triangular facet, the method is: construct three straight line equations with the three sides of the triangle respectively, for example: the straight line equation of one side is in the form of x=x0+a*t; y=y0+b*t; z=z0+c*t, where x0, y0, z0 are known points of the straight line, a, b, c are the direction vectors of the straight line; t is a parameter; the straight line equation and the cutting plane equation are combined to find the intersection point of the straight line and the plane; if the intersection point is between the two endpoints of the line segment formed by the side of the triangular facet, then the point is the intersection point of the triangular facet and the cutting plane. The intersection point of the other two sides of the triangular facet is obtained in the same way. According to the above method, the intersection points of all triangular facets on the STL model and the cutting plane are obtained, and the above points are connected in sequence to obtain the contour line of the cutting surface.

[0085] Step 6: Match the above weld point cloud with the template point cloud to obtain the translation matrix and rotation matrix from the template point cloud to the weld point cloud. The method is as follows: find the centroid coordinates of the weld point cloud (W x , W y , W z ) and the centroid coordinates of the template point cloud (C x , C y , C z ), the center of gravity is the weighted average of all the point coordinates of the weld point cloud or template point cloud. For example, there are n points in the point cloud, and the coordinates of each point are (x i ,y i , z i ), where i ranges from 1 to n. The x coordinates of the n points in the point cloud are accumulated and divided by the total number of points in the point cloud n to obtain the x coordinate of the center of gravity C x , use the same method to calculate the y coordinate C of the center of gravity y , the z coordinate of the center of gravity Cz . The centroid coordinates (W x , W y , W z ) of the weld seam point cloud are obtained by the same method. Move each point of the template point cloud so that the centroid coordinates (C x , C y , C z ) of the template point cloud coincide with the centroid coordinates (W x , W y , W z ) of the weld seam point cloud. The method is as follows: First, calculate the centroid coordinate deviation of the two point clouds, Δx = W x - C x , Δy = W y - C y , Δz = W z - C z . Add the deviation of the centroid to the coordinates (x i , y i , z i ) of each point of the template point cloud. That is, the new coordinates of each point of the template point cloud are (x i + Δx, y i + Δy, z i + Δz). Traverse all points in the weld seam point cloud and find the point Pw that is farthest from the centroid W. Rotate the template point cloud around the x, y, and z axes respectively. The rotation angles around the x, y, and z axes are (θ, δ, γ), and the value ranges of θ, δ, and γ are (0 - 360°). Each time θ, δ, and γ are rotated by 1°. Set a variable d0 to infinity. Rotate the template point cloud around the x, y, and z axes by θ, δ, and γ degrees respectively to obtain the new template point cloud coordinates. Find the point Pc that is farthest from the centroid C in the new template point cloud. Calculate the distance d1 between Pw and Pc. If d1 < d0, record the new rotation matrix (θ, δ, γ), and assign d0 to d1. After n loops, the minimum rotation matrix (θ, δ, γ) will be obtained, where the value of n is 360 * 360 * 360. Calculate the translation matrix (W x - C x , W y - C y , W z - C z ) and the rotation matrix (θ, δ, γ).

[0086] Step 7: Slice the weld template to obtain a welding trajectory. The slicing method is: use a plane parallel to the Z axis to cut the weld template to obtain the contour of the model. Create a series of parallel lines that intersect with the contour, each parallel line has two intersections with the contour, take one of the intersections as the welding start point, and the other intersection as the welding end point, construct the welding filling trajectory within the contour, and use the rotation matrix (W obtained in claim 8) x -C x , W y -C y , W z -C z ) and the rotation matrix (θ, δ, γ) are used to transform the welding trajectory and obtain the actual welding trajectory.

[0087] Step 8: The actual welding trajectory is sent to the six-degree-of-freedom robotic arm, which controls the robotic arm to drive the welding gun to perform welding, thereby completing the multi-layer and multi-pass welding process of the weld.

[0088] Example 2

[0089] like Figure 1 As shown in the figure, the automated welding method based on binocular vision is composed of a line laser (1), a binocular camera (2), and a welding workpiece (3). After the steps of scanning ①, data processing (②→③), template generation (③→④), discretization (④→⑤), comparison (⑥→⑦), and slice trajectory generation (⑧→⑨), the welding process is made efficient, high-precision, and high-quality, realizing standardized, automated, and intelligent processing.

[0090] Adjust the position of the line laser (1) so that the angle between the line laser and the welding workpiece (3) is about 30° to 150°. Turn on the line laser (1) to present a bright and continuous laser light on the workpiece. Adjust the position of the welding gun so that the welding gun and the welding workpiece (3) are kept at an appropriate distance. The binocular camera (2) is fixed on the six-degree-of-freedom robot arm so that the two industrial cameras can obtain clear and complete images.

[0091] At the beginning of scanning①, the control line laser light scans the weld along the length direction of the weld to collect and obtain the data of the welded workpiece (3). Unprocessed weld point cloud data is generated in the industrial computer;

[0092] Cut the point cloud data on the industrial computer operation panel② and retain only the point cloud related to the weld, i.e., the weld point cloud③;

[0093] According to the three-dimensional spatial coordinate data of the weld point cloud, the length, width, height and other information of the weld are analyzed to create a three-dimensional model of the weld, namely the weld template④. The weld template is discretized to obtain the template point cloud. The algorithm process of template discretization is as follows: Place the weld template in a rectangular coordinate system, cut the template vertically with a plane parallel to the yz plane to obtain a section of the model, unfold the section in the yz plane, extract the edge information of the section, and obtain the outer contour of the section. Use straight lines with equal intervals and parallel to the z-axis to divide the section, extract the points where the straight lines intersect the section contour, and obtain the point cloud of one section of the weld model. Use multiple planes parallel to the yz plane to intercept the model, and perform the same operation to obtain the template point cloud⑤.

[0094] Calculate the centroid coordinates of the weld point cloud (W x , W y , W z ) and the centroid coordinates of the template point cloud (C x , C y , C z ), the center of gravity is the weighted average of all the point coordinates of the weld point cloud or template point cloud. For example, there are n points in the point cloud, and the coordinates of each point are (x i ,y i , z i ), where i ranges from 1 to n. The x coordinates of the n points in the point cloud are accumulated and divided by the total number of points in the point cloud n to obtain the x coordinate of the center of gravity C x , use the same method to calculate the y coordinate C of the center of gravity y , the z coordinate of the center of gravity C z The same method is used to obtain the centroid coordinates of the weld point cloud (W x , W y , W z ). Move each point of the template point cloud so that the centroid coordinates of the template point cloud (C x , C y , C z ) and the centroid coordinates of the weld point cloud (W x , W y , W z ) The position coincides, the method is: first calculate the centroid coordinate deviation of the two point clouds Δx = W x -C x , Δy=W y -C y , Δz=W z -C z , the coordinates of each point in the template point cloud (x i ,y i , z i ) are added with the deviation of the center of gravity, that is, the new coordinates of each point in the template point cloud are (x i +Δx,y i+Δy, z i +(Δz). After the centers of gravity are made to coincide, all the points in the weld point cloud are traversed to find the point Pw that is farthest from the center of gravity W. The template point cloud is rotated around the x, y, and z axes respectively, where the rotation angles around the x, y, and z axes are (θ, δ, γ), and the value ranges of θ, δ, and γ are (0 - 360°). Each time θ, δ, and γ are rotated, they are increased by 1° respectively. A variable d0 is set to infinity. All the angles of rotation around the x, y, and z axes are traversed to obtain the new coordinates of the template point cloud, and the point Pc that is farthest from the center of gravity C in the new template point cloud is found. And the distance d1 between Pw and Pc is calculated. If d1 < d0, the new rotation matrix (θ, δ, γ) is recorded, and d0 is assigned the value of d1. After n loops, the minimum rotation matrix (θ, δ, γ) will be obtained, where the value of n is 360 * 360 * 360.

[0095] Slice the weld template, set parameters such as the slice thickness and welding layer thickness, slice the template to obtain the welding trajectory, and transform the coordinates of the welding trajectory according to the minimum rotation matrix (θ, δ, γ) obtained above to get the welding trajectory on the workpiece; use an industrial computer to transmit the welding trajectory to a six-degree-of-freedom robotic arm through a control system, and control the robotic arm to drive the welding torch to weld according to the planned welding trajectory.

[0096] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and deformations can still be made, and these changes and deformations should also be regarded as the protection scope of the present invention.

[0097] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and deformations can still be made, and these changes and deformations should also be regarded as the protection scope of the present invention.

Claims

1. An automated welding method based on binocular vision, characterized in that: Including: S1: Data acquisition: Scanning the weld seam of the workpiece to be welded through a binocular camera and a line laser. S2: Transmitting the collected raw data to an industrial computer in real time for analysis and processing to obtain weld seam point cloud data. S3: Creating a target weld seam template according to the shape of the weld seam, and generating template point cloud data according to the weld seam template. S4: Calculating the weld seam point cloud data to obtain the centroids of the weld seam point cloud and the template point cloud. S5: Conducting template comparison to make the centroids of the template point cloud and the weld seam point cloud coincide, and continuously iterating to reduce the registration error between the point clouds to find the optimal transformation matrix. Finally, slicing the weld seam template to obtain the optimal welding trajectory. S6: Sending the welding trajectory to a six-degree-of-freedom robotic arm through a computer to control the robotic arm for welding.

2. The automatic welding method based on binocular vision according to claim 1 is characterized in that: The data acquisition described in S1 includes: Using a line laser to scan the weld seam. The laser beam emitted by the line laser projects a bright and continuous laser line segment on the surface of the weld seam. The laser line segment is clearly visible within the field of view of the cameras of the binocular camera. When the laser line segment scans a flat surface, the line segment captured by the camera is a straight line segment; while when the laser line segment scans an uneven weld seam, the line segment deforms and changes from a straight line segment to a broken line segment. The two industrial cameras of the binocular camera serve as image capture devices, synchronously capturing images of the laser line segment in the weld seam area from two different positions on the left and right respectively, and then transmitting the collected raw data to an industrial computer in real time for analysis and processing.

3. The automatic welding method based on binocular vision according to claim 1, characterized in that S2 Including: S21: Traversing each row in the left and right views scanned by the binocular camera, calculating the pixel position difference distance of the laser rays in the left and right views in this row, and generating a disparity map by calculating the disparity value distance for each pixel in the left and right images. S22: Calculating the depth value of the corresponding pixel points. Setting the distance between the center lines of the two cameras as BASE_LINE, the depth value of each pixel represents its actual distance Z = (fx × BASE_LINE) / distance from the midpoint of the two cameras, where fx is the focal length. S23: Analyze all pixel points corresponding to the left and right views to obtain an image containing the position (x, y) and depth value z of each pixel point; the pixel coordinates and the three-dimensional coordinates are calculated by the formula: X=((xc x )×Z) / fx;Y=((yc y )×Z) / fx, where c x and c y is the constant of the camera. Similarly, the transformation of each pixel point and three-dimensional coordinates is performed to obtain the spatial coordinates of each pixel point, that is, the three-dimensional point cloud. S24: Converting the two-dimensional pixel positions and depth values into actual coordinates in three-dimensional space to generate weld seam point cloud data; removing the redundant points and only retaining the point cloud data of the weld seam.

4. The automated welding method based on binocular vision according to claim 1, characterized in that: S3 includes: Importing the STL file of the weld seam template and discretizing it to obtain the template point cloud. The algorithm process of discretization is as follows: According to the three-dimensional model of the weld seam, creating a spatial rectangular coordinate system xyz, where the x, y, and z axes respectively represent the position information of the model in three different directions. Vertically cutting the template with a plane parallel to the yz plane to obtain a cross-section ABC of the model. Unfolding the cross-section in the yz plane and extracting the edge information of the cross-section to obtain the outer contour of the cross-section. Dividing the cross-section with equally spaced lines parallel to the z axis (0 < y < y', where y' is the maximum length of the cross-section), extracting the intersection points of the lines and the cross-section contour to obtain the point cloud data of the ABC cross-section, that is, the point cloud of one cross-section of the weld seam model. Intercepting the model with multiple planes parallel to the yz plane and performing the same operation to finally obtain the template point cloud.

5. The binocular vision-based automated welding method according to claim 1, characterized in that: The centroid coordinates of the weld point cloud in S4 (W x , W y , W z ) and the centroid coordinates of the template point cloud (C x , C y , C z ), where the centroid is the weighted average of all point coordinates of the weld point cloud or template point cloud.

6. The binocular vision-based automated welding method according to claim 1, characterized in that: In S5, each point of the template point cloud is moved so that the centroid coordinates (C x , C y , C z ) and the centroid coordinates of the weld point cloud (W x , W y , W z ) The position coincides, the method is: calculate the centroid coordinate deviation of the two point clouds Δx = W x -C x , Δy=W y -C y , Δz=W z -C z , the coordinates of each point in the template point cloud (x i ,y i , z i ) are added with the deviation of the center of gravity, that is, the new coordinates of each point in the template point cloud are (x i +Δx,y i +Δy,z i +Δz); Traverse all the points in the weld point cloud to obtain the point Pw that is farthest from the centroid W; rotate the template point cloud around the x, y, and z axes respectively, where the rotation angles around the x, y, and z axes are (θ, δ, γ), and the value ranges of θ, δ, and γ are (0 - 360°). Each time θ, δ, and γ are rotated, they increase by 1° respectively. Set a variable d0 to infinity. Traverse all the angles of rotation around the x, y, and z axes to obtain the new template point cloud coordinates, and find the point Pc in the new template point cloud that is farthest from the centroid C. Calculate the distance d1 between Pw and Pc. If d1 < d0, record the new rotation matrix (θ, δ, γ), and assign d0 the value of d1. After n loops, the minimum rotation matrix (θ, δ, γ) will be obtained, where the value of n is 360 * 360 * 360. The translation matrix (W x -C x , W y -C y , W z -C z ) and the rotation matrix (θ, δ, γ) are applied to the template point cloud to obtain the new position of the template point cloud so that the template point cloud is aligned with the weld point cloud.

7. The binocular vision-based automated welding method according to claim 1, characterized in that: In S5, perform slicing operations on the weld template to obtain the optimal welding trajectory, including: Import the STL format file of the weld template. Use a plane parallel to the x - y plane as the slicing plane. The equation of the slicing plane is z = i * tk, where the value of i ranges from 1 to n, n is an integer, and n multiplied by the welding layer thickness is equal to the height of the weld template STL model; tk is the welding layer thickness. Use the slicing plane to cut the template. Traverse all the triangular facets of the weld template STL model. If the three points of the triangular facet are not all on the same side of the slicing plane, find the intersection line of the slicing plane and the triangular facet. The method is as follows: Take the three sides of the triangle as line segments and construct the straight - line equations passing through the line segments of the three sides of the triangle respectively. Simultaneously solve the straight - line equation and the slicing - plane equation z = i * tk to find the intersection point of the straight line and the plane. If the intersection point is between the two endpoints of the side of the triangular facet, then this point is the intersection point of the triangular facet and the slicing plane. The method for finding the intersection points of the other two sides of the triangular facet is the same. Find the intersection points of all the triangular facets on the STL model and the slicing plane, and connect these points in sequence to obtain the contour line of this section. Slice the STL model to obtain the contours of each slice. This contour is the edge of the welding trajectory. Fill the contour area according to the trajectory to obtain the welding trajectory of this contour. On the plane of the obtained contour, create a series of parallel lines that intersect the contour. Each parallel line has 2 intersection points with the contour. Take one intersection point as the welding start point and the other intersection point as the welding end point to construct the welding filling trajectory within the contour.

8. The control system of the method according to claims 1-7, characterized in that: It includes a line laser, a binocular camera (2), a welding workpiece (3), a six - degree - of - freedom robotic arm, and an industrial computer. The binocular camera (2) consists of two industrial cameras, and the binocular camera (2) and the line laser (1) are installed on the six - degree - of - freedom robotic arm.

Images