A method and device for generating a grinding trajectory of a straight weld seam with self-adaptive welding deformation amount

Abstract

The present invention relates to a method for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount, including the following steps: obtaining multiple frames of two-dimensional images of a target welding plate through a color RGB camera in a Kinect V2 camera, and obtaining multiple frames of three-dimensional point cloud images of the target welding plate through an infrared IR camera; performing rough extraction of a three-dimensional straight weld according to the obtained multiple frames of the two-dimensional images and the three-dimensional point cloud images to obtain three-dimensional weld information; performing noise reduction processing on the three-dimensional weld information, and removing outliers and non-real weld trajectory points to obtain a three-dimensional weld point cloud pi; generating path points that can be actually executed by a robot according to the three-dimensional weld point cloud pi; controlling the robot to perform weld grinding according to the path points. The whole process of the present invention is completed fully automatically, which can avoid a series of problems caused by manual processing of welds. Moreover, this method has high accuracy in weld recognition and grinding path planning, and can complete intelligent grinding of welds efficiently and stably.

Description

Technical Field

[0001] The present invention relates to the technical field of robot vision positioning, and particularly relates to a method and device for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount. Background Art

[0002] Welding technology is widely used in enterprise production. After welding, the weld seam sometimes needs to be polished to meet the requirements of the plate in terms of size, flatness and aesthetics. Since there are many control factors required in the process of weld seam grinding and the scene is complex and changeable, it is very difficult to achieve the accuracy requirements by using a robot for grinding. Therefore, at present, the process of weld seam grinding is basically solved manually.

[0003] If it is solved manually, on the one hand, the noise, dust, harmful gases, sparks, etc. generated during the grinding process will affect the physical and mental health of the operators. The grinding process usually consumes a lot of physical strength, and the tools used generally rotate at high speed. If not careful during use, it will cause a certain degree of physical harm to the body; on the other hand, the cost is relatively high, and it is difficult to unify the standardization degree. Summary of the Invention

[0004] The purpose of the present invention is to at least solve one of the deficiencies of the prior art, and provide a method and device for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount.

[0005] In order to achieve the above purpose, the present invention adopts the following technical solutions.

[0006] Specifically, a method for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount is proposed, including the following:

[0007] Obtain multiple frames of two-dimensional images of the target welding plate through the color RGB camera in the kinect V2 camera, and obtain multiple frames of three-dimensional point cloud images of the target welding plate through the infrared ir camera;

[0008] According to the obtained multiple frames of the two-dimensional images and three-dimensional point cloud images, perform rough extraction of the three-dimensional straight weld seam to obtain three-dimensional weld seam information;

[0009] After performing noise reduction processing on the three-dimensional weld seam information, removing outliers and non-real weld seam trajectory points, obtain three-dimensional weld seam point cloud pi;

[0010] According to the three-dimensional weld seam point cloud pi, generate path points that can be actually executed by the robot for production;

[0011] Control the robot to perform weld seam grinding according to the path points.

[0012] Further, specifically, the three-dimensional straight weld seam information is roughly extracted by the following method.

[0013] After grayscale processing the acquired two-dimensional image, perform Gaussian filtering, then extract the texture information therein through Canny edge detection. After that, use the Hough line transform to extract the straight lines in the texture information to obtain multiple straight lines, and extract the target straight line, which is the weld seam, based on the BDSCAN idea. At this time, the position of the weld seam in the two-dimensional image is obtained;

[0014] Calibrate the internal and external parameters of the color RGB camera and the infrared ir camera. Specifically,

[0015] Let be the spatial inhomogeneous coordinates of a point in the ir camera coordinate system, be the projection point of on the image plane of the ir camera, be the pre-calibrated internal parameter matrix of the ir depth camera, then there is the following relationship:

[0016] (1);

[0017] Similarly, the corresponding relationship of the RGB camera can be obtained:

[0018] (2);

[0019] Let , be the external parameters of the infrared camera, , be the external parameters of the color camera, , be the spatial pose transformation of the color camera coordinate system relative to the infrared camera coordinate system, then there is:

[0020] (3);

[0021] (4);

[0022] (5);

[0023] Substitute (1) and (2) into (5), and and can be eliminated:

[0024] (6);

[0025] According to the established corresponding relationship of pixel points between the color RGB camera and the infrared ir camera above, project the position of the weld seam in the two-dimensional image into the three-dimensional image.

[0026] Furthermore, specifically, the process of obtaining the three-dimensional weld seam point cloud pi includes the following,

[0027] The three-dimensional weld seam information is refined by the DBSCAN algorithm to obtain the three-dimensional weld seam point cloud pi, denoted as the weld seam point cluster line_2.

[0028] Furthermore, specifically, according to the three-dimensional weld seam point cloud pi, path points that can be actually executed by the robot are generated, including the following:

[0029] Step 510: Calculate the average unit point normal vector of the trajectory points in line_2 , and based on the unit direction vector of the straight line fitted by line_2 Cross multiply Calculate the three-dimensional unit vector m: m = × ;

[0030] Step 520: Based on the unit vector m, offset each point in the point cloud cluster line_2 by 1.5 times in the direction of the m vector, and calculate the corresponding neighborhood points = + 1.5 * m, = - 1.5 * m, to obtain the neighborhood point clusters line_2+ and line_2- of line_2;

[0031] Step 530: Calculate and respectively, and the nearest neighbor points and located on the target welding plate point cloud, After that, calculate and point normal vectors , then the grinding position and posture of the real grinding trajectory points are respectively: = ([[]]END]] + ) / 2; = ([[]]END]] + ) / 2, at this time, all points in Pi are recorded as the point cloud cluster line_3;

[0032] Step 540: Sort the grinding positions and postures of the grinding trajectory points based on the point cloud cluster line_3 to obtain the sorted point cloud cluster line_4;

[0033] Step 550: Use the B-spline curve to smooth the position and posture information in the point cloud cluster line_4 to obtain the point cloud cluster line_5.

[0034] Step 560: Use the B-spline curve to process the point cloud cluster line_5 again to obtain a point cloud cluster line_6 with equal point intervals, and the normal vectors are also fitted synchronously.

[0035] Step 570: Calculate the kinematic inverse solution based on the point cloud cluster line_6, and set the sixth axis to a fixed value in the inverse solution result to obtain the path points that can be actually executed by the robot for production.

[0036] The present invention also proposes a device for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount, including:

[0037] An image acquisition module, configured to acquire multiple frames of two-dimensional images of the target welding plate through the color RGB camera in the kinect V2 camera, and acquire multiple frames of three-dimensional point cloud images of the target welding plate through the infrared ir camera;

[0038] A three-dimensional weld information acquisition module, configured to perform rough extraction of the three-dimensional straight weld based on the acquired multiple frames of the two-dimensional images and three-dimensional point cloud images to obtain three-dimensional weld information;

[0039] A refinement module, configured to perform noise reduction processing on the three-dimensional weld information, and remove outliers and non-genuine weld trajectory points to obtain the three-dimensional weld point cloud pi;

[0040] A path point generation module, configured to generate path points that can be actually executed by the robot for production based on the three-dimensional weld point cloud pi;

[0041] An execution module, configured to control the robot to perform weld grinding according to the path points.

[0042] The present invention also proposes a weld grinding platform, which applies a method for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount described in any one of the above, including,

[0043] A robot;

[0044] A kinect V2 camera, configured to acquire multiple frames of two-dimensional images of the target welding plate through the color RGB camera therein, and acquire multiple frames of three-dimensional point cloud images of the target welding plate through the infrared ir camera;

[0045] A platform, configured to fix the target welding plate;

[0046] An industrial control computer based on the ROS robot control system, configured to receive multiple frames of two-dimensional images and three-dimensional point cloud images transmitted by the kinect V2 camera, and perform grinding trajectory planning and execution based on this.

[0047] The present invention also provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the steps of the method for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount described in any one of the above are implemented.

[0048] The beneficial effects of the present invention are as follows:

[0049] By providing a method for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount, the present invention first identifies the weld in the two-dimensional space, then projects the two-dimensional weld into three dimensions to obtain the weld in the three-dimensional space, and finally processes the weld in the three-dimensional space to generate path points that can be actually executed by the robot in production. The whole process is fully automated, which can avoid a series of problems caused by manual processing of the weld. Moreover, this method has high accuracy in weld identification and grinding path planning, and can complete intelligent grinding of the weld efficiently and stably. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] By describing the embodiments shown in the accompanying drawings in detail, the above and other features of the present disclosure will become more obvious. The same reference numerals in the drawings of the present disclosure denote the same or similar elements. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. In the drawings:

[0051] Figure 1 It shows a flowchart of the method for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount according to the present invention;

[0052] Figure 2 It shows an implementation flowchart of the method for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount according to the present invention;

[0053] Figure 3 It shows an example diagram of a target welding plate in an embodiment of the method for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount according to the present invention;

[0054] Figure 4 It shows a schematic diagram of projecting a two-dimensional weld into a three-dimensional weld in the method for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount according to the present invention;

[0055] Figure 5 It shows a schematic diagram of the Hough line transformation in the method for generating a grinding trajectory of a straight weld with self-adaptive welding deformation amount according to the present invention;

[0056] Figure 6The figure shows a schematic diagram of the trajectory calculated based on line_2 in an embodiment of a method for generating a grinding trajectory of a straight weld with self - adaptation to welding deformation amount according to the present invention;

[0057] Figure 7 The figure shows the rendering effect of rough processing of a three - dimensional weld grinding trajectory in an embodiment of a method for generating a grinding trajectory of a straight weld with self - adaptation to welding deformation amount according to the present invention;

[0058] Figure 8 The figure shows a schematic diagram of drawing grinding trajectory points and corresponding normal vectors based on line_5 in an embodiment of a method for generating a grinding trajectory of a straight weld with self - adaptation to welding deformation amount according to the present invention;

[0059] Figure 9 The figure shows the rendering effect of the improved three - dimensional weld grinding trajectory in an embodiment of a method for generating a grinding trajectory of a straight weld with self - adaptation to welding deformation amount according to the present invention. Specific Embodiment

[0060] The following will clearly and completely describe the concept, specific structure, and technical effects generated by the present invention in combination with embodiments and drawings, so as to fully understand the purpose, solution, and effects of the present invention. It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0061] Refer to Figure 1 、 Figure 2 and Figure 3 , Example 1, the present invention proposes a method for generating a grinding trajectory of a straight weld with self - adaptation to welding deformation amount, including the following:

[0062] Step 110: Obtain multiple frames of two - dimensional images of the target welding plate through the color RGB camera in the kinect V2 camera, and obtain multiple frames of three - dimensional point cloud images of the target welding plate through the infrared ir camera;

[0063] Step 120: According to the obtained multiple frames of the two - dimensional images and three - dimensional point cloud images, perform rough extraction of the three - dimensional straight weld to obtain three - dimensional weld information;

[0064] Step 130: Perform noise reduction processing on the three - dimensional weld information, and remove outliers and non - real weld trajectory points to obtain the three - dimensional weld point cloud pi;

[0065] Step 140: Generate path points that can be actually executed by the robot according to the three - dimensional weld point cloud pi;

[0066] Step 150: Control the robot to perform weld grinding according to the path points.

[0067] Reference Figure 4 and Figure 5 As a preferred embodiment of the present invention, specifically, the three-dimensional straight weld seam information is roughly extracted by the following method:

[0068] The obtained two-dimensional image is grayscale processed and then Gaussian filtered, and then the texture information therein is extracted by canny edge detection. After that, the Hough line transform is used to extract the straight lines in the texture information to obtain multiple straight lines. Based on the BDSCAN idea, the target straight line is extracted as the weld seam, and at this time, the position of the weld seam in the two-dimensional image is obtained;

[0069] As can be seen from the figure, multiple straight lines will be obtained after the Hough transform. We represent them with the straight line cluster L1, and the number is represented by size1. And what we need is the unique weld seam information. The following is the principle of extracting the straight line corresponding to the weld seam:

[0070] In the Hough line transform, a straight line is described by two parameters (such as Figure 5 ): θ, r. r is the distance from the origin to the line segment of the straight line, and θ is the angle between the line segment and the X-axis of the coordinate system. At this time, we can sort the straight line cluster L1 based on r, set a coefficient λ1 (0 < λ1 < 1), discard the first (size1 × λ1 / 2) straight lines and the last (size1 × λ1 / 2) straight lines after sorting based on r to obtain the straight line cluster L2; after the first refinement, a second refinement is still needed. Using the same principle, the straight line cluster L2 is sorted and refined based on θ to obtain the straight line cluster L3; finally, the average value of the parameters θ and r in the straight line cluster L3 is calculated, and the straight line represented by the average value corresponds to the weld seam.

[0071] If we want the robot to polish the weld seam in the real space, we must obtain the weld seam point position information in the three-dimensional space. Since the two-dimensional space lacks one-dimensional information compared to the three-dimensional space, we need to find a corresponding relationship between each pixel of the RGB image and the point cloud. The following is the implementation principle:

[0072] The kinect V2 camera consists of two cameras. One is a color RBG camera, and the other is an infrared ir camera. The ir camera can obtain a three-dimensional point cloud and an infrared grayscale image. Each point in the three-dimensional point cloud corresponds to each pixel point in the infrared grayscale image. The infrared camera is mainly used for point cloud imaging, and the obtained ir grayscale image has very poor imaging quality. Therefore, after two-dimensional recognition with the RGB camera, the corresponding pixels are projected onto the ir image, and the three-dimensional space point center of the corresponding pixels can be obtained. Before projection, the internal parameters and external parameters of the two cameras need to be calibrated.

[0073] Based on this, calibrate the internal parameters and external parameters of the color RGB camera and the infrared ir camera, specifically including,

[0074] Let is the spatial inhomogeneous coordinate of a point in the IR camera coordinate system, is the projection point on the IR camera image plane , is the pre-calibrated internal parameter matrix of the IR depth camera, then the following relationship holds:

[0075] (1);

[0076] Similarly, the corresponding relationship of the RGB camera can be obtained:

[0077] (2);

[0078] Let , be the external parameters of the IR camera, , be the external parameters of the color camera, , be the spatial pose transformation of the color camera coordinate system relative to the IR camera coordinate system, then there is:

[0079] (3);

[0080] (4);

[0081] (5);

[0082] Substitute (1) and (2) into (5) to eliminate and :

[0083] (6);

[0084] According to the established corresponding relationship between the pixels of the color RGB camera and the IR camera above, project the position of the weld in the two-dimensional image into the three-dimensional image. At this time, we have found the corresponding relationship between the pixels of the color camera and the IR camera. Input a certain pixel value to obtain the corresponding , and at this time, the point position information in the three-dimensional space can be obtained.

[0085] Referring to Figure 6 , Figure 7 , Figure 8 and Figure 9 , as the preferred implementation manner of the present invention. Specifically, after obtaining the corresponding three-dimensional space weld information (abbreviated as weld point cluster line_1 here), the robot still cannot perform grinding based on these points, and the point cluster needs to be further processed, and the point normal information suitable for grinding is calculated. The process of obtaining the three-dimensional weld point cloud pi includes the following,

[0086] The three-dimensional weld information is refined by the DBSCAN algorithm to obtain the three-dimensional weld point cloud pi, denoted as the weld point cluster line_2.

[0087] As a preferred embodiment of the present invention, specifically, according to the three-dimensional weld point cloud pi, path points that can be actually executed by the robot are generated. If the line_2 is directly used to calculate the normal vector of the trajectory point, the result is as Figure 6 , and the pose of the actual grinding path of the robot planned is as Figure 7 In the simulation effect, it can be seen that the running speed of the robot is not uniform and the flutter is serious. It is very easy to have safety accidents during the actual grinding process and the grinding effect cannot be satisfied. The solutions include the following:

[0088] Step 510: Calculate the average value of the unit point normal vectors of the trajectory points in line_2 , and based on the unit direction vector of the straight line fitted by line_2 Cross multiply Calculate the three-dimensional unit vector m: m = × ;

[0089] Step 520: Based on the unit vector m, offset each point in the point cloud cluster line_2 by 1.5 times in the direction of the m vector, and calculate the corresponding neighborhood points = + 1.5*m, = - 1.5*m, to obtain the neighborhood point clusters line_2+ and line_2- of line_2;

[0090] Step 530: Calculate and respectively, and the nearest neighbor points , on the target welding plate point cloud. After that, calculate , of the point normal vector , then the grinding position and pose of the real grinding trajectory point are respectively: = ( + ) / 2; = ( + ) / 2. At this time, all points in Pi are denoted as the point cloud cluster line_3;

[0091] Step 540: Sort the grinding positions and poses of the grinding trajectory points based on the point cloud cluster line_3 to obtain the sorted point cloud cluster line_4;

[0092] Step 550: Use the B-spline curve to smooth the position and attitude information in the point cloud cluster line_4 to obtain the point cloud cluster line_5. This is to make the speed and acceleration of the robot continuous during the grinding process and obtain better processing effects. At this time, the point and normal vector information in line_5 is as Figure 8 , compared with Figure 6 , there are many improvements.

[0093] Step 560: Use the B-spline curve to process the point cloud cluster line_5 again to obtain the point cloud cluster line_6 with equal point intervals, and the normal vector is also fitted synchronously; Figure 8 In the straight line point cloud in Figure 9 , the density of the point cloud is different. When the ROS-I robot control system plans the motion, the running time and distance of each segment are not proportional, resulting in an inconsistent speed during the planning process, sometimes fast and sometimes slow. At this time, it is necessary to use the B-spline again to obtain the point cloud cluster line_6 with equal point intervals, and the normal vector is also fitted synchronously. At this time, as shown in

[0094] Step 570: Calculate the kinematic inverse solution according to the point cloud cluster line_6, and set the sixth axis to a fixed value in the inverse solution result to obtain the path points that can be actually executed by the robot in production.

[0095] There are multiple pairs of point and normal vector information in the initial grinding trajectory. Each point corresponds to a unique position, while the normal vector corresponds to an infinite number of end-effector postures of the robot. Therefore, each pair of point normal vectors corresponds to an infinite number of inverse solution states of the robot. Here, an optimal inverse solution needs to be uniquely determined:

[0096] The end-effector posture information of the robot, that is, the spatial rotation transformation relationship between the tool coordinate system and the polar coordinate system. For the convenience of solving, it is represented in the form of axis-angle angel-aix here. The cross product of the unit vector (0, 0, 1) of the Z axis in the base coordinate system and the normal vector in the trajectory is used to obtain the corresponding rotation axis aix. The unit vector of the Z axis and each normal vector correspond to an included angle, which is the rotation angle angel. The axis-angle can uniquely determine the spatial rotation transformation relationship, but we require that the sixth axis of the robot be fixed at zero during the grinding process. At this time, the grinding trajectory posture information does not yet meet the application conditions.

[0097] This grinding system is established based on ROS-I. The robot kinematics solution library trakIK can use the DH parameter information of the robot in ROS to solve the inverse solution of the robot, and can set a group of current joint space postures of the robot. The target posture in the optimal joint space is solved based on the current joint space posture. The single solution speed is within 0.5 ms, which fully meets the usage conditions.

[0098] Based on the inverse solution of the initial grinding trajectory, the trajectory information in the corresponding joint space is obtained. After setting the sixth joint value to 0, the KDL forward kinematics solution library is used to solve the corresponding final grinding trajectory points in the Cartesian space. Finally, based on the final trajectory points, the Moveit in ROS is used for Cartesian space path planning to execute the grinding task.

[0099] One implementation of step 540 is as follows:

[0100] At this time, the obtained Pi and its corresponding Ni are still a group of disordered points, and such point clusters cannot be used for trajectory planning. Here, a fast sorting method for line point clouds is proposed: Based on the point cloud cluster line_3, a straight line Line is fitted. A point Line_p on the line and the normal vector Line_n = (Line_nx, Line_ny, Line_nz) are used to represent it. The following is the pseudo-code for point cloud sorting (theta_x and theta_y represent the angles between Line_n and the X-axis and Y-axis respectively):

[0101] if (0° <= theta_x <= 60° || 120° <= theta_x <= 180°)

[0102] { Obtain the point Line_p2 on the line Line when x = 100, calculate the distances between all points Pi of the point cloud cluster line_3 and the point Line_p2, and sort the point cloud cluster line_3 according to the distance. The corresponding normal vector Ni is sorted synchronously. The sorted point cloud cluster is denoted as Line_4;

[0103] } else if (0° <= theta_y <= 60° || 120° <= theta_y <= 180°)

[0104] { Obtain the point Line_p3 on the line Line when y = 100, calculate the distances between all points Pi of the point cloud cluster line_3 and the point Line_p3, and sort the point cloud cluster line_3 according to the distance. The corresponding normal vector Ni is sorted synchronously. The sorted point cloud cluster is denoted as Line_4;

[0105] } else

[0106] { Obtain the point Line_p4 on the line Line when z = 100, calculate the distances between all points Pi of the point cloud cluster line_3 and the point Line_p4, and sort the point cloud cluster line_3 according to the distance. The corresponding normal vector Ni is sorted synchronously. The sorted point cloud cluster is denoted as Line_4;}

[0107] Note: The unit of the points in the point cloud is meters. When x, y, or z takes 100, the points on the obtained straight line will be somewhere far from the point cloud cluster line_3. Therefore, this method of sorting the point cloud has very high robustness.

[0108] The present invention also proposes a device for generating a grinding trajectory of a straight weld seam with self-adaptive welding deformation amount, including:

[0109] An image acquisition module, configured to acquire multiple frames of two-dimensional images of the target welding plate through the color RGB camera in the kinect V2 camera, and acquire multiple frames of three-dimensional point cloud images of the target welding plate through the infrared ir camera;

[0110] A three-dimensional weld seam information acquisition module, configured to perform rough extraction of the three-dimensional straight weld seam based on the acquired multiple frames of the two-dimensional images and three-dimensional point cloud images to obtain three-dimensional weld seam information;

[0111] A refinement module, configured to perform noise reduction processing on the three-dimensional weld seam information, and remove outlier points and non-real weld seam trajectory points to obtain a three-dimensional weld seam point cloud pi;

[0112] A path point generation module, configured to generate path points that can be actually executed by the robot according to the three-dimensional weld seam point cloud pi;

[0113] An execution module, configured to control the robot to perform weld seam grinding according to the path points.

[0114] The present invention also proposes a weld seam grinding platform, which applies a method for generating a grinding trajectory of a straight weld seam with self-adaptive welding deformation amount described in any one of the above, including,

[0115] A robot;

[0116] A kinect V2 camera, configured to acquire multiple frames of two-dimensional images of the target welding plate through the color RGB camera therein, and acquire multiple frames of three-dimensional point cloud images of the target welding plate through the infrared ir camera;

[0117] A platform, configured to fix the target welding plate;

[0118] An industrial control computer based on a ROS robot control system, configured to receive multiple frames of two-dimensional images and three-dimensional point cloud images transmitted by the kinect V2 camera, and perform grinding trajectory planning and execution based on this.

[0119] In this embodiment, by first identifying the weld seam in the two-dimensional space, then projecting the two-dimensional space weld seam into three dimensions to obtain the three-dimensional space weld seam, and finally processing the three-dimensional space weld seam to generate path points that can be actually executed by the robot, the whole process is completed fully automatically, which can avoid a series of problems caused by manual processing of the weld seam. Moreover, this method has high accuracy in weld seam recognition and grinding path planning, and can efficiently and stably complete intelligent grinding of the weld seam.

[0120] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method for generating a grinding path for a linear weld seam with self-adaptive welding deformation amount as described in any one of the above.

[0121] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this embodiment.

[0122] In addition, in each embodiment of the present invention, the functional modules may be integrated into one processing module, or each module may exist physically alone, or two or more modules may be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0123] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or system, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0124] Although the description of the present invention has been quite detailed and several of the described embodiments have been specifically described, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather should be regarded as providing a broad interpretation of these claims in view of the prior art by reference to the appended claims, so as to effectively cover the intended scope of the present invention. In addition, the present invention has been described above in terms of embodiments foreseeable by the inventors for the purpose of providing a useful description, and non-substantive modifications to the present invention that are not currently foreseeable may still represent equivalent modifications of the present invention.

[0125] As described above, these are only the preferred embodiments of the present invention. The present invention is not limited to the above-described embodiments. As long as the same means are used to achieve the technical effects of the present invention, they should fall within the protection scope of the present invention. Within the protection scope of the present invention, various different modifications and changes can be made to its technical solutions and / or implementation manners.

Claims

1. A method for generating a grinding trajectory of a straight weld with self - adapting welding deformation amount, characterized in that, Including the following: Obtaining multiple frames of two-dimensional images of the target welding plate through the color RGB camera in the Kinect V2 camera, and obtaining multiple frames of three-dimensional point cloud images of the target welding plate through the infrared IR camera; Performing rough extraction of three-dimensional straight weld seams based on the obtained multiple frames of the two-dimensional images and three-dimensional point cloud images to obtain three-dimensional weld seam information; Performing noise reduction processing on the three-dimensional weld seam information, and removing outliers and non-genuine weld seam trajectory points to obtain three-dimensional weld seam point cloud pi; Generating path points that can be actually executed by the robot according to the three-dimensional weld seam point cloud pi; Controlling the robot to perform weld grinding according to the path points; Specifically, the process of obtaining the three-dimensional weld seam point cloud pi includes the following: Performing refinement processing on the three-dimensional weld seam information through the DBSCAN algorithm to obtain the three-dimensional weld seam point cloud pi, denoted as the weld point cluster line_2; Specifically, generating path points that can be actually executed by the robot according to the three-dimensional weld seam point cloud pi includes the following: Step 510: Calculate the average unit normal vector of the trajectory points in line_2 , and cross multiply with the unit direction vector of the line fitted based on line_2 Cross multiply to calculate the three-dimensional unit vector m: m = × ; Step 520: Based on the unit vector m, shift each point in the point cloud cluster line_2 by 1.5 times in the direction of the m vector and calculate the corresponding neighborhood points = + 1.5 * m, = - 1.5 * m to obtain the neighborhood point clusters line_2+ and line_2- of line_2; Step 530, calculate respectively and the nearest neighbor points on the target welding plate point cloud , , then calculate , the point normal vectors of , then the grinding position and posture of the real grinding trajectory points are respectively: = ([[]] + ) / 2; = ([[]] + ) / 2, at this time, record all points in as the point cloud cluster line_3; Step 540: Sorting the grinding positions and postures of the grinding trajectory points based on the point cloud cluster line_3 to obtain the sorted point cloud cluster line_4; Step 550: Use a B-spline curve to smooth the position and attitude information in the point cloud cluster line_4 to obtain the point cloud cluster line_5; Step 560: Using the B-spline curve to process the point cloud cluster line_5 again to obtain a point cloud cluster line_6 with equal point intervals, and synchronously fitting the normal vectors; Step 570: Calculating the kinematic inverse solution according to the point cloud cluster line_6, and setting the sixth axis to a fixed value in the inverse solution result to obtain path points that can be actually executed by the robot.

2. A method for generating a grinding trajectory of a straight weld with self - adapting welding deformation amount according to claim 1, characterized in that Specifically, the three-dimensional straight weld seam information is roughly extracted in the following manner: Performing grayscale processing on the obtained two-dimensional images followed by Gaussian filtering, then extracting the texture information through canny edge detection, and then using the Hough line transform to extract the straight lines in the texture information to obtain multiple straight lines. Based on the BDSCAN idea, the target straight line is extracted as the weld seam, and at this time, the position of the weld seam in the two-dimensional image is obtained; Calibrating the internal and external parameters of the color RGB camera and the infrared IR camera, specifically including: Let be the spatial inhomogeneous coordinates of a point in the ir camera coordinate system, be the projection point on the ir camera image plane , be the pre-calibrated internal parameter matrix of the ir depth camera, then the following relationship holds: （1）； Similarly, the corresponding relationship of the RGB camera can be obtained: （2）； Let and be the external parameters of the infrared camera, and be the external parameters of the color camera, and be the spatial pose transformation of the color camera coordinate system relative to the infrared camera coordinate system, then we have: （3）； （4）； （5）； Substituting (1) and (2) into (5) can eliminate and : （6）； According to the established corresponding relationship between the pixel points of the color RGB camera and the infrared IR camera, projecting the position of the weld seam in the two-dimensional image into the three-dimensional image.

3. A straight weld grinding trajectory generation device with self-adaptive welding deformation amount, characterized in that, Including: An image acquisition module for obtaining multiple frames of two-dimensional images of the target welding plate through the color RGB camera in the Kinect V2 camera, and obtaining multiple frames of three-dimensional point cloud images of the target welding plate through the infrared IR camera; A three-dimensional weld seam information acquisition module for performing rough extraction of three-dimensional straight weld seams based on the obtained multiple frames of the two-dimensional images and three-dimensional point cloud images to obtain three-dimensional weld seam information; A refinement module for performing noise reduction processing on the three-dimensional weld seam information, and removing outliers and non-genuine weld seam trajectory points to obtain three-dimensional weld seam point cloud pi; A path point generation module for generating path points that can be actually executed by the robot according to the three-dimensional weld seam point cloud pi; An execution module for controlling the robot to perform weld grinding according to the path points; Specifically, the process of obtaining the three-dimensional weld seam point cloud pi includes the following: The three-dimensional weld information is refined by the DBSCAN algorithm to obtain the three-dimensional weld point cloud pi, denoted as the weld point cluster line_2; Specifically, according to the three-dimensional weld point cloud pi, path points that can be actually executed by the robot in production are generated, including the following: Step 510: Calculate the average unit normal vector of the trajectory points in line_2 , and cross multiply with the unit direction vector of the straight line fitted based on line_2 Cross multiply to calculate the three-dimensional unit vector m: m = × ; Step 520: Based on the unit vector m, shift each point in the point cloud cluster line_2 in the direction of the m vector by 1.5 times, and calculate the corresponding neighborhood points = + 1.5*m, = - 1.5*m, to obtain the neighborhood point clusters line_2+ and line_2- of line_2; Step 530, calculate respectively and the nearest neighbor points located on the target welding plate point cloud , . After that, calculate , the point normal vectors of . Then the grinding position and posture of the real grinding trajectory points are respectively: = ([[]]END]] + ) / 2; = ([[]]END]] + ) / 2. At this time, record all points in as the point cloud cluster line_3; Step 540: Sort the grinding positions and postures of the grinding trajectory points based on the point cloud cluster line_3 to obtain the sorted point cloud cluster line_4; Step 550. Use the B-spline curve to smooth the position and attitude information in the point cloud cluster line_4 to obtain the point cloud cluster line_5; Step 560: Use the B-spline curve to process the point cloud cluster line_5 again to obtain a point cloud cluster line_6 with equal point intervals, and the normal vectors are also fitted synchronously; Step 570: Calculate the kinematic inverse solution according to the point cloud cluster line_6, and set the sixth axis to a fixed value in the inverse solution result to obtain path points that can be actually executed by the robot in production.

4. A weld grinding platform, characterized in that, A method for generating a straight weld grinding trajectory with adaptive welding deformation amount according to any one of claims 1-2 above is applied, including: A robot; A kinect V2 camera, which is used to obtain multiple frames of two-dimensional images of the target welding plate through the color RGB camera therein, and obtain multiple frames of three-dimensional point cloud images of the target welding plate through the infrared ir camera; A platform for fixing the target welding plate; An industrial computer based on the ROS robot control system, which is used to receive multiple frames of two-dimensional images and three-dimensional point cloud images transmitted by the kinect V2 camera, and perform grinding trajectory planning and execution based on this.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1-2 are implemented.

Images