Welding robot control method, device, welding robot and readable medium

Abstract

The present application provides a welding robot control method, device, welding robot and readable medium. The method includes: using a three-dimensional camera at the end of the welding robot to collect point cloud data of a three-dimensional ball on a target workpiece at multiple designated collection positions, obtaining first point cloud data and a posture matrix corresponding to each collection position; performing data conversion on the first point cloud data according to the hand-eye transformation matrix and the posture matrix to obtain second point cloud data; determining the first sphere center and the second sphere center corresponding to the three-dimensional ball at each collection position respectively according to the first point cloud data and the second point cloud data, obtaining a first sphere center set and a second sphere center set; generating a compensation matrix of the hand-eye transformation matrix according to the first sphere center set and the second sphere center set; and driving the welding robot to move the end of the welding gun of the welding robot to the designated position of the target workpiece according to the compensation matrix and the hand-eye transformation matrix. The method accurately positions the workpiece, which is conducive to improving welding efficiency.

Description

Technical Field

[0001] The present application relates to the field of robots, and in particular to a welding robot control method, device, welding robot and readable medium. Background Art

[0002] With the development of computer science and automatic control, various automatic processing robots have been rapidly developed in various fields. In the field of industrial robots, automatic controlled welding robots have been widely used.

[0003] In the related art, the control process of the welding robot sets relevant operating data according to the process of manually controlling the robot to perform welding, and then the robot performs welding according to the set operating data based on the image captured by its camera.

[0004] However, in the above scheme, since the operating data is set according to the manual control process, the welding technical level and errors of the controller in the manual control process have a greater impact on the robot's automatic welding process, resulting in deviations in the welding process and reducing the accuracy of welding. Summary of the invention

[0005] Based on the above technical problems, the present application provides a welding robot control method, device, welding robot and readable medium to accurately locate the welding points, which is conducive to improving welding efficiency.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0007] According to one aspect of an embodiment of the present application, a welding robot control method is provided, comprising:

[0008] The three-dimensional camera at the end of the welding robot collects point cloud data of the three-dimensional ball on the target workpiece at multiple designated collection positions to obtain first point cloud data and a pose matrix corresponding to each collection position;

[0009] Performing data conversion on the first point cloud data according to the hand-eye transformation matrix and the posture matrix to obtain second point cloud data;

[0010] Determine the first sphere center and the second sphere center corresponding to each acquisition position of the three-dimensional sphere according to the first point cloud data and the second point cloud data, respectively, to obtain a first sphere center set and a second sphere center set;

[0011] Generate a compensation matrix of the hand-eye transformation matrix according to the first spherical center set and the second spherical center set;

[0012] According to the compensation matrix and the hand-eye transformation matrix, the welding robot is driven to move a welding gun end of the welding robot to a specified position of the target workpiece.

[0013] In some embodiments of the present application, based on the above technical solution, determining the first sphere center and the second sphere center corresponding to each acquisition position of the three-dimensional sphere according to the first point cloud data and the second point cloud data respectively to obtain the first sphere center set and the second sphere center set includes:

[0014] Sampling is performed according to the spatial distribution of the first point cloud data to obtain a first data set corresponding to the three-dimensional sphere;

[0015] Sampling is performed according to the spatial distribution of the second point cloud data to obtain a second data set corresponding to the three-dimensional sphere;

[0016] Spherical fitting is performed respectively according to the first data set and the second data set corresponding to the three-dimensional sphere, and the first sphere center of each acquisition position in the first point cloud data and the second sphere center of each acquisition position in the second point cloud data are determined to obtain a first sphere center set and a second sphere center set.

[0017] In some embodiments of the present application, based on the above technical solution, sampling is performed according to the spatial distribution of the first point cloud data to obtain a first data set corresponding to the three-dimensional sphere, including:

[0018] Filtering the first point cloud data according to a preset range on the three-dimensional coordinate axis to obtain filtered point cloud data;

[0019] According to a plurality of preset spatial boxes, for the filtered point cloud data falling into the same preset spatial box, extract the filtered point cloud data closest to the box center of the preset spatial box to obtain the sampled point cloud data, wherein the preset spatial box is obtained by dividing the three-dimensional space where the target workpiece is located;

[0020] Clustering is performed according to the spatial positions of the sampling point cloud data to obtain a first data set of the three-dimensional sphere.

[0021] In some embodiments of the present application, based on the above technical solution, generating a compensation matrix of the hand-eye transformation matrix according to the first sphere center set and the second sphere center set includes:

[0022] Performing bounding sphere fitting according to each sphere center in the second sphere center set to obtain a fitting sphere center;

[0023] Determine a set of inverse solution sphere centers of the fitting sphere in a three-dimensional camera coordinate system according to the fitting sphere centers and the hand-eye transformation matrix;

[0024] A compensation matrix of the hand-eye transformation matrix is ​​generated according to the inverse solution sphere center set and the first sphere center set.

[0025] In some embodiments of the present application, based on the above technical solution, generating a compensation matrix of the hand-eye transformation matrix according to the inverse solution sphere center set and the first sphere center set includes:

[0026] Determine a deviation mean value according to the deviation between the inverse solution sphere center set and the corresponding sphere center in the first sphere center set;

[0027] A compensation matrix of the hand-eye transformation matrix is ​​determined according to the deviation mean.

[0028] In some embodiments of the present application, based on the above technical solution, the three-dimensional camera at the end of the welding robot collects point cloud data of the three-dimensional ball on the target workpiece at multiple specified collection positions to obtain first point cloud data and a pose matrix corresponding to each collection position, including:

[0029] According to the preset path information, the end of the welding robot is driven to reach each designated collection position in sequence;

[0030] At each designated acquisition position, a frame of point cloud data of the target workpiece is captured by a three-dimensional camera at the end of the welding robot and a pose matrix of the current position is recorded, and the three-dimensional image contains the point cloud data of the three-dimensional sphere.

[0031] In some embodiments of the present application, based on the above technical solution, driving the welding robot to move the welding gun end of the welding robot to the specified position of the target workpiece according to the compensation matrix and the hand-eye transformation matrix includes:

[0032] Acquiring position information of a position to be welded on a target workpiece by means of the three-dimensional camera;

[0033] The position information of the position to be welded is converted into the welding robot coordinate system through the compensation matrix and the hand-eye transformation matrix to obtain the target position information;

[0034] According to the target position information, the end of the welding gun is driven to weld the position to be welded.

[0035] According to one aspect of an embodiment of the present application, there is provided a welding robot control device, comprising:

[0036] A point cloud data acquisition module is used to acquire point cloud data of a three-dimensional ball on a target workpiece at a plurality of designated acquisition positions through a three-dimensional camera at the end of the welding robot, and obtain first point cloud data and a posture matrix corresponding to each acquisition position;

[0037] A point cloud data conversion module, used for performing data conversion on the first point cloud data according to the hand-eye transformation matrix and the posture matrix to obtain second point cloud data;

[0038] A sphere center position determination module, used to determine the first sphere center and the second sphere center corresponding to each acquisition position of the three-dimensional sphere according to the first point cloud data and the second point cloud data, respectively, to obtain a first sphere center set and a second sphere center set;

[0039] A compensation matrix generation module, used to generate a compensation matrix of the hand-eye transformation matrix according to the first spherical center set and the second spherical center set;

[0040] The driving module is used to drive the welding robot to move the end of the welding gun of the welding robot to a specified position of the target workpiece according to the compensation matrix and the hand-eye transformation matrix.

[0041] According to one aspect of an embodiment of the present application, a welding robot is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute a welding robot control method as in the above technical solution by executing the executable instructions.

[0042] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the welding robot control method in the above technical solution is implemented.

[0043] In an embodiment of the present application, the point cloud data of the three-dimensional sphere on the target workpiece is collected by the three-dimensional camera at the end of the welding robot, and then the inverse solution in the camera coordinate system is determined based on the fitting result of the center of the three-dimensional sphere in the robot base coordinate system, and then the compensation matrix is ​​determined based on the inverse solution and the center of the collected three-dimensional sphere point cloud data, so as to compensate for the deviation between the end of the welding gun of the welding robot and the optical center of the three-dimensional camera, thereby accurately positioning the welding point, which is beneficial to improving welding efficiency.

[0044] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0046] Figure 1 The following schematically shows an exemplary system architecture diagram of the technical solution of the present application in an application scenario;

[0047] Figure 2 A schematic diagram of a target workpiece and a three-dimensional ball in an embodiment of the present application;

[0048] Figure 3 A schematic flow chart of a welding robot control method in an embodiment of the present application;

[0049] Figure 4 It is a schematic flow chart of the overall process of the welding robot control method in the embodiment of the present application;

[0050] Figure 5 The composition block diagram of the welding robot control device in the embodiment of the present application is schematically shown;

[0051] Figure 6 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0052] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0053] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0054] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0055] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0056] The application scenarios of the solution of this application are introduced below. Figure 1 The following schematically shows an exemplary system architecture diagram of the technical solution of the present application in an application scenario. Figure 1 As shown, the application scenario includes a welding robot 110, a 3D camera 120 and a target workpiece 130. The solution of the present application drives the welding robot 110 to weld the target workpiece 130 according to the 3D point cloud data captured by the 3D camera 120. Figure 1 As shown, a three-dimensional ball is disposed on the target workpiece 130. Depending on the specific shape of the target workpiece and the requirements of the welding position, the position of the three-dimensional ball may be different. For example, see Figure 2 , Figure 2 Schematic diagram of the target workpiece and the three-dimensional ball in the embodiment of the present application. Figure 2 As shown, during the shooting process of the 3D camera, the 3D ball will appear in the 3D image of the target workpiece according to the preset rules, and the 3D camera will shoot the 3D ball and the target workpiece from various angles. The welding robot will calculate the compensation matrix of the 3D camera and the end of the welding gun according to the center of the point cloud data of the 3D ball, so as to compensate and convert the position relationship between the end of the welding gun and the optical center of the 3D image when the welding robot performs welding according to the 3D image, so as to accurately control the position of the end of the welding gun and complete the welding process.

[0057] The following is a detailed description of the technical solution provided by this application in conjunction with the specific implementation method. Figure 3 , Figure 3 This is a schematic flow chart of a welding robot control method in an embodiment of the present application. The method can be applied to the above-mentioned welding robot. In the embodiment of the present application, the welding robot control method is introduced with the welding robot as the execution subject, and the payment method may include the following steps S310 to S350:

[0058] Step S310, using the three-dimensional camera at the end of the welding robot, collects point cloud data of the three-dimensional sphere on the target workpiece at multiple designated collection positions to obtain first point cloud data and a pose matrix corresponding to each collection position.

[0059] Specifically, the end of the welding robot will move according to a predetermined trajectory, stop at a designated acquisition position, and shoot the target workpiece through a three-dimensional camera. The data frame obtained by shooting will contain the point cloud data of the three-dimensional sphere on the target workpiece. Data collection is performed on the point cloud data of the three-dimensional sphere, so that the first point cloud data can be obtained. The first point cloud data will contain the point cloud data collected at each position. When collecting point cloud data, the welding robot will also record the pose matrix at that time. The pose matrix is ​​preset in the welding robot and is used to convert the position of the three-dimensional camera to the base coordinate system of the welding robot. The matrix can be determined by pre-measurement and calculation and recorded in the welding robot. When the welding robot stops the three-dimensional camera at the designated acquisition position, it can directly obtain the pose matrix corresponding to the acquisition position.

[0060] In one embodiment of the present application, the above step S310, through the three-dimensional camera at the end of the welding robot, collects point cloud data of the three-dimensional ball on the target workpiece at multiple designated collection positions to obtain first point cloud data and a pose matrix corresponding to each collection position, including:

[0061] According to the preset path information, the end of the welding robot is driven to reach each designated collection position in sequence;

[0062] At each designated acquisition position, a frame of point cloud data of the target workpiece is captured by a three-dimensional camera at the end of the welding robot and a pose matrix of the current position is recorded, and the three-dimensional image contains the point cloud data of the three-dimensional sphere.

[0063] Through the planned "e"-shaped spiral ascending motion trajectory, the welding robot is driven to multiple designated sampling points, and the point cloud data of the three-dimensional sphere is obtained by using the three-dimensional data camera at different angles in turn, and the current robot TCP transformation matrix toolPos is recorded to obtain the point cloud in the multi-frame camera coordinate system, and obtain the point cloud pcdn (n=1,2,…,n) and the transformation matrix toolPos (n=1,2,…,n), where n is the number of data frames sampled at each position.

[0064] Step S320, performing data conversion on the first point cloud data according to the hand-eye transformation matrix and the posture matrix to obtain second point cloud data.

[0065] Specifically, for the point cloud data of the three-dimensional ball collected at each position, the welding robot will transform the point cloud data according to the pose matrix of the corresponding position and the uncompensated hand-eye transformation matrix, thereby transforming the point cloud data to the base coordinate system of the welding robot. For example, the welding robot will transform the pcdn point cloud data from handEye to the robot's base coordinate system according to the uncompensated hand-eye transformation matrix, recorded as pcdn', where the pcdn' transformation relationship is:

[0066] pcdn'=transform(pcdn,toolPosn*handEye), where (n=1,2,...,n).

[0067] Through the above method, the first point cloud data pcdn and the second point cloud data pcdn' can be obtained.

[0068] Step S330: Determine the first sphere center and the second sphere center corresponding to each acquisition position of the three-dimensional sphere according to the first point cloud data and the second point cloud data, and obtain a first sphere center set and a second sphere center set.

[0069] Specifically, the welding robot performs spherical fitting according to the point cloud data corresponding to each position in the first point cloud data, thereby obtaining the first sphere center of each position as the first sphere center set, and performs spherical fitting according to the point cloud data corresponding to each position in the second point cloud data, thereby obtaining the second sphere center of each position as the second sphere center set. It can be understood that the sphere centers in the first sphere center set are the sphere center positions in the three-dimensional camera coordinate system, while the sphere centers in the second sphere center set are the sphere center positions in the robot's base coordinate system, and the sphere center positions in the second sphere center set have not been compensated, so there are errors.

[0070] In one embodiment of the present application, the above step of determining the first sphere center and the second sphere center corresponding to each acquisition position of the three-dimensional sphere according to the first point cloud data and the second point cloud data, respectively, to obtain the first sphere center set and the second sphere center set includes:

[0071] Sampling is performed according to the spatial distribution of the first point cloud data to obtain a first data set corresponding to the three-dimensional sphere;

[0072] Sampling is performed according to the spatial distribution of the second point cloud data to obtain a second data set corresponding to the three-dimensional sphere;

[0073] Spherical fitting is performed respectively according to the first data set and the second data set corresponding to the three-dimensional sphere, and the first sphere center of each acquisition position in the first point cloud data and the second sphere center of each acquisition position in the second point cloud data are determined to obtain a first sphere center set and a second sphere center set.

[0074] Sampling is performed according to the spatial distribution of the first point cloud data to obtain a first data set corresponding to the three-dimensional sphere, including:

[0075] Filtering the first point cloud data according to a preset range on the three-dimensional coordinate axis to obtain filtered point cloud data;

[0076] According to a plurality of preset spatial boxes, for the filtered point cloud data falling into the same preset spatial box, extract the filtered point cloud data closest to the box center of the preset spatial box to obtain the sampled point cloud data, wherein the preset spatial box is obtained by dividing the three-dimensional space where the target workpiece is located;

[0077] Clustering is performed according to the spatial positions of the sampling point cloud data to obtain a first data set of the three-dimensional sphere.

[0078] Specifically, the welding robot samples the point cloud data in the first point cloud data. Specifically, according to the preset information such as the specified size of the target workpiece and the overall relative position of the welding robot, the spatial value range of the point cloud data can be set, that is, the coordinate range in the three-dimensional coordinate system, usually based on the base coordinate system of the robot. When sampling, the point cloud data is first filtered according to the coordinates of the point cloud data in the three-dimensional coordinate system and the spatial value range, so as to filter out the data that is not in the value space. Then, for the filtered data, the three-dimensional space can be divided into multiple space boxes according to the predetermined sampling rules. The point cloud data of the three-dimensional sphere will fall into a certain space box. Each space box acts as a sampler. For the point cloud data in a space box, a representative data will be determined according to certain rules to replace all other point cloud data in the space box. For example, the point cloud data closest to the center of the box can be selected, or the mean of the three-dimensional coordinates of the point cloud data in the box can be calculated as the representative point cloud data. For the obtained sampled point cloud data, a clustering algorithm can be used to cluster according to the spatial position or the distance between them. The data belonging to the three-dimensional sphere will be classified into the same cluster, while the outlier data can be discarded to obtain the first data set. A sphere is fitted according to the first data set, and the position of the sphere center is determined according to the fitted sphere, that is, the coordinates of the sphere center in the three-dimensional camera coordinate system. The position of the sphere center is determined for the data at each acquisition position, thereby obtaining the first sphere center set.

[0079] The second sphere center set is processed in the same way as the first sphere center set. The second data set is obtained by filtering within a preset range, sampling a spatial box, and clustering. Then, a sphere is fitted based on the second data set to obtain the coordinates of the sphere center in the robot base coordinate system. The sphere center position is determined for the data at each acquisition position, thereby obtaining the second sphere center set.

[0080] Step S340: generating a compensation matrix of the hand-eye transformation matrix according to the first sphere center set and the second sphere center set.

[0081] Specifically, the welding robot can calculate a unique center based on the second sphere center set. For example, the minimum enclosing sphere is calculated based on each second sphere center, and the center of the minimum enclosing sphere or the second center closest to the center of the minimum enclosing sphere is used as the unique center. Alternatively, based on the spatial position of each sphere center in the second sphere center set, the distance between each sphere center in the three-dimensional space is calculated, and the position information of the unique center is determined based on the mean of the distance, for example, based on the mean of the distance. The unique center is mapped to the camera coordinate system, and then the compensation matrix is ​​determined based on the deviation between the unique center in the camera coordinate system and the first sphere center set.

[0082] In one embodiment of the present application, the above step generates a compensation matrix of the hand-eye transformation matrix according to the first sphere center set and the second sphere center set, including:

[0083] Performing bounding sphere fitting according to each sphere center in the second sphere center set to obtain a fitting sphere center;

[0084] Determine a set of inverse solution sphere centers of the fitting sphere in a three-dimensional camera coordinate system according to the fitting sphere centers and the hand-eye transformation matrix;

[0085] A compensation matrix of the hand-eye transformation matrix is ​​generated according to the inverse solution sphere center set and the first sphere center set.

[0086] In one embodiment of the present application, the above step of generating a compensation matrix of the hand-eye transformation matrix according to the inverse solution sphere center set and the first sphere center set includes:

[0087] Determine a deviation mean value according to the deviation between the inverse solution sphere center set and the corresponding sphere center in the first sphere center set;

[0088] A compensation matrix of the hand-eye transformation matrix is ​​determined according to the deviation mean.

[0089] After obtaining the first sphere center set cir1 and the second sphere center set cir2 in the first point cloud data and the second point cloud data, the minimum enclosing sphere is calculated for all the sphere centers in the second sphere center set cir2 of the point cloud set pcdn' to obtain the unique sphere center cir2' of the minimum enclosing sphere. Finally, the inverse solution of the sphere center cir2' in the camera coordinate system is obtained to obtain the sphere center set cir2" in the camera coordinate system.

[0090] Among them, the inverse solution method of the sphere center cir2' is:

[0091] cir2"=(toolPosn*handEye)-1*cir2'.

[0092] After obtaining the inverse solution set cir2' of the minimum enclosing sphere center cir2' of the point cloud pcdn sphere center in the camera coordinate system, the inverse solution set cir2' is deviated by ΔT from the corresponding sphere center in the point cloud pcd1 sphere center set cir1. i Calculate, calculate the average value of the ΔT set That is, the compensation matrix of the hand-eye transformation matrix. The final compensated hand-eye transformation matrix T T ' C for:

[0093]

[0094] Where T TC It is the representation of the 3D camera coordinate system C in the robot end tool coordinate system T, that is, the hand-eye transformation matrix before compensation.

[0095] Step S350: driving the welding robot to move the end of the welding gun of the welding robot to a specified position of the target workpiece according to the compensation matrix and the hand-eye transformation matrix.

[0096] According to the hand-eye transformation matrix and the obtained compensation matrix, the welding robot can calculate the specific position of the welding point of the target workpiece, thereby driving the end of the welding gun to weld the welding point of the target workpiece.

[0097] In one embodiment of the present application, the above step, according to the compensation matrix and the hand-eye transformation matrix, drives the welding robot to move the welding gun end of the welding robot to the specified position of the target workpiece, includes:

[0098] Acquiring position information of a position to be welded on a target workpiece by means of the three-dimensional camera;

[0099] The position information of the position to be welded is converted into the welding robot coordinate system through the compensation matrix and the hand-eye transformation matrix to obtain the target position information;

[0100] According to the target position information, the end of the welding gun is driven to weld the position to be welded.

[0101] In an embodiment of the present application, the point cloud data of the three-dimensional sphere on the target workpiece is collected by the three-dimensional camera at the end of the welding robot, and then the inverse solution in the camera coordinate system is determined based on the fitting result of the center of the three-dimensional sphere in the robot base coordinate system, and then the compensation matrix is ​​determined based on the inverse solution and the center of the collected three-dimensional sphere point cloud data, so as to compensate for the deviation between the end of the welding gun of the welding robot and the optical center of the three-dimensional camera, thereby accurately positioning the welding point, which is beneficial to improving welding efficiency.

[0102] The overall process of the welding robot control method in the embodiment of the present application is introduced below. Figure 4 . Figure 4 FIG. 1 is a schematic flow chart of the overall process of the welding robot control method in the embodiment of the present application. Figure 4 As shown, at the beginning of the method, in step 401, the welding robot starts the three-dimensional data camera to shoot the target workpiece, and in step 402, the three-dimensional ball point cloud data on the target workpiece is collected. Subsequently, in step 403, the three-dimensional ball point cloud data in the coordinate system is converted to obtain the three-dimensional ball point cloud data in the robot base coordinate system. Subsequently, in step 404, the three-dimensional ball point cloud data in the two coordinate systems are through-filtered to remove irrelevant point clouds, such as ground point clouds, and the three-dimensional point cloud data are uniformly sampled in step 405. In step 406, cluster segmentation is performed based on the result of uniform sampling to remove spatial noise points. Subsequently, in step 407, spherical fitting is performed on the three-dimensional ball point cloud to obtain the center of the three-dimensional ball. In step 408, it is determined whether the number of the currently obtained ball centers meets the number threshold. If not, return to step 401 to continue sampling to calculate the ball center. If it meets, in step 409, the center of the impromptu minimum enclosing sphere of all the ball centers in the robot base coordinate system is calculated to obtain the center of the enclosing sphere. Then, in step 410, the inverse solution of the center of the bounding sphere in the camera coordinate system is calculated. In step 411, the mean error between all the centers of the spheres in the camera coordinate system and the inverse solution centers in the x-axis, y-axis and z-axis directions is calculated. And in step 412, the hand-eye transformation matrix is ​​compensated using the mean error in the x-axis, y-axis and z-axis directions, so as to output the hand-eye compensation matrix of the robot in step 413.

[0103] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0104] The following introduces the device implementation of the present application, which can be used to execute the welding robot control method in the above-mentioned embodiment of the present application. Figure 5 The block diagram of the welding robot control device in the embodiment of the present application is schematically shown. Figure 5 As shown, the welding robot control device 500 mainly includes:

[0105] The point cloud data acquisition module 510 is used to acquire point cloud data of the three-dimensional ball on the target workpiece at multiple designated acquisition positions through the three-dimensional camera at the end of the welding robot, and obtain first point cloud data and a pose matrix corresponding to each acquisition position;

[0106] A point cloud data conversion module 520, configured to perform data conversion on the first point cloud data according to the hand-eye transformation matrix and the posture matrix to obtain second point cloud data;

[0107] A sphere center position determination module 530, configured to determine a first sphere center and a second sphere center corresponding to each acquisition position of the three-dimensional sphere according to the first point cloud data and the second point cloud data, respectively, to obtain a first sphere center set and a second sphere center set;

[0108] A compensation matrix generation module 540, configured to generate a compensation matrix of the hand-eye transformation matrix according to the first spherical center set and the second spherical center set;

[0109] The driving module 550 is used to drive the welding robot to move the end of the welding gun of the welding robot to a specified position of the target workpiece according to the compensation matrix and the hand-eye transformation matrix.

[0110] In one embodiment of the present application, in the above technical solution, the ball center position determination module 530 includes:

[0111] A first sampling unit, configured to perform sampling according to the spatial distribution of the first point cloud data to obtain a first data set corresponding to the three-dimensional sphere;

[0112] A second sampling unit, configured to perform sampling according to the spatial distribution of the second point cloud data to obtain a second data set corresponding to the three-dimensional sphere;

[0113] A sphere center determination unit is used to perform spherical fitting according to the first data set and the second data set corresponding to the three-dimensional sphere, determine the first sphere center of each acquisition position in the first point cloud data and the second sphere center of each acquisition position in the second point cloud data, and obtain a first sphere center set and a second sphere center set.

[0114] In one embodiment of the present application, in the above technical solution, the first sampling unit includes:

[0115] A filtering subunit, configured to filter the first point cloud data according to a preset range on the three-dimensional coordinate axis to obtain filtered point cloud data;

[0116] a sampling subunit, for extracting, for the filtered point cloud data falling into the same preset space box, the filtered point cloud data closest to the box center of the preset space box according to a plurality of preset space boxes, to obtain sampling point cloud data, wherein the preset space box is obtained by dividing the three-dimensional space where the target workpiece is located;

[0117] The clustering subunit is used to perform clustering division according to the spatial position of the sampling point cloud data to obtain the first data set of the three-dimensional sphere.

[0118] In one embodiment of the present application, the above technical solution, the compensation matrix generation module 540, includes:

[0119] An enclosing sphere fitting unit, configured to perform enclosing sphere fitting according to each sphere center in the second sphere center set to obtain a fitting sphere center;

[0120] An inverse solution determination unit, used to determine a set of inverse solution sphere centers of the fitting sphere in a three-dimensional camera coordinate system according to the fitting sphere centers and the hand-eye transformation matrix;

[0121] A matrix generating unit is used to generate a compensation matrix of the hand-eye transformation matrix according to the inverse solution sphere center set and the first sphere center set.

[0122] In one embodiment of the present application, in the above technical solution, the matrix generation unit includes:

[0123] a deviation determination subunit, configured to determine a deviation mean value according to deviations between the inverse solution sphere center set and corresponding sphere centers in the first sphere center set;

[0124] The matrix determination subunit is used to determine a compensation matrix of the hand-eye transformation matrix according to the deviation mean.

[0125] In one embodiment of the present application, in the above technical solution, the point cloud data acquisition module 510 includes:

[0126] A driving unit, used to drive the end of the welding robot to reach each designated collection position in sequence according to preset path information;

[0127] The shooting unit is used to shoot a frame of point cloud data of the target workpiece at each designated collection position through the three-dimensional camera at the end of the welding robot and record the pose matrix of the current position, and the three-dimensional image contains the point cloud data of the three-dimensional ball.

[0128] In one embodiment of the present application, in the above technical solution, the driving module 550 includes:

[0129] An information acquisition unit, used to acquire position information of a position to be welded on a target workpiece through the three-dimensional camera;

[0130] A coordinate conversion unit, used to convert the position information of the position to be welded into the welding robot coordinate system through the compensation matrix and the hand-eye transformation matrix to obtain the target position information;

[0131] The welding gun driving unit is used to drive the end of the welding gun to weld the position to be welded according to the target position information.

[0132] It should be noted that the apparatus provided in the above embodiment and the method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module performs the operation has been described in detail in the method embodiment and will not be repeated here.

[0133] Figure 6 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0134] It should be noted that Figure 6 The computer system 600 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0135] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603. In the RAM 603, various programs and data required for system operation are also stored. The CPU 601, the ROM 602 and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0136] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.

[0137] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 609, and / or installed from a removable medium 611. When the computer program is executed by a central processing unit (CPU) 601, various functions defined in the system of the present application are executed.

[0138] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0139] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0140] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0141] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0142] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0143] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A welding robot control method, characterized in that: include: The three-dimensional camera at the end of the welding robot collects point cloud data of the three-dimensional ball on the target workpiece at multiple designated collection positions to obtain first point cloud data and a pose matrix corresponding to each collection position; Performing data conversion on the first point cloud data according to the hand-eye transformation matrix and the posture matrix to obtain second point cloud data; Determine the first sphere center and the second sphere center corresponding to each acquisition position of the three-dimensional sphere according to the first point cloud data and the second point cloud data, respectively, to obtain a first sphere center set and a second sphere center set; Performing bounding sphere fitting according to each sphere center in the second sphere center set to obtain a fitting sphere center; Determine a set of inverse solution sphere centers of the fitting sphere in a three-dimensional camera coordinate system according to the fitting sphere centers and the hand-eye transformation matrix; Determine a deviation mean value according to the deviation between the inverse solution sphere center set and the corresponding sphere center in the first sphere center set; Determining a compensation matrix of the hand-eye transformation matrix according to the deviation mean; According to the compensation matrix and the hand-eye transformation matrix, the welding robot is driven to move a welding gun end of the welding robot to a specified position of the target workpiece.

2. The method according to claim 1, characterized in that The step of respectively determining the first sphere center and the second sphere center corresponding to each acquisition position of the three-dimensional sphere according to the first point cloud data and the second point cloud data to obtain a first sphere center set and a second sphere center set comprises: Sampling is performed according to the spatial distribution of the first point cloud data to obtain a first data set corresponding to the three-dimensional sphere; Sampling is performed according to the spatial distribution of the second point cloud data to obtain a second data set corresponding to the three-dimensional sphere; Spherical fitting is performed respectively according to the first data set and the second data set corresponding to the three-dimensional sphere, and the first sphere center of each acquisition position in the first point cloud data and the second sphere center of each acquisition position in the second point cloud data are determined to obtain a first sphere center set and a second sphere center set.

3. The method according to claim 2, characterized in that The sampling according to the spatial distribution of the first point cloud data to obtain a first data set corresponding to the three-dimensional sphere includes: Filtering the first point cloud data according to a preset range on the three-dimensional coordinate axis to obtain filtered point cloud data; According to a plurality of preset spatial boxes, for the filtered point cloud data falling into the same preset spatial box, extract the filtered point cloud data closest to the box center of the preset spatial box to obtain the sampled point cloud data, wherein the preset spatial box is obtained by dividing the three-dimensional space where the target workpiece is located; Clustering is performed according to the spatial positions of the sampling point cloud data to obtain a first data set of the three-dimensional sphere.

4. The method according to claim 2, characterized in that: The method uses a three-dimensional camera at the end of the welding robot to collect point cloud data of a three-dimensional ball on a target workpiece at multiple designated collection positions to obtain first point cloud data and a pose matrix corresponding to each collection position, including: According to the preset path information, the end of the welding robot is driven to reach each designated collection position in sequence; At each designated acquisition position, a frame of point cloud data of the target workpiece is captured by a three-dimensional camera at the end of the welding robot and a pose matrix of the current position is recorded.

5. The method according to claim 1, characterized in that The method of driving the welding robot to move the end of the welding gun of the welding robot to a specified position of the target workpiece according to the compensation matrix and the hand-eye transformation matrix includes: Acquiring position information of a position to be welded on a target workpiece by means of the three-dimensional camera; The position information of the position to be welded is converted into the welding robot coordinate system through the compensation matrix and the hand-eye transformation matrix to obtain the target position information; According to the target position information, the end of the welding gun is driven to weld the position to be welded.

6. A welding robot control device, characterized in that: include: A point cloud data acquisition module is used to acquire point cloud data of a three-dimensional ball on a target workpiece at a plurality of designated acquisition positions through a three-dimensional camera at the end of the welding robot, and obtain first point cloud data and a pose matrix corresponding to each acquisition position; A point cloud data conversion module, used for performing data conversion on the first point cloud data according to the hand-eye transformation matrix and the posture matrix to obtain second point cloud data; A sphere center position determination module, used to determine the first sphere center and the second sphere center corresponding to each acquisition position of the three-dimensional sphere according to the first point cloud data and the second point cloud data, respectively, to obtain a first sphere center set and a second sphere center set; a compensation matrix generation module, configured to fit a bounding sphere according to each sphere center in the second sphere center set to obtain a fitted sphere center, determine an inverse solution sphere center set of the fitted sphere in a three-dimensional camera coordinate system according to the fitted sphere center and the hand-eye transformation matrix, determine a deviation mean according to deviations between the inverse solution sphere center set and corresponding sphere centers in the first sphere center set, and determine a compensation matrix for the hand-eye transformation matrix according to the deviation mean; The driving module is used to drive the welding robot to move the end of the welding gun of the welding robot to a specified position of the target workpiece according to the compensation matrix and the hand-eye transformation matrix.

7. A welding robot, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the welding robot control method described in any one of claims 1 to 5 by executing the executable instructions.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the welding robot control method according to any one of claims 1 to 5 is implemented.

Images