Robot Hand-Eye Calibration Method, Device, Computer Equipment and Medium

Abstract

This application relates to the field of machine vision technology. Specifically, it relates to a method, device, computer equipment and medium for robot hand-eye calibration in the field of machine vision. To a certain extent, it can solve the problem of low calibration accuracy of the determined pose relationship caused by linear solution of hand-eye calibration. Determine the robot hand-eye calibration equation, as well as the initial rotation matrix and initial translation matrix of the hand-eye calibration equation; determine the target distance of the hand-eye calibration equation through the rotation distance and translation distance. If the target distance does not meet the convergence condition, based on the second-order cone programming model and the Kronecker product method, determine the alternative rotation matrix and alternative translation matrix; update the target rotation matrix, target translation matrix, and update the preset error distance; The embodiments of this application are based on the non-linear iterative optimization of the second-order cone programming model, optimizing both the rotation and translation parts simultaneously, and improving the pose relationship conversion accuracy of robot hand-eye calibration.

Description

Technical Field

[0001] The present application relates to the field of machine vision, and more particularly, to a robot hand-eye calibration method, apparatus, computer device, and medium. Background Art

[0002] With the development of intelligent manufacturing, higher requirements are put forward for the machine vision of robots. Among them, in machine vision, the vision guidance technology mainly obtains the rigid body transformation relationship between the coordinate system of the camera for capturing information and the coordinate system of the robot for performing tasks through hand-eye calibration, so that the robot can locate components or machines for operations such as component alignment.

[0003] In related technologies, hand-eye calibration obtains the pose (i.e., rotation and translation) relationship by collecting and recording the camera information and robot motion information of the robot in different motion postures, establishing a hand-eye calibration equation, and using methods such as the closed-form solution of linear solution. Among them, accurate closed-form solutions can only be obtained by linear solution without systematic errors.

[0004] However, during the hand-eye calibration process of the robot, the influence of repeated positioning errors, robot self-motion errors, camera imaging errors, etc. leads to low calibration accuracy of the pose relationship obtained by linear solution. Summary of the Invention

[0005] To solve the problem of low calibration accuracy of the determined pose relationship caused by linear solution of hand-eye calibration, the present application provides a robot hand-eye calibration method, apparatus, computer device, and medium.

[0006] The embodiments of the present application are implemented as follows:

[0007] The first aspect of the embodiments of the present application provides a robot hand-eye calibration method, including the following steps:

[0008] Determine the initial rotation matrix and the initial translation matrix of the robot hand-eye calibration equation, where the hand-eye calibration equation is determined with the camera fixedly installed on the end effector of the robot and the base of the robot as the world coordinate system;

[0009] If the target distance of the hand-eye calibration equation does not meet the convergence condition, determine the alternative rotation matrix and the alternative translation matrix based on the second-order cone programming model and the Kronecker product method, where the target distance is determined according to the rotation distance and the translation distance;

[0010] Update the target rotation matrix, the target translation matrix, and the preset error distance, where the target rotation matrix is determined according to the alternative rotation matrix and the initial rotation matrix, and the target translation matrix is determined according to the alternative translation matrix and the initial translation matrix; wherein, the preset error distance is used to determine the convergence condition.

[0011] A second aspect of the embodiments of the present application provides a robot hand-eye calibration device, including an acquisition module, a determination module, and an update module;

[0012] The acquisition module is configured to determine the initial rotation matrix and the initial translation matrix of the robot hand-eye calibration equation, where the hand-eye calibration equation is determined with the camera fixedly arranged on the end effector of the robot and with the base of the robot as the world coordinate system;

[0013] The determination module is configured to, if the target distance of the hand-eye calibration equation does not meet the convergence condition, determine the alternative rotation matrix and the alternative translation matrix based on the second-order cone programming model and the Kronecker product method, where the target distance is determined according to the rotation distance and the translation distance;

[0014] The update module is configured to update the target rotation matrix, the target translation matrix, and the preset error distance, where the target rotation matrix is determined according to the alternative rotation matrix and the initial rotation matrix, and the target translation matrix is determined according to the alternative translation matrix and the initial translation matrix; wherein, the preset error distance is used to determine the convergence condition.

[0015] A third aspect of the embodiments of the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the robot hand-eye calibration method in the first aspect are implemented.

[0016] A fourth aspect of the embodiments of the present application provides a computer storage medium, where a computer program is stored on the computer storage medium, and when the computer program is executed by the processor, the processor is caused to execute the steps of the robot hand-eye calibration method in the first aspect.

[0017] Advantages of the present application: By fixedly installing a camera on the end effector of a robot and using the robot's base as the world coordinate system, the robot hand-eye calibration equation, as well as the initial rotation matrix and initial translation matrix of the hand-eye calibration equation, can be determined; the target distance of the hand-eye calibration equation is determined through the rotation distance and translation distance. If the target distance does not meet the convergence condition, an alternative rotation matrix and an alternative translation matrix can be determined based on the second-order cone programming model and the Kronecker product method; the target rotation matrix can be updated according to the alternative rotation matrix and the initial rotation matrix, the target translation matrix can be updated according to the alternative translation matrix and the initial translation matrix, and the preset error distance can be updated. The updated preset error distance is used to update the convergence condition until the pose relationship is determined; the embodiments of the present application are based on the non-linear iterative optimization of the second-order cone programming model, simultaneously optimizing the rotation and translation parts, and improving the accuracy of the pose relationship conversion in robot hand-eye calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following-described drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 The structural schematic diagram of the vision system with the hand on the eye is shown;

[0020] Figure 2 The flowchart of a robot hand-eye calibration method provided by an embodiment of the present application is shown;

[0021] Figure 3 The flowchart of determining the initial rotation matrix and the initial translation matrix in an embodiment of the present application is shown;

[0022] Figure 4 The flowchart of target distance determination in an embodiment of the present application is shown;

[0023] Figure 5 The flowchart of determining the alternative rotation matrix and the alternative translation matrix in an embodiment of the present application is shown;

[0024] Figure 6 The structural schematic diagram of a robot hand-eye calibration device provided by an embodiment of the present application is shown;

[0025] Wherein, 10 - robot base; 20 - end effector of the robot; 30 - 3D camera; 40 - calibration object. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] To make the objectives, embodiments, and advantages of this application clearer, the following will clearly and completely describe the exemplary embodiments of this application in conjunction with the accompanying drawings in the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only a part, rather than all, of the embodiments of this application.

[0027] It should be noted that the brief description of the terms in this application is only for the convenience of understanding the embodiments described next, rather than intending to limit the embodiments of this application. Unless otherwise specified, these terms should be understood in their ordinary and general meanings.

[0028] In this application, the terms "first", "second", "third", etc. in the specification, claims, and the above-mentioned drawings are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0029] The terms "comprising" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components does not necessarily have to be limited to all the components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0030] Figure 1 The structural schematic diagram of the visual system with the hand on the eye is shown, as Figure 1 shown, the 3D camera is fixedly installed on the end effector of the robot (i.e., the eye is on the hand), and the calibration object is set in a coordinate system relatively fixed to the world coordinate system of the robot base; based on this, the coordinate system O of the 3D camera 30 is established C , the coordinate system O of the end effector 20 of the robot (such as a robotic arm) A , the world coordinate system O of the robot base 10 B , the coordinate system O of the calibration object 40 T .

[0031] Among them, represents the pose rigid body transformation relationship from the coordinate system O of the robot end effector A to the world coordinate system O of the robot base B ; represents the rigid body transformation relationship from the coordinate system O of the calibration object T to the world coordinate system O of the robot base B ; represents the rigid body transformation relationship from the camera coordinate system O C to the coordinate system O of the robot end effector A , represents the coordinate system O of the calibration object T to the camera coordinate system OC The rigid body transformation relationship.

[0032] The corresponding information can be directly and real-time read from the control panel of the robot; It can be obtained in real-time by a conventional 3D point cloud rough and fine registration algorithm; and are all unknown and constant.

[0033] The following relationships can be established for each coordinate system:

[0034]

[0035] By controlling the robot to take pictures of the calibration board in more than two different n postures, the following relationships can be obtained:

[0036]

[0037] …

[0038]

[0039] Through transformation, we can get:

[0040]

[0041] …

[0042]

[0043] In the formula, the superscript -1 is the inverse matrix;

[0044] Construct the standard hand-eye calibration equation AX = XB, where:

[0045]

[0046] As can be seen from the above, A and B are the attitude relationships between the robot base and the end effector, the camera and the calibration object between two consecutive time frames respectively, and X is the transformation relationship to be solved between the robot end effector and the 3D camera.

[0047] Hand-eye calibration is carried out by collecting and recording the camera information and the robot motion information of the robot in different motion postures, establishing the hand-eye calibration equation, and respectively obtaining the rotation matrix and translation matrix to be solved in the hand-eye calibration equation, that is, the attitude relationship, by means of the closed solution of linear solution and least squares.

[0048] However, accurate closed solutions can only be obtained by linear solution without systematic errors. There are influences such as repeated positioning errors, robot self-motion errors, and camera imaging errors in the robot's vision system, resulting in low calibration accuracy of the attitude relationship between the robot end effector and the 3D camera obtained by linear solution.

[0049] To solve the problem of low accuracy in the attitude relationship between the end effector of the robot and the 3D camera obtained linearly above, the embodiments of the present application propose a robot hand-eye calibration method, device, computer device, and medium. By fixing the camera on the end effector of the robot and using the base of the robot as the world coordinate system, the robot hand-eye calibration equation, as well as the initial rotation matrix and initial translation matrix of the hand-eye calibration equation, can be determined. The target distance of the hand-eye calibration equation is determined by the rotation distance and translation distance. If the target distance does not meet the convergence condition, based on the second-order cone programming model and the Kronecker product method, the alternative rotation matrix and alternative translation matrix can be determined. The target rotation matrix can be updated according to the alternative rotation matrix and the initial rotation matrix, the target translation matrix can be updated according to the alternative translation matrix and the initial translation matrix, and the preset error distance is updated. The updated preset error distance is used to update the convergence condition until the attitude relationship is determined. The embodiments of the present application are based on the non-linear iterative optimization of the second-order cone programming model, optimizing both the rotation and translation parts simultaneously, and improving the accuracy of the attitude relationship conversion in robot hand-eye calibration.

[0050] The following will detail the robot hand-eye calibration method, device, computer device, and medium of the embodiments of the present application with reference to the accompanying drawings.

[0051] Figure 2 The flowchart of a robot hand-eye calibration method provided by an embodiment of the present application is shown. As Figure 2 shown, an embodiment of the present application provides a robot hand-eye calibration method.

[0052] The robot hand-eye calibration method includes the following steps:

[0053] S110. Determine the initial rotation matrix and initial translation matrix of the robot hand-eye calibration equation. The hand-eye calibration equation is determined with the camera fixed on the end effector of the robot and using the base of the robot as the world coordinate system.

[0054] Among them, based on the hand-eye calibration equation of the robot, the initial rotation matrix and initial translation matrix can be obtained by linear solution methods such as least squares. Figure 3 The flowchart of determining the initial rotation matrix and initial translation matrix in an embodiment of the present application is shown. As Figure 3 shown, step 110 of determining the initial rotation matrix and initial translation matrix of the robot hand-eye calibration equation includes the following steps:

[0055] S111. Determine the deformed calibration equation based on the homogeneous matrix of the robot hand-eye calibration equation.

[0056] Based on the homogeneous matrix form of the hand-eye calibration equation AX = XB:

[0057]

[0058] In the formula, R A is the rotation matrix of matrix A, and R B is the rotation matrix of matrix B, and R X is the rotation matrix of matrix X, and t A , t B and t X are translation matrices

[0059] Among them, the rotation matrix is a 3*3 matrix, and the translation matrix is a 3*1 matrix.

[0060] The deformed calibration equation is obtained:

[0061] R A R X = R X R B

[0062] R A t X + t A = R X t B + t X

[0063] S112. Based on the deformed calibration equation, determine the equality relationships of the initial rotation matrix, the first rotation axis array, and the second rotation axis array.

[0064] Among them, the first rotation axis array is determined by the first rotation matrix of the robot in M poses, and the second rotation axis array is determined by the second rotation matrix of the robot in M poses.

[0065] For the vision system of the robot, for the rotation matrix R A and the rotation matrix R B corresponding to each pose i, the rotation axes n Ai and the rotation axis n Bi have the following relationships:

[0066] R X n Bi = n Ai

[0067] Then, for two poses (the first pose and the second pose), there are the following relationships:

[0068] R X (n B1 , n B2 ) = (n A1 , n A2 )

[0069] In summary, it can be obtained that:

[0070] R X (n B1 ,n B2 ,n B1 ×n B2 )=(n A1 ,n A2 ,n A1 ×n A2 )

[0071] For the hand-eye calibration process in M poses, an equation relationship can be generated:

[0072] N A =R X N B

[0073] where,

[0074] N A =(n A1 ,n A2 ,...,n AM ,n A1，A2 ,n A1，A3 ,...,n Ai,Aj )

[0075] N B =(n B1 ,n B2 ,...,n BM ,n B1，B2 ,n B1,B3 ,...,n Bi,Bj )

[0076] where 1 ≤ i, j ≤ M, N A is the first rotation axis array, and N B is the second rotation axis array.

[0077] S113. Determine the initial rotation matrix and the initial translation matrix based on the least squares and the equation relationship.

[0078] Obtain the initial rotation matrix R X :

[0079]

[0080] It can be obtained through the deformation calibration equation:

[0081] (R A -I)t X =R X t B -t A

[0082] Substitute the initial rotation matrix R X into the above equation and solve the above equation by least squares to obtain the initial translation vector t X .

[0083] As shown in Figure 2 it also includes: S120 determining the target distance of the hand-eye calibration equation according to the rotation distance and the translation distance.

[0084] For the target distance, it is obtained by calculating the maximum distance on both sides of the hand-eye calibration equation, and the distance is the sum value of the rotation distance and the translation distance. Among them, the maximum distance refers to: subtracting the matrices on both sides of the equation, and taking the modulus of the obtained matrix, that is, the two-norm.

[0085] Figure 4 shows the schematic flow chart of the target distance determination in the embodiment of the present application. As shown in Figure 4 it, step 120 determining the target distance of the hand-eye calibration equation according to the rotation distance and the translation distance includes the following steps:

[0086] S121, determining the rotation distance based on the rotation error.

[0087] Among them, the rotation error is determined by the robot under N sets of postures.

[0088] By collecting N sets of postures, N*(N-1)*0.5 equations are generated. i represents the serial number of the equation, and the rotation error is obtained as follows:

[0089]

[0090] The variables in the above formula are described in the above process and will not be elaborated here.

[0091] Convert the above formula to quaternion representation. A quaternion is a 4-dimensional vector, which can reduce the 9 parameters of the rotation matrix to 4 parameters. Specifically, it is as follows:

[0092]

[0093] In the formula, q is the symbol of the quaternion, that is, the rotation matrix R is represented by the quaternion q.

[0094] The error is represented by distance.

[0095] S122, determining the translation distance based on the translation error.

[0096] Among them, the translation error is determined by the robot under N sets of postures.

[0097] Similarly, the translation error can be obtained as follows:

[0098]

[0099] The variables in the formula are described in the above process and will not be elaborated here.

[0100] S123. Determine the target distance by the sum of the rotation distance and the translation distance.

[0101] Such as Figure 2 shown, it further includes: if the target distance of the hand-eye calibration equation does not meet the convergence condition, S130. Based on the second-order cone programming model and the Kronecker product method, determine the alternative rotation matrix and the alternative translation matrix.

[0102] Among them, the convergence condition is determined by the inequality relationship between the preset error distance, the target distance and the preset threshold;

[0103] Among them, the inequality relationship is obtained by the following formula:

[0104] abs(e - α) / e < μ

[0105] In the formula, α is the target distance, e is the preset error distance, μ is the preset threshold, and abs is the absolute value function.

[0106] Figure 5 shows the flow diagram for determining the alternative rotation matrix and the alternative translation matrix in the embodiment of the present application. Such as Figure 5 shown, in step 130, based on the second-order cone programming model and the Kronecker product method, determining the alternative rotation matrix and the alternative translation matrix includes the following steps:

[0107] S131. Based on the Kronecker product, vectorize both sides of the Sylvester equation to convert it into the first equation.

[0108] Among them, the Kronecker product is an operation between two matrices of any size and is a special form of the tensor product. Given two matrices M ∈ R i×j and N ∈ R p×q , then the Kronecker product of the two matrices is a block matrix in the space R ip×jq :

[0109]

[0110] The Sylvester equation AX + XB = C, combined with the Kronecker product method, can be transformed into the first equation:

[0111]

[0112] In the formula, I n and I m are the n-order and m-order identity matrices respectively, and vec is the row-wise arranged vector.

[0113] S132. Determine a second equation based on the hand-eye calibration equation and the first equation, where the second equation includes an alternative rotation matrix and an alternative translation matrix.

[0114] Combining the first equation of the above Kronecker product method and the hand-eye calibration equation deformation calibration equation, the second equation can be obtained. The reasoning process is as follows:

[0115]

[0116] The variables in the formula have been described in the above process and will not be elaborated here.

[0117] S133. Determine the alternative rotation matrix and the alternative translation matrix based on the second-order cone programming model and the second equation.

[0118] Among them, the standard form of the second-order cone programming model is:

[0119] min f T x

[0120] s.t.|||Ax + B||2 ≤ C T x + d

[0121] In the formula, f T is the transpose of the factor column vector of the unknown matrix x to be solved, x is the column vector of n-dimensional unknowns, and T is the transpose matrix of the matrix.

[0122] Combining the second equation in step 132, the corresponding algebraic expressions in the second-order cone programming model are as follows:

[0123]

[0124] C = [0 0 0 0 0 0 0 0 0 0 0 0 1] T

[0125] d = 0

[0126] In the formula, ε is the rotation and translation error value.

[0127] As Figure 2 shown, it also includes: S140. Update the target rotation matrix, the target translation matrix, and the preset error distance, where the target rotation matrix is determined according to the alternative rotation matrix and the initial rotation matrix, and the target translation matrix is determined according to the alternative translation matrix and the initial translation matrix.

[0128] Among them, the preset error distance is used to determine the convergence condition.

[0129] The target rotation matrix is updated by the following formula:

[0130] R = R K R′

[0131] In the formula, R is the updated target rotation matrix, R K is the alternative rotation matrix, and R' is the initial rotation matrix or the target rotation matrix updated last time;

[0132] The target translation matrix is updated by the following formula:

[0133] t = R K t'+t K

[0134] In the formula, t is the updated target translation matrix, R K is the alternative rotation matrix, t K is the alternative translation matrix, and t' is the initial translation matrix or the target matrix updated last time;

[0135] The preset error distance is updated by the following formula:

[0136] e = α

[0137]

[0138] In the formula, e is the updated preset error distance, and α is the target distance.

[0139] At this time, steps 120 and 130 will continue to be repeated until the convergence condition is met.

[0140] As Figure 2 shown, it also includes: when the target distance of the hand-eye calibration equation meets the convergence condition, S150, the corresponding target rotation matrix, and the target translation matrix are the pose relationships of hand-eye calibration.

[0141] At this time, it can also be said that the inequality for the target distance of the hand-eye calibration equation meets the convergence condition, that is, for the target distance α, the following formula is satisfied:

[0142] abs(e - α) / e < μ

[0143] At this time, the iteration is exited, and the optimized result of the final hand-eye calibration equation is obtained, that is, the target rotation matrix and the target translation matrix at this time are the pose relationships of hand-eye calibration.

[0144] Use linear solution of the hand-eye calibration equation to obtain the initial solution (initial rotation matrix and initial translation matrix) of nonlinear optimization; secondly, use the iterative nonlinear optimization algorithm based on the second-order cone, which can effectively improve the accuracy of solving the hand-eye calibration equation and enhance the anti-interference ability of the hand-eye calibration algorithm.

[0145] An embodiment of the present application proposes a robot hand-eye calibration method. By fixedly installing a camera on the end effector of the robot and using the base of the robot as the world coordinate system, the robot hand-eye calibration equation, as well as the initial rotation matrix and initial translation matrix of the hand-eye calibration equation, can be determined. The target distance of the hand-eye calibration equation is determined by the rotation distance and the translation distance. If the target distance does not meet the convergence condition, based on the second-order cone programming model and the Kronecker product method, the alternative rotation matrix and alternative translation matrix can be determined. The target rotation matrix can be updated according to the alternative rotation matrix and the initial rotation matrix, the target translation matrix can be updated according to the alternative translation matrix and the initial translation matrix, and the preset error distance can be updated. The updated preset error distance is used to update the convergence condition until the pose relationship is determined. The embodiment of the present application is based on the non-linear iterative optimization of the second-order cone programming model, optimizing both the rotation and translation parts simultaneously, and improving the pose relationship conversion accuracy of robot hand-eye calibration.

[0146] Figure 6 FIG. shows the schematic mechanism diagram of a robot hand-eye calibration device according to an embodiment of the present application, as Figure 6 shown, the robot hand-eye calibration device 600 includes an acquisition module 610, a determination module 620, and an update module 630.

[0147] The acquisition module is used to determine the initial rotation matrix and the initial translation matrix of the robot hand-eye calibration equation. The hand-eye calibration equation is determined by fixedly installing a camera on the end effector of the robot and using the base of the robot as the world coordinate system.

[0148] The determination module is used to determine the alternative rotation matrix and the alternative translation matrix based on the second-order cone programming model and the Kronecker product method if the target distance of the hand-eye calibration equation does not meet the convergence condition, where the target distance is determined according to the rotation distance and the translation distance.

[0149] The update module is used to update the target rotation matrix, the target translation matrix, and the preset error distance, where the target rotation matrix is determined according to the alternative rotation matrix and the initial rotation matrix, and the target translation matrix is determined according to the alternative translation matrix and the initial translation matrix; where the preset error distance is used to determine the convergence condition.

[0150] Among them, the convergence condition is determined by the inequality relationship between the preset error distance, the target distance, and the preset threshold.

[0151] When the target distance does not satisfy the inequality relationship, the target distance does not meet the convergence condition.

[0152] Among them, the inequality relationship is calculated by the following formula:

[0153] abs(e - α) / e < μ

[0154] Wherein, α is the target distance, e is the preset error distance, μ is the preset threshold, and abs is the absolute value function.

[0155] In some embodiments, the determination module is further configured to, when the target distance of the hand-eye calibration equation satisfies the convergence condition, the corresponding target rotation matrix and target translation matrix are the pose relationships of the hand-eye calibration.

[0156] In some embodiments, the determination module further includes an execution sub-module, and the execution sub-module is configured to determine the alternative rotation matrix and the alternative translation matrix based on the second-order cone programming model and the Kronecker product method, specifically including:

[0157] Based on the Kronecker product, vectorize both sides of the Sylvester equation and convert it into the first equation;

[0158] Based on the hand-eye calibration equation and the first equation, determine the second equation, wherein the second equation includes the alternative rotation matrix and the alternative translation matrix;

[0159] Based on the second-order cone programming model and the second equation, determine the alternative rotation matrix and the alternative translation matrix.

[0160] In some embodiments, the update module includes a rotation update sub-module, a translation update sub-module, and a preset error update sub-module; specifically configured to:

[0161] In the rotation update sub-module, the target rotation matrix is updated by the following formula:

[0162] R = R K R′

[0163] Wherein, R is the updated target rotation matrix, R K is the alternative rotation matrix, and R′ is the initial rotation matrix or the target rotation matrix of the previous update;

[0164] In the translation update sub-module, the target translation matrix is updated by the following formula:

[0165] t = R K t + t K

[0166] Wherein, t is the updated target translation matrix, R K is the alternative rotation matrix, t K is the alternative translation matrix, and t′ is the initial translation matrix or the target matrix of the previous update;

[0167] In the preset error update sub-module, the preset error distance is updated by the following formula:

[0168] e = α

[0169] Wherein, e is the updated preset error distance, and α is the target distance.

[0170] In some embodiments, the target distance is determined based on the rotational distance and the translational distance, including:

[0171] Determining the rotational distance based on the rotational error;

[0172] Determining the translational distance based on the translational error;

[0173] Determining the target distance through the sum of the rotational distance and the translational distance;

[0174] Wherein, the rotational error and the translational error are determined by the robot at N sets of poses.

[0175] In some embodiments, determining the initial rotation matrix and the initial translation matrix of the robot hand-eye calibration equation includes:

[0176] Determining the deformed calibration equation based on the homogeneous matrix of the robot hand-eye calibration equation;

[0177] Based on the deformed calibration equation, determining the equation relationship of the initial rotation matrix, the first rotation axis array and the second rotation axis array, wherein the first rotation axis array is determined by the first rotation matrix of the robot at M sets of poses, and the second rotation axis array is determined by the second rotation matrix of the robot at M sets of poses;

[0178] Determining the initial rotation matrix and the initial translation matrix based on the least squares and the equation relationship.

[0179] An embodiment of the present application proposes a robot hand-eye calibration device, including an acquisition module, a determination module and an update module. The acquisition module is fixedly arranged on the end effector of the robot with a camera, and with the base of the robot as the world coordinate system, the robot hand-eye calibration equation, as well as the initial rotation matrix and the initial translation matrix of the hand-eye calibration equation, can be determined; the determination module determines the target distance of the hand-eye calibration equation through the rotational distance and the translational distance. If the target distance does not meet the convergence condition, based on the second-order cone programming model and the Kronecker product method, the alternative rotation matrix and the alternative translation matrix can be determined; the update module can update the target rotation matrix according to the alternative rotation matrix and the initial rotation matrix, can update the target translation matrix according to the alternative translation matrix and the initial translation matrix, and update the preset error distance. The updated preset error distance is used to update the convergence condition until the pose relationship is determined; the embodiment of the present application is based on the non-linear iterative optimization of the second-order cone programming model, optimizing both the rotational and translational parts simultaneously, and improving the accuracy of the pose relationship conversion in robot hand-eye calibration.

[0180] The computer device provided by the embodiments of the present application further includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program, which is used to implement the above-mentioned robot hand-eye calibration method. The implementation principle and technical effect are similar to those of the above method embodiments, and will not be elaborated here.

[0181] The embodiments of the present application further provide a computer storage medium, on which a computer program is stored. The computer program is executed by the processor to implement the above-mentioned robot hand-eye calibration method. The implementation principle and technical effect are similar to those of the above method embodiments, and will not be elaborated here.

[0182] The following paragraphs will list and compare the Chinese terms involved in the specification of the present application and their corresponding English terms for easy reading and understanding.

[0183] For the sake of convenience in explanation, the above description has been made in combination with specific implementation manners. However, the above discussion in some embodiments is not intended to be exhaustive or to limit the implementation manners to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above implementation manners are for better explaining the principle and the actual application, so that those skilled in the art can better use the implementation manners and various different modified implementation manners suitable for specific use considerations.

Claims

1. A robot hand-eye calibration method, characterized in that, Including: Determine an initial rotation matrix and an initial translation matrix of a robot hand-eye calibration equation, where the hand-eye calibration equation is determined with a camera fixedly arranged on an end effector of the robot and with the base of the robot as a world coordinate system; If a target distance of the hand-eye calibration equation does not satisfy a convergence condition, determine an alternative rotation matrix and an alternative translation matrix based on a second-order cone programming model and a Kronecker product method, where the target distance is determined according to a rotation distance and a translation distance; Update a target rotation matrix, a target translation matrix, and a preset error distance, where the target rotation matrix is determined according to the alternative rotation matrix and the initial rotation matrix, and the target translation matrix is determined according to the alternative translation matrix and the initial translation matrix; where the preset error distance is used to determine the convergence condition; The convergence condition is determined by an inequality relationship between the preset error distance, the target distance, and a preset threshold; When the target distance does not satisfy the inequality relationship, the target distance does not satisfy the convergence condition; Wherein, the inequality relationship is obtained by calculating the following formula: abs(e - α) / e < μ In the formula, α is the target distance, e is the preset error distance, μ is the preset threshold, and abs is an absolute value function; The determining the alternative rotation matrix and the alternative translation matrix based on the second-order cone programming model and the Kronecker product method includes: Based on the Kronecker product, vectorize both sides of the Sylvester equation to convert it into a first equation; Based on the hand-eye calibration equation and the first equation, determine a second equation, where the second equation includes the alternative rotation matrix and the alternative translation matrix; Based on the second-order cone programming model and the second equation, determine the alternative rotation matrix and the alternative translation matrix; The updating the target rotation matrix, the target translation matrix, and the preset error distance includes: The target rotation matrix is updated by the following formula: R = R K R' where \(R\) is the updated target rotation matrix, \(R\) K K is the alternative rotation matrix, and \(R'\) is the initial rotation matrix or the target rotation matrix updated last time; The target translation matrix is updated by the following formula: t = R K t′ + t K where t is the updated target translation matrix, R K is an alternative rotation matrix, t K is an alternative translation matrix, and t′ is the initial translation matrix or the target matrix updated last time; The preset error distance is updated by the following formula: e＝α In the formula, e is the updated preset error distance, and α is the target distance; Continue to determine the target distance of the hand-eye calibration equation according to the rotation distance and the translation distance; Based on the second-order cone programming model and the Kronecker product method, determine the alternative rotation matrix and the alternative translation matrix until the convergence condition is satisfied.

2. The robot hand-eye calibration method according to claim 1, wherein, If the target distance of the hand-eye calibration equation satisfies the convergence condition, the corresponding target rotation matrix and target translation matrix are the pose relationships of the hand-eye calibration.

3. The robot hand-eye calibration method according to claim 1, characterized in that The target distance is determined according to the rotation distance and the translation distance, including: Determine the rotation distance based on the rotation error; Determine the translation distance based on the translation error; Determine the target distance through the sum value of the rotation distance and the translation distance; Wherein, the rotation error and the translation error are determined by the robot in N sets of poses.

4. The robot hand-eye calibration method according to claim 1, wherein, The determining the initial rotation matrix and the initial translation matrix of the robot hand-eye calibration equation includes: Based on the homogeneous matrix of the robot hand-eye calibration equation, determine a deformed calibration equation; Based on the deformation calibration equation, determine the equation relationships of the initial rotation matrix, the first rotation axis array, and the second rotation axis array, where the first rotation axis array is determined by the first rotation matrix of the robot in M poses, and the second rotation axis array is determined by the second rotation matrix of the robot in M poses; Based on the least squares and the equation relationships, determine the initial rotation matrix and the initial translation matrix.

5. A robot hand-eye calibration device, characterized in that, The robot hand-eye calibration method applicable to any one of claims 1 to 4 includes: An acquisition module, configured to determine an initial rotation matrix and an initial translation matrix of a robot hand-eye calibration equation, where the hand-eye calibration equation is determined with the camera fixedly arranged on the end effector of the robot and with the base of the robot as the world coordinate system; A determination module, configured to, if the target distance of the hand-eye calibration equation does not satisfy the convergence condition, determine an alternative rotation matrix and an alternative translation matrix based on a second-order cone programming model and a Kronecker product method, where the target distance is determined according to a rotation distance and a translation distance; An update module, configured to update a target rotation matrix, a target translation matrix, and a preset error distance, where the target rotation matrix is determined according to the alternative rotation matrix and the initial rotation matrix, and the target translation matrix is determined according to the alternative translation matrix and the initial translation matrix; where the preset error distance is used to determine the convergence condition.

6. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the robot hand-eye calibration method according to any one of claims 1 to 4 are implemented.

7. A computer storage medium, characterized in that, A computer program is stored on the computer storage medium. When the computer program is executed by the processor, the processor is caused to execute the steps of the robot hand-eye calibration method according to any one of claims 1 to 4.

Images