Robot hand-eye calibration method, device, computer equipment and medium

Abstract

This application relates to the field of machine vision technology. Specifically, it relates to a robot hand-eye calibration method, device, computer device, and medium, which can, to a certain extent, solve the problem of low calibration accuracy of the determined rigid body transformation relationship caused by linear solution of hand-eye calibration. Determine the hand-eye calibration equation based on the position where the camera is fixed on the end effector of the robot or outside the robot, and determine the initial rigid body transformation relationship; the objective function can be determined through the first alternative rigid body transformation relationship or the initial rigid body transformation relationship by Gauss-Newton iteration, and then the second alternative rigid body transformation relationship is further determined; if the currently updated second alternative rigid body transformation relationship is different from the previously updated first alternative rigid body transformation relationship, determine the corresponding error distance, and add the second alternative rigid body transformation relationship and the error distance to the alternative data; otherwise, screen out the target rigid body transformation relationship from the alternative data; the embodiments of this application improve the conversion accuracy of the rigid body transformation relationship in robot hand-eye calibration.

Description

Technical Field

[0001] The present application relates to the field of machine vision, and more particularly, to a robot hand-eye calibration method, apparatus, computer device, and medium. Background Art

[0002] With the development of intelligent manufacturing, higher requirements are put forward for the machine vision of robots. Among them, in machine vision, the vision guidance technology mainly obtains the rigid body transformation relationship between the coordinate system where the camera for capturing information is located and the coordinate system where the robot performing tasks is located through hand-eye calibration, so that the robot can locate components or machines for operations such as component alignment.

[0003] In the related art, hand-eye calibration obtains the attitude (i.e., rotation and translation) relationship by collecting and recording camera information and robot motion information of the robot in different motion postures, establishing a hand-eye calibration equation, and using methods such as the closed-form solution of linear solution. Among them, the linear solution can obtain an accurate closed-form solution only in the absence of systematic errors.

[0004] However, in the process of robot hand-eye calibration, there are influences such as repeated positioning errors, robot self-motion errors, and camera imaging errors, resulting in low calibration accuracy of the rigid body transformation relationship obtained by linear solution. Summary of the Invention

[0005] In order to solve the problem of low calibration accuracy of the determined rigid body transformation relationship caused by linear solution of hand-eye calibration, the present application provides a robot hand-eye calibration method, apparatus, computer device, and medium.

[0006] The embodiments of the present application are implemented as follows:

[0007] In a first aspect of the embodiments of the present application, a robot hand-eye calibration method is provided, including:

[0008] Determine an initial rigid body transformation relationship of the robot hand-eye calibration equation, where the hand-eye calibration equation is determined with the camera fixedly installed on the end effector of the robot or at a position outside the robot;

[0009] If the currently updated second alternative rigid body transformation relationship is different from the previously updated first alternative rigid body transformation relationship, determine the error distance corresponding to the second alternative rigid body transformation relationship, and add the second alternative rigid body transformation relationship and the error distance to the alternative data;

[0010] If the currently updated second alternative rigid body transformation relationship is the same as the previously updated first alternative rigid body transformation relationship, screen out the target rigid body transformation relationship from the alternative data;

[0011] Among them, both the first alternative rigid body transformation relationship and the second alternative rigid body transformation relationship are determined by an objective function, and the objective function is determined by Gaussian-Newton iteration of the first alternative rigid body transformation relationship or the initial rigid body transformation relationship.

[0012] In combination with the first aspect, in a possible implementation manner, after adding the second alternative rigid body transformation relationship and the error distance to the alternative data, it further includes:

[0013] When the currently updated iteration number is greater than a preset iteration threshold, filter out the target rigid body transformation relationship from the alternative data.

[0014] A second aspect of the embodiments of the present application provides a robot hand-eye calibration device, including an acquisition module, an analysis module, and a determination module;

[0015] The acquisition module is configured to determine the initial rigid body transformation relationship of the robot hand-eye calibration equation, and the hand-eye calibration equation is determined with the camera fixedly arranged on the end effector of the robot, or with the camera fixedly arranged at a position outside the robot;

[0016] The analysis module is configured to determine the first alternative rigid body transformation relationship or the second alternative rigid body transformation relationship based on an objective function, where the objective function is determined by Gaussian-Newton iteration of the first alternative rigid body transformation relationship or the initial rigid body transformation relationship;

[0017] The determination module is configured to, if the currently updated second alternative rigid body transformation relationship is different from the first alternative rigid body transformation relationship updated last time, determine the error distance corresponding to the second alternative rigid body transformation relationship, and add the second alternative rigid body transformation relationship and the error distance to the alternative data;

[0018] The determination module is further configured to, if the currently updated second alternative rigid body transformation relationship is the same as the first alternative rigid body transformation relationship updated last time, filter out the target rigid body transformation relationship from the alternative data.

[0019] A third aspect of the embodiments of the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the robot hand-eye calibration method in the first aspect are implemented.

[0020] A fourth aspect of the embodiments of the present application provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the processor is caused to execute the steps of the robot hand-eye calibration method in the first aspect.

[0021] Advantages of the present application: By fixedly installing a camera on the end effector of a robot, or by fixedly installing the camera at a position outside the robot, the hand-eye calibration equation can be determined to obtain the initial rigid body transformation relationship. The target function can be determined through the first alternative rigid body transformation relationship or the initial rigid body transformation relationship by means of Gauss-Newton iteration. Based on the target function, the second alternative rigid body transformation relationship is determined. If the currently updated second alternative rigid body transformation relationship is different from the previously updated first alternative rigid body transformation relationship, the error distance corresponding to the second alternative rigid body transformation relationship is determined, and the second alternative rigid body transformation relationship and the error distance are added to the alternative data. If the currently updated second alternative rigid body transformation relationship is the same as the previously updated first alternative rigid body transformation relationship, the target rigid body transformation relationship is screened out from the alternative data. The embodiments of the present application improve the conversion accuracy of the rigid body transformation relationship in robot hand-eye calibration based on the non-linear iteration of Gauss-Newton. Description of the Drawings

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 Shows a schematic structural diagram of a vision system with the hand on the eye;

[0024] Figure 2 Shows a schematic flow diagram of a robot hand-eye calibration method provided by an embodiment of the present application;

[0025] Figure 3 Shows a schematic flow diagram of determining the initial rigid body transformation relationship in an embodiment of the present application;

[0026] Figure 4 Shows a schematic flow diagram of determining the second alternative rigid body transformation relationship in an embodiment of the present application;

[0027] Figure 5 Shows a schematic structural diagram of a robot hand-eye calibration device provided by an embodiment of the present application;

[0028] Among them, 10 - robot base; 20 - end effector of the robot; 30 - 3D camera; 40 - calibration object. Detailed Embodiments

[0029] To make the objectives, implementation manners, and advantages of this application clearer, the following will clearly and completely describe the exemplary implementation manners of this application with reference to the accompanying drawings in the exemplary embodiments of this application. Apparently, the described exemplary embodiments are only a part rather than all of the embodiments of this application.

[0030] It should be noted that the brief description of the terms in this application is only for facilitating the understanding of the subsequent described implementation manners, rather than intending to limit the implementation manners of this application. Unless otherwise specified, these terms should be understood in their ordinary and common meanings.

[0031] In this application, terms such as "first", "second", "third", etc. in the description, claims, and the above-mentioned accompanying drawings are used to distinguish similar or homogeneous objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0032] The terms "include" and "have" and any of their variations are intended to cover but not exclusively include. For example, a product or device including a series of components does not necessarily have to be limited to all the clearly listed components, but may include other components that are not clearly listed or are inherent to these products or devices.

[0033] Eye-hand calibration refers to solving the coordinate transformation relationship between the end coordinate system of a robot (mainly an industrial robot in this application) and the camera coordinate system, or the coordinate transformation relationship between the base coordinate system of the robot and the camera coordinate system. There are two situations for eye-hand calibration: The first is that the camera (eye) is fixed at the end of the end effector (hand), the camera is fixed relative to the end of the end effector, and the camera moves with the end effector. This type of eye-hand calibration is called Eye-in-hand; the second is that the camera (eye) and the end effector (hand) are separated, the camera is fixed relative to the base of the robot, and the movement of the end effector has no impact on the camera. This type of eye-hand calibration is called Eye-to-hand.

[0034] Taking Eye-in-hand as an example, Figure 1 shows a schematic structural diagram of a vision system with the hand on the eye, as Figure 1 shown, the 3D camera is fixedly installed on the end effector of the robot (i.e., the eye is on the hand), and the calibration object is set in a coordinate system that is relatively fixed with respect to the world coordinate system of the robot base; based on this, the coordinate system O of the 3D camera 30 is established C , the coordinate system O of the end effector 20 (such as a robotic arm) of the robot A , the world coordinate system O of the robot base 10 B , and the coordinate system O of the calibration object 40 T .

[0035] Among them, represents the rigid body transformation relationship from the coordinate system O of the robot end effector A to the world coordinate system O of the robot base B ; represents the rigid body transformation relationship from the coordinate system OT of the calibration object to the world coordinate system O of the robot base B ; represents the rigid body transformation relationship from the camera coordinate system O C to the coordinate system O of the robot end effector A , represents the rigid body transformation relationship from the coordinate system OT of the calibration object to the camera coordinate system O C .

[0036] can be directly and real-time read by the control panel of the robot to obtain the corresponding information; can be obtained in real time by a conventional 3D point cloud coarse-fine combined registration algorithm; and are both unknown and constant.

[0037] The following relational expressions can be established for each coordinate system:

[0038]

[0039] Controlling the robot to take pictures of the calibration board in more than two different poses of n times can obtain the following relationship:

[0040]

[0041] …

[0042]

[0043] Through transformation, we can get:

[0044]

[0045] …

[0046]

[0047] In the formula, the superscript -1 represents the inverse matrix;

[0048] Construct the standard hand-eye calibration equation AX = XB, where:

[0049]

[0050]

[0051]

[0052] As described above, A and B are the rigid body transformation relationships between the robot base and the end effector, the camera and the calibration object respectively between two consecutive time frames, and X is the transformation relationship to be determined between the robot end effector and the 3D camera.

[0053] Eye-in-hand calibration collects and records camera information and robot motion information when the robot is in different motion postures, establishes the eye-in-hand calibration equation, and respectively obtains the rotation matrix and translation matrix to be solved in the eye-in-hand calibration equation, that is, the rigid body transformation relationship, by means of the closed solution of linear solution and least squares.

[0054] However, linear solution can obtain an accurate closed solution only when there is no systematic error. There are influences such as repeated positioning error, robot self-motion error, and camera imaging error in the robot vision system, resulting in low calibration accuracy of the rigid body transformation relationship between the robot end effector and the 3D camera obtained by linear solution.

[0055] To solve the problem of low accuracy of the rigid body transformation relationship obtained linearly above, the embodiments of the present application provide a robot eye-in-hand calibration method, device, computer device and medium. When the camera is fixedly arranged on the end effector of the robot or at a position outside the robot, the eye-in-hand calibration equation is determined, and the initial rigid body transformation relationship can be determined; the objective function can be determined by the first alternative rigid body transformation relationship or the initial rigid body transformation relationship through Gauss-Newton iteration; based on the objective function, the second alternative rigid body transformation relationship is determined. If the currently updated second alternative rigid body transformation relationship is different from the previously updated first alternative rigid body transformation relationship, the error distance corresponding to the second alternative rigid body transformation relationship is determined, and the second alternative rigid body transformation relationship and the error distance are added to the alternative data; if the currently updated second alternative rigid body transformation relationship is the same as the previously updated first alternative rigid body transformation relationship, the target rigid body transformation relationship is screened out from the alternative data.

[0056] The following details the robot eye-in-hand calibration method, device, computer device and medium of the embodiments of the present application with reference to the accompanying drawings.

[0057] Figure 2 The flowchart of a robot eye-in-hand calibration method provided by the embodiments of the present application is shown. As Figure 2 shown, the embodiments of the present application provide a robot eye-in-hand calibration method.

[0058] The robot eye-in-hand calibration method includes the following steps:

[0059] S110. Determine the initial rigid body transformation relationship of the robot eye-in-hand calibration equation, where the eye-in-hand calibration equation is determined with the camera fixedly arranged on the end effector of the robot or at a position outside the robot.

[0060] For the solution of the hand-eye calibration equation, it includes the solution of the hand-eye calibration equation with the eye-to-hand configuration and the solution of the hand-eye calibration equation with the eye-to-object configuration.

[0061] Among them, for the solution of the hand-eye calibration equation with the eye-to-hand configuration: Given the transformation relationship from the end-effector coordinate system to the world reference coordinate system and the transformation relationship from the calibration board coordinate system to the camera coordinate system, obtain the rigid body transformation relationship from the camera to the end-effector coordinate system.

[0062] For the solution of the hand-eye calibration equation with the eye-to-object configuration: Given the transformation relationship from the world reference coordinate system to the end-effector coordinate system and the transformation relationship from the calibration board coordinate system to the camera coordinate system, obtain the rigid body transformation relationship from the camera to the world reference coordinate system.

[0063] In step 110, based on the hand-eye calibration equation of the robot, the initial rigid body transformation relationship (including the initial rotation matrix and the initial translation matrix) can be obtained by linear solution methods such as least squares, and it also includes but is not limited to common two-step methods such as the Tsai-Lenz algorithm, the NAVY algorithm (Park), the INRIA algorithm (Horaud), etc., which first obtain the initial rotation matrix and then obtain the initial translation matrix, to obtain the initial rotation matrix and the initial translation matrix of the hand-eye calibration equation.

[0064] Figure 3 The flowchart showing the determination of the initial rigid body transformation relationship in the embodiment of the present application is as follows Figure 3 As shown, step 110 for determining the initial rigid body transformation relationship of the robot hand-eye calibration equation includes the following steps:

[0065] S111. Determine the deformed calibration equation based on the homogeneous matrix of the robot hand-eye calibration equation.

[0066] Based on the homogeneous matrix form of the hand-eye calibration equation AX = XB:

[0067]

[0068] In the formula, R A is the rotation matrix of matrix A, R B is the rotation matrix of matrix B, R X is the rotation matrix of matrix X, t A , t B and t X are translation matrices; among them, the rotation matrix is a 3*3 matrix, and the translation matrix is a 3*1 matrix.

[0069] Obtain the deformed calibration equation:

[0070] R A R X = R X R B

[0071] R A t X +t A =R X t B +t X

[0072] S112. Determine the equation relationships of the initial rotation matrix, the first rotation axis array, and the second rotation axis array based on the deformation calibration equation.

[0073] Among them, the first rotation axis array is determined by the first rotation matrix of the robot in M poses, and the second rotation axis array is determined by the second rotation matrix of the robot in M poses.

[0074] For the vision system of the robot, the rotation matrix R A and the rotation matrix R B corresponding rotation axis n Ai and the rotation axis n Bi have the following relationships:

[0075] R X n Bi =n Ai

[0076] Then, for two poses (the first pose and the second pose), there are the following relationships:

[0077] R X (n B1 , n B2 )=(n A1 , n A2 )

[0078] In summary, it can be obtained that:

[0079] R X (n B1 , n B2 , n B1 ×n B2 )=(n A1 , n A2 , n A1 ×n A2 )

[0080] For the hand-eye calibration process in M poses, equation relationships can be generated:

[0081] N A =R X N B

[0082] Among them,

[0083] N A=(n A1 , n A2 , …, n AM , n A1，A2 , n A1，A3 , …, n Ai，Aj )

[0084] N B =(n B1 , n B2 , …, n BM , n B1，B2 , n B1，B3 , …, n Bi，Bj )

[0085] where 1 ≤ i, j ≤ M, N A is the first rotation axis array, and N B is the second rotation axis array.

[0086] S113. Determine the initial rotation matrix and the initial translation matrix based on the least squares and the equality relationship.

[0087] Obtain the initial rotation matrix R X :

[0088]

[0089] Alternatively, let the covariance matrix and obtain the initial rotation matrix R X :

[0090]

[0091] In the formula, k is the total number of the motion postures of the end effector, T is the matrix transpose, and n Bi , n Ai are the rotation axes of the rotation matrices A and B corresponding to the posture i, respectively.

[0092] The following can be obtained by transforming the calibration equation:

[0093] (R A - I)t X = R X t B - t A

[0094] Substitute the initial rotation matrix R X into the above formula and solve the above equation by least squares to obtain the initial translation vector t X .

[0095] For example Figure 2As shown, it further includes S120, and determines the first alternative rigid body transformation relationship or the second alternative rigid body transformation relationship through an objective function, where the objective function is determined by Gaussian-Newton iteration of the first alternative rigid body transformation relationship or the initial rigid body transformation relationship.

[0096] It should be understood that the currently updated second alternative rigid body transformation relationship and the previously updated first alternative rigid body transformation relationship refer to any two adjacent alternative rigid body transformation relationships. In the process of continuous iterative judgment, any two adjacent alternative rigid body transformation relationships can be referred to as the currently updated second alternative rigid body transformation relationship and the previously updated first alternative rigid body transformation relationship.

[0097] That is to say, if the objective function is determined by Gaussian-Newton iteration of the initial rigid body transformation relationship, the first alternative rigid body transformation relationship can be determined through the objective function; if the objective function is determined by Gaussian-Newton iteration of the first alternative rigid body transformation relationship, the second alternative rigid body transformation relationship can be determined through the objective function, and each second alternative rigid body transformation relationship can be used as the first alternative rigid body transformation relationship in the next iteration to obtain a new second alternative rigid body transformation relationship.

[0098] When the objective function is determined by Gaussian-Newton iteration of the initial rigid body transformation relationship, it includes:

[0099] Establish an objective function, which includes a first objective function and a second objective function, and the first objective function and the second objective function are obtained through the following formula:

[0100] F1(ω X )=R A exp([ω X ^)-exp([ω X ^)R B

[0101] F2(ω X ,t X )=exp([ω X ^)t B +(I-R A )t X -t A

[0102] Where,

[0103] In the formula, R X is the initial rotation matrix in the initial rigid body transformation relationship, t X is the initial translation matrix in the initial rigid body transformation relationship, ω X is the rotation axis corresponding to the initial rotation matrix in the initial rigid body transformation relationship, [ω X^ is the cross product matrix corresponding to the rotation axis, F1 is the first objective function, F2 is the second objective function, R A 、R B is the rotation matrix of the hand-eye calibration equation, t A 、t B is the translation matrix of the hand-eye calibration equation;

[0104] Among them, In the formula, ω X is a three-dimensional vector, ω x 、ω y 、ω z are the three variables of ω X .

[0105] Or, when the objective function is determined by Gauss-Newton iteration of the first alternative rigid body transformation relationship, an objective function is established based on the first alternative rigid body transformation relationship. At this time, the variables in the objective function are determined by the first alternative rigid body transformation relationship. For example, R X at this time is the first alternative rotation matrix in the first alternative rigid body transformation relationship, t X at this time is the first alternative translation matrix in the first alternative rigid body transformation relationship, ω X at this time is the rotation axis corresponding to the first alternative rotation matrix in the first alternative rigid body transformation relationship. Figure 4 shows a schematic flow chart for determining the second alternative rigid body transformation relationship in an embodiment of the present application; as Figure 4 shown, determining the second alternative rigid body transformation relationship includes the following steps:

[0106] S121. Establish an objective function, which includes a first objective function and a second objective function.

[0107] This step is the same as the above process when the objective function is determined by Gauss-Newton iteration of the initial rigid body transformation relationship, and will not be repeated here.

[0108] S122. Based on the objective function, determine the Jacobian matrix, the target optimization value, and the objective function value.

[0109] Step 122 determines the Jacobian matrix, the target optimization value, and the objective function value based on the objective function, including the following process:

[0110] First, determine the partial derivatives of the rotation axis corresponding to the initial rotation matrix in the objective function and the initial translation matrix.

[0111] Second, through the partial derivatives, determine the Jacobian matrix, the target optimization value, and the objective function value. Among them, the Jacobian matrix, the target optimization value, and the objective function value are calculated by the following formula:

[0112]

[0113]

[0114]

[0115] In the formula, J is the Jacobian matrix, Δx is the target optimization value, F is the target function value, and F1 col1 is the first column of the first target function matrix.

[0116] S123. Determine the equation relationship through the Jacobian matrix, the target optimization value, and the target function value, including:

[0117] The equation relationship is obtained by the following formula:

[0118] JΔx = F

[0119] where J = [J1, J2,..., J n T | 12n×6 , F = [F1, F2,..., F n T | 12n×1 ;

[0120] S124. Determine the target optimization value from the equation relationship by the least squares method.

[0121] S125. Determine the second alternative rigid body transformation relationship based on the target optimization value.

[0122] The target optimization value includes the initial translation matrix in the initial rigid body transformation relationship and the initial rotation matrix in the initial rigid body transformation relationship, or the target optimization value includes the first alternative translation matrix in the first alternative rigid body transformation relationship and the first alternative rotation matrix in the first alternative rigid body transformation relationship, specifically including:

[0123]

[0124]

[0125] In the formula, is the second alternative rotation matrix in the second alternative rigid body transformation relationship, is the second alternative translation matrix in the second alternative rigid body transformation relationship, is the first alternative rotation matrix in the first alternative rigid body transformation relationship, the first alternative translation matrix in the first alternative rigid body transformation relationship.

[0126] Such as Figure 2 ​​As shown, it further includes: if the currently updated second alternative rigid body transformation relationship is different from the previously updated first alternative rigid body transformation relationship, S130, determining the error distance corresponding to the second alternative rigid body transformation relationship, and adding the second alternative rigid body transformation relationship and the error distance to the alternative data.

[0127] In step 130, determining the error distance corresponding to the second alternative rigid body transformation relationship includes:

[0128] First, obtain the rotation error and the translation error. Among them, by collecting N groups of postures, multiple equations are generated. i represents the serial number of the equation. The rotation error is calculated by the following formula:

[0129]

[0130] The variables in the formula are described in the above process and will not be elaborated here.

[0131] Convert the above formula to quaternion representation. A quaternion is a 4-dimensional vector, which can reduce the 9 parameters of the rotation matrix to 4 parameters. Specifically as follows:

[0132]

[0133] In the formula, q is the symbol of the quaternion, that is, the rotation matrix R is represented by the quaternion q.

[0134] The translation error is calculated by the following formula:

[0135]

[0136] The variables in the formula are described in the above process and will not be elaborated here.

[0137] Secondly, determine the error distance through the sum of the rotation error and the translation error.

[0138] In step 130, after determining the error distance corresponding to the second alternative rigid body transformation relationship, add the second alternative rigid body transformation relationship and the error distance to the alternative data.

[0139] In some embodiments, the alternative data is a data set, which can be stored in the form of a database or in the form of a file in the cache.

[0140] In some embodiments, after step 130, it further includes: if the currently updated iteration number is greater than the preset iteration threshold, S140, screening out the target rigid body transformation relationship from the alternative data.

[0141] Otherwise, repeat the relevant content of step 120 and step 130.

[0142] Such as Figure 2As shown, it further includes: if the currently updated second alternative rigid body transformation relationship is the same as the previously updated first alternative rigid body transformation relationship, S150, screening out the target rigid body transformation relationship from the alternative data.

[0143] It should be understood that the target rigid body transformation relationship is screened out from the alternative data, and the target rigid body transformation relationship is the alternative rigid body transformation relationship corresponding to the minimum error distance.

[0144] The target rigid body transformation relationship is used as the optimal hand-eye matrix, that is, the corresponding target rotation matrix and target translation matrix are the pose relationships of hand-eye calibration.

[0145] Use linear solution of the hand-eye calibration equation to obtain the initial rotation matrix and initial translation matrix of nonlinear optimization; secondly, use the nonlinear optimization algorithm based on Gauss-Newton iteration, which can effectively improve the accuracy of solving the hand-eye calibration equation and improve the anti-interference ability of the hand-eye calibration algorithm.

[0146] An embodiment of the present application proposes a robot hand-eye calibration method. The hand-eye calibration equation is determined with the camera fixedly installed on the end effector of the robot or at a position outside the robot, and the initial rigid body transformation relationship can be determined; the first alternative rigid body transformation relationship or the initial rigid body transformation relationship can be used to determine the objective function through Gauss-Newton iteration; based on the objective function, determine the second alternative rigid body transformation relationship. If the currently updated second alternative rigid body transformation relationship is different from the previously updated first alternative rigid body transformation relationship, determine the error distance corresponding to the second alternative rigid body transformation relationship, and add the second alternative rigid body transformation relationship and the error distance to the alternative data; if the currently updated second alternative rigid body transformation relationship is the same as the previously updated first alternative rigid body transformation relationship, screen out the target rigid body transformation relationship from the alternative data, and use the nonlinear optimization algorithm based on Gauss-Newton iteration to improve the accuracy of solving the hand-eye calibration equation.

[0147] Figure 5 The mechanism diagram of a robot hand-eye calibration device according to an embodiment of the present application is shown, as Figure 5 As shown, the robot hand-eye calibration device 500 includes an acquisition module 510, an analysis module 520, and a determination module 530.

[0148] The acquisition module is used to determine the initial rigid body transformation relationship of the robot hand-eye calibration equation, and the hand-eye calibration equation is determined with the camera fixedly installed on the end effector of the robot or at a position outside the robot;

[0149] The analysis module is used to determine the first alternative rigid body transformation relationship or the second alternative rigid body transformation relationship based on the objective function, where the objective function is determined by Gauss-Newton iteration of the first alternative rigid body transformation relationship or the initial rigid body transformation relationship;

[0150] A determination module, configured to determine an error distance corresponding to a second alternative rigid body transformation relationship if the currently updated second alternative rigid body transformation relationship is different from the previously updated first alternative rigid body transformation relationship, and add the second alternative rigid body transformation relationship and the error distance to alternative data;

[0151] The determination module is further configured to screen out a target rigid body transformation relationship from the alternative data if the currently updated second alternative rigid body transformation relationship is the same as the previously updated first alternative rigid body transformation relationship.

[0152] In some embodiments, the determination module further includes, after adding the second alternative rigid body transformation relationship and the error distance to the alternative data, screening out a target rigid body transformation relationship from the alternative data if the currently updated number of iterations is greater than a preset iteration threshold.

[0153] In some embodiments, the judgment module includes a data analysis sub-module, configured to obtain a rotation error and a translation error; and determine the error distance through the sum value of the rotation error and the translation error.

[0154] In some embodiments, the analysis module includes a first analysis sub-module, configured to determine a second alternative rigid body transformation relationship through an objective function, specifically including:

[0155] Based on the objective function, determine a Jacobian matrix, a target optimization value, and an objective function value;

[0156] Determine the target optimization value from the equation relationship through least squares, where the equation relationship is determined by the Jacobian matrix, the target optimization value, and the objective function value;

[0157] Based on the target optimization value, determine the second alternative rigid body transformation relationship.

[0158] In some embodiments, the analysis module includes a second analysis sub-module, configured to, when the objective function is determined by Gauss-Newton iteration of an initial rigid body transformation relationship, include:

[0159] Establish an objective function, where the objective function includes a first objective function and a second objective function, and the first objective function and the second objective function are obtained through the following formula:

[0160] F1(ω X )=R A exp([ω X ^)-exp([ω X ^)R B

[0161] F2(ω X ,t X )=exp([ω X ^)t B+(I - R A )t X -t A

[0162] wherein,

[0163] In the formula, R X is the initial rotation matrix in the initial rigid body transformation relationship, t X is the initial translation matrix in the initial rigid body transformation relationship, ω X is the rotation axis corresponding to the initial rotation matrix in the initial rigid body transformation relationship, [ω X ^ is the cross product matrix corresponding to the rotation axis, F1 is the first objective function, F2 is the second objective function, R A 、R B are the rotation matrices of the hand-eye calibration equation, t A 、t B are the translation matrices of the hand-eye calibration equation.

[0164] In some embodiments, based on the objective function, the first analysis sub-module determines the Jacobian matrix, the target optimization value, and the objective function value, specifically including:

[0165] Determine the partial derivatives of the rotation axis corresponding to the initial rotation matrix and the initial translation matrix in the objective function;

[0166] Through the partial derivatives, determine the Jacobian matrix, the target optimization value, and the objective function value, where the Jacobian matrix, the target optimization value, and the objective function value are obtained by the following formula:

[0167]

[0168]

[0169]

[0170] In the formula, J is the Jacobian matrix, Δx is the target optimization value, F is the objective function value, F1 col1 is the first column of the first objective function matrix.

[0171] And, determine the equality relationship through the Jacobian matrix, the target optimization value, and the objective function value, including:

[0172] The equality relationship is obtained by the following formula:

[0173] JΔx = F

[0174] where J = [J1, J2,..., J n T | 12n×6 , F = [F1, F2,..., F n ​​T | 12n×1 ;

[0175] Determine the target optimization value from the equation relationship by least squares, including:

[0176]

[0177]

[0178] In the formula, is the second alternative rotation matrix in the second alternative rigid body transformation relationship, is the second alternative translation matrix in the second alternative rigid body transformation relationship, is the first alternative rotation matrix in the first alternative rigid body transformation relationship, The first alternative translation matrix in the first alternative rigid body transformation relationship.

[0179] An embodiment of the present application provides a robot hand-eye calibration device, including an acquisition module, an analysis module, and a determination module. The acquisition module determines the hand-eye calibration equation with the camera fixedly arranged on the end effector of the robot or at a position outside the robot, and can determine the initial rigid body transformation relationship; the analysis module can determine the objective function through the first alternative rigid body transformation relationship or the initial rigid body transformation relationship by Gauss-Newton iteration; based on the objective function, determine the second alternative rigid body transformation relationship. If the currently updated second alternative rigid body transformation relationship is different from the previously updated first alternative rigid body transformation relationship, determine the error distance corresponding to the second alternative rigid body transformation relationship, and add the second alternative rigid body transformation relationship and the error distance to the alternative data; if the currently updated second alternative rigid body transformation relationship is the same as the previously updated first alternative rigid body transformation relationship, screen out the target rigid body transformation relationship from the alternative data, and use the non-linear optimization algorithm based on Gauss-Newton iteration to improve the accuracy of solving the hand-eye calibration equation.

[0180] The computer device provided by the embodiment of the present application further includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program, and this computer program is used to implement the above-mentioned robot hand-eye calibration method. The implementation principle and technical effect are similar to those of the above method embodiment, and will not be elaborated here.

[0181] The embodiment of the present application further provides a computer storage medium. A computer program is stored on the computer-readable storage medium, and the computer program is executed by the processor to implement the above-mentioned robot hand-eye calibration method. The implementation principle and technical effect are similar to those of the above method embodiment, and will not be elaborated here.

[0182] The following paragraphs will list and compare the Chinese terms and their corresponding English terms involved in the specification of the present application for easy reading and understanding.

[0183] For ease of explanation, the above description has been presented in connection with specific embodiments. However, the discussion in some of the above embodiments is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Many modifications and variations can be derived from the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and the practical applications, so that those skilled in the art can better use the embodiments and various different modified embodiments suitable for specific use considerations.

Claims

1. A robot hand-eye calibration method, characterized in that, Including: Determine the initial rigid body transformation relationship of the robot hand-eye calibration equation, where the hand-eye calibration equation is determined with the camera fixedly arranged on the end effector of the robot or at a position outside the robot; If the currently updated second alternative rigid body transformation relationship is different from the previously updated first alternative rigid body transformation relationship, determine the error distance corresponding to the second alternative rigid body transformation relationship, and add the second alternative rigid body transformation relationship and the error distance to the alternative data; If the currently updated second alternative rigid body transformation relationship is the same as the previously updated first alternative rigid body transformation relationship, screen out the target rigid body transformation relationship from the alternative data; Wherein, both the first alternative rigid body transformation relationship and the second alternative rigid body transformation relationship are determined by an objective function, and the objective function is determined by Gaussian-Newton iteration of the first alternative rigid body transformation relationship or the initial rigid body transformation relationship; Determine the second alternative rigid body transformation relationship through the objective function, including: Based on the objective function, determine the Jacobian matrix, the target optimization value, and the objective function value; Determine the target optimization value from the equation relationship by least squares, where the equation relationship is determined by the Jacobian matrix, the target optimization value, and the objective function value; Based on the target optimization value, determine the second alternative rigid body transformation relationship.

2. The robot hand-eye calibration method according to claim 1, wherein After adding the second alternative rigid body transformation relationship and the error distance to the alternative data, it further includes: If the currently updated iteration number is greater than a preset iteration threshold, screen out the target rigid body transformation relationship from the alternative data.

3. The robot hand-eye calibration method according to claim 1, characterized in that, Determine the error distance corresponding to the second alternative rigid body transformation relationship, including: Obtain the rotation error and the translation error; Determine the error distance through the sum value of the rotation error and the translation error.

4. The robot hand-eye calibration method according to claim 1, wherein When the objective function is determined by Gaussian-Newton iteration of the initial rigid body transformation relationship, it includes: Establish the objective function, where the objective function includes a first objective function and a second objective function, and the first objective function and the second objective function are obtained through the following formula: F1(ω X ) = R A exp([ω X ^) - exp([ω X ^)R B F2(ω X , t X ) = exp([ω X ^)t B +(I - R A )t X -t A Among them, In the formula, R X is the initial rotation matrix in the initial rigid body transformation relationship, t X is the initial translation matrix in the initial rigid body transformation relationship, ω X is the rotation axis corresponding to the initial rotation matrix in the initial rigid body transformation relationship, [ω X ^ is the cross product matrix corresponding to the rotation axis, F1 is the first objective function, F2 is the second objective function, R A and R B are the rotation matrices of the hand-eye calibration equation, t A and t B are the translation matrices of the hand-eye calibration equation.

5. The robot hand-eye calibration method according to claim 4, wherein Based on the objective function, determine the Jacobian matrix, the target optimization value, and the objective function value, including: Determine the partial derivatives of the rotation axis corresponding to the initial rotation matrix and the initial translation matrix in the objective function; Through the partial derivatives, determine the Jacobian matrix, the target optimization value, and the objective function value, where the Jacobian matrix, the target optimization value, and the objective function value are obtained through the following formula: where J is the Jacobian matrix, Δx is the target optimization value, F is the target function value, and F1 col1 is the first column of the first objective function matrix.

6. The robot hand-eye calibration method according to claim 5, wherein Determine the equation relationship through the Jacobian matrix, the target optimization value, and the objective function value, including: The equation relationship is obtained through the following formula: JΔx = F wherein, J = [J1, J2,..., J n T | 12n×6 , F = [F1, F2,..., F n T | 12n×1 ;​​ Determine the target optimization value from the equation relationship by least squares, including: wherein, is the second alternative rotation matrix in the second alternative rigid body transformation relationship, is the second alternative translation matrix in the second alternative rigid body transformation relationship, is the first alternative rotation matrix in the first alternative rigid body transformation relationship, is the first alternative translation matrix in the first alternative rigid body transformation relationship.

7. A robot hand-eye calibration device, characterized in that, Including: An acquisition module for determining the initial rigid body transformation relationship of the robot hand-eye calibration equation, where the hand-eye calibration equation is determined with the camera fixedly arranged on the end effector of the robot or at a position outside the robot; An analysis module, configured to determine a first alternative rigid body transformation relationship or a second alternative rigid body transformation relationship based on an objective function, where the objective function is determined by Gaussian-Newton iteration of the first alternative rigid body transformation relationship or an initial rigid body transformation relationship; determining the second alternative rigid body transformation relationship through the objective function includes: determining a Jacobian matrix, an objective optimization value, and an objective function value based on the objective function; determining the objective optimization value from an equation relationship through least squares, where the equation relationship is determined by the Jacobian matrix, the objective optimization value, and the objective function value; determining the second alternative rigid body transformation relationship based on the objective optimization value; A determination module, configured to, if the currently updated second alternative rigid body transformation relationship is different from the previously updated first alternative rigid body transformation relationship, determine an error distance corresponding to the second alternative rigid body transformation relationship, and add the second alternative rigid body transformation relationship and the error distance to alternative data; The determination module is further configured to, if the currently updated second alternative rigid body transformation relationship is the same as the previously updated first alternative rigid body transformation relationship, screen out a target rigid body transformation relationship from the alternative data.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the robot hand-eye calibration method according to any one of claims 1 to 6 are implemented.

9. A computer storage medium, characterized in that, A computer program is stored on the computer storage medium, and when the computer program is executed by a processor, the processor is caused to execute the steps of the robot hand-eye calibration method according to any one of claims 1 to 6.

Images