An improved hand-eye calibration method based on structured light camera

Abstract

The present invention belongs to the technical field of hand-eye calibration, and specifically relates to an improved hand-eye calibration method based on a structured light camera, comprising the following steps: Step 1: Establish a pinhole camera model, then use a depth camera to detect three-dimensional coordinates, and obtain the physical coordinates of each point in the image coordinate system of known depth relative to the camera coordinate system; Step 2: Establish a robot arm DH model, and then use inverse kinematics to move to determine the coordinates; Step 3: Collect n groups of point cloud data, substitute the depth scaling coefficient for SVD decomposition, and use Nelder‑Mead to solve the optimal coefficient; Step 4: Complete the hand-eye calibration of the robot arm according to the solved optimal coefficient. The present invention has the advantages of strong operability and strong robustness for hand-eye calibration based on a structured light camera.

Description

Technical Field

[0001] The present invention belongs to the technical field of hand-eye calibration, and in particular relates to an improved hand-eye calibration method based on a structured light camera. Background Art

[0002] As the automation level of industrial robots continues to improve, machine vision has become an important part of automated processing; when using vision to locate, grasp, and move target objects, the first step is to obtain the three-dimensional spatial information of the target object through a camera, so hand-eye calibration is required. The traditional nine-point industrial hand-eye calibration is limited to a two-dimensional plane and can only obtain the two-dimensional plane information of the target object; it lacks z-axis information in space; it can only grasp on a plane at a fixed height; if the robot grasps in space, a three-dimensional space hand-eye calibration is required, that is, to obtain the conversion relationship between the robot coordinate system and the camera coordinate system; the so-called hand-eye calibration is to unify the coordinate system of the visual system and the robot, so that the object posture determined by the visual system can be converted to the robot coordinate system, and the robot arm can complete the operation of the object. The most common hand-eye systems include Eye-to-Hand and Eye-in-Hand. In the Eye-to-Hand system, the position of the camera and the robot base is relatively fixed, and the hand-eye relationship solves the conversion relationship between the camera coordinate system and the robot base coordinate system. In the Eye-in-Hand system, the camera is fixed at the end of the robot arm, and the hand-eye relationship solves the conversion relationship between the camera coordinate system and the coordinate system at the end of the robot arm. When the robot is in different positions and postures, the relationship between the "eye" and the calibration object is obtained. Then, combined with the robot's posture (which can be read from the robot controller), a calibration equation can be established, and the hand-eye relationship matrix can be obtained by solving the equation; when solving the calibration equation, due to the nonlinearity and instability of the equation solution, how to obtain a solution with small error and meaningfulness is an important issue of hand-eye calibration; in the prior art, for example, patent No.: CN202211097455.0 discloses a hand-eye calibration method and system for a hair transplant robot; when the coordinates for hand-eye calibration are obtained by a structured light camera, no processing is performed, and the source of the calibration data is unstable and there are random errors; at the same time, the optimization of the calibration structure does not consider the influence of sensor errors, but only takes the best of multiple calibration results.

[0003] The main disadvantage of the existing hand-eye calibration scheme is that the optimization step of the calibration result is only to find the best from multiple groups of results, and the optimization factors are limited, making it difficult to obtain the optimal solution; the hand-eye calibration scheme based on structured light camera through decomposition is relatively simple and easy to use in operation and software implementation, but the actual hardware is bound to have errors in measuring the depth information required by this method, and the error of the structured light camera is difficult to model, so the calibration accuracy is not stable enough in practical applications;

[0004] To address the problem that the error of structured light camera is difficult to estimate in the hand-eye calibration scheme based on structured light camera decomposition, this paper proposes a depth estimation method based on Kalman filtering, and compensates for the error of structured light camera through algorithm and calibration results, ultimately improving the accuracy of hand-eye calibration. Summary of the invention

[0005] The purpose of the present invention is to provide an improved hand-eye calibration method based on a structured light camera. The error caused by the method is analyzed by substituting the method into an SVD decomposition process, and the optimal solution is solved by the Nelder-Mead optimization algorithm. For the hand-eye calibration based on a structured light camera, the present invention has the advantages of strong operability and strong robustness.

[0006] The technical solution adopted by the present invention is as follows:

[0007] An improved hand-eye calibration method based on a structured light camera comprises the following steps:

[0008] Step 1: Establish a pinhole camera model, and then use the depth camera to detect the three-dimensional coordinates to obtain the physical coordinates of each point in the image coordinate system with known depth relative to the camera coordinate system;

[0009] Step 2: Build the DH model of the robot arm, and then use inverse kinematics to move to the determined coordinates;

[0010] Step 3: Collect n groups of point cloud data, substitute the depth scaling coefficient for SVD decomposition, and use Nelder-Mead to solve the optimal coefficient; in step 3, the robot arm is driven by a given set of known joint angles, and the end position of the robot arm is obtained using the robot arm kinematic model Manually measure the axial difference between a specific chessboard corner point and the end of the robotic arm Combined Repeat the above steps to obtain a set of point clouds relative to the robot base coordinate system; the depth camera obtains the pixel coordinates of the chessboard corner points at the aforementioned specific positions relative to the depth camera image coordinate system and the depth value of the chessboard corner points relative to the camera coordinate system through corner point detection. Substituting the two into the pinhole camera model, the three-dimensional coordinates of the chessboard corner points relative to the camera can be obtained. Repeat the above steps to obtain a set of point clouds relative to the depth camera coordinate system. The depth of all camera-collected point clouds is processed by a first-order Kalman filter. After the depth value measured by the camera converges after filtering, it is calculated through the camera intrinsic parameter matrix to obtain the complete point cloud data.

[0011] In the step 3, during the SVD decomposition process, the two groups of point clouds are first decentralized; the covariance matrix is ​​constructed using the normalized two groups of point clouds; the covariance matrix is ​​decomposed to obtain matrices U, S, and V respectively; the rotation matrix R and the translation vector t are calculated according to the decomposition results to obtain the calibration result;

[0012] Analyze and compensate for the axial scaling of the camera depth; multiply the depth value obtained by the depth camera by a scaling factor, and the chessboard corners change as a whole relative to the camera's point cloud data; the point cloud collected by the depth camera is multiplied by a scaling factor compared to its mean. According to the properties of SVD decomposition, U and V remain unchanged, and S is scaled by the same multiple of the decomposed matrix:

[0013] [US*z_scale V] = SVD(H ， )

[0014]

[0015] In step 3, the depth scaling factor z_scale is optimized according to the calibration result, and the error function is defined with the scaling factor as the independent variable:

[0016]

[0017] The coordinates of the calibration plate relative to the robot base coordinate system are taken as the true value, and the calibration error is measured by subtracting the coordinates measured by the camera and the calculated result of the calibration result. The function argument is the depth scaling factor; combine the formulas:

[0018]

[0019] In order to obtain the relationship between the error function and the depth scaling factor, the error size is calculated and the optimization target is minimized. The Nelder-Mead mathematical optimization algorithm is used to find the minimum value and solve the optimal depth scaling factor.

[0020] Step 4: Complete the hand-eye calibration of the robot arm based on the solved optimal coefficients.

[0021] Preferably, in step 1: taking the camera origin O im The camera three-dimensional coordinate system is established as the coordinate origin to represent the three-dimensional coordinates of a point in space in physical length units; the image origin O pix The image coordinate system is established as the coordinate origin to represent the pixel coordinates of a point in space projected on the imaging plane, in units of pixels; the imaging origin O xy An imaging coordinate system is established for the coordinate origin to represent the two-dimensional coordinates of a point in space projected on the imaging plane, in units of physical length.

[0022] Preferably, the three-dimensional coordinates of the space point and the imaging coordinate system are both in units of physical length, and the following equation can be established based on the principle of similar triangles:

[0023]

[0024] In the formula, x and y are the coordinates of the space point P in the imaging coordinate system, and X, Y, and Z are the coordinates of the space point P in the camera coordinate system. In the image coordinate system and the imaging coordinate system, the projection of point P can have the following conversion formula:

[0025]

[0026] Where: u and v are the coordinates of the projection of point P in the image coordinate system, α and β represent the number of pixels corresponding to the unit physical length in the x and y directions, and c x 、c y O xy Coordinates in the image coordinate system;

[0027] Combining the above two equations, we can get: Let αf = f x , βf=f y , the formula can be written as a matrix product:

[0028]

[0029] Where K is the intrinsic parameter matrix of the camera. Using the above formula, we can obtain the physical coordinates of each point in the image coordinate system of known depth relative to the camera coordinate system, given the known depth camera intrinsic parameter matrix.

[0030] Preferably, in step 2, a model based on Dnavit-Hartenberg is used to represent the transformation relationship between the connecting rods; the motion relationship between adjacent connecting rods can be represented as the rotation of connecting rod i relative to connecting rod i-1 around the Z axis in the coordinate system n; there are 4 parameters in the DH model, namely d, a, α, θ; the specific definition is:

[0031] d i : Link length, the distance between adjacent joints m and joint m-1 on the Z axis in coordinate system n;

[0032] a i-1 : Link offset, the distance between the Z axis of coordinate system n and the Z axis of coordinate system n-1 on the X axis in n-1;

[0033] α i-1 : connecting rod torsion angle, the angle between the Z axis in coordinate system n and n-1 and the X axis in n-1;

[0034] θ i: Joint angle, the angle between the X-axis in coordinate system n and n-1 and the Z-axis in n.

[0035] Preferably, the transformation between adjacent links can be considered to follow the following steps: First, the coordinate system n-1 is rotated around the X axis by α i-1 ; Then translate along the X axis by a i-1 ; Then rotate θ around the Z axis i ; Finally, translate along the Z axis by d i ; Finally, the coordinate system n is completely overlapped; the homogeneous transformation matrix corresponding to the above target posture is equal to the transformation matrix of the end of the robot arm relative to the robot arm base coordinate system. A set of equations is constructed, and solving the set of equations can obtain the joint angles required to reach the target posture.

[0036] The technical effects achieved by the present invention are:

[0037] In the present invention, Kalman filtering is used to calibrate the stability of data sampling: in the hand-eye calibration process based on structured light camera, the singular value decomposition (SVD) method is used for preliminary calibration; in order to reduce the noise and fluctuation in the calibration data sampling process, Kalman filtering technology is applied to smooth the data. This technology can dynamically update and correct the calibration data, significantly improve the stability and accuracy of the data, and avoid the influence of unstable factors in the sampling process on the calibration results.

[0038] In the present invention, the depth scaling factor variable is introduced: after the calibration is completed, the systematic error of the depth camera sensor is processed by introducing the depth scaling factor variable. The sensor error of the depth camera is often difficult to accurately describe by traditional modeling methods. The present invention further corrects the calibration result by adjusting the depth scaling factor to compensate for the error caused by the sensor characteristics. This variable can be adjusted dynamically to improve the accuracy of the calibration result.

[0039] In the present invention, the Nelder-Mead optimization algorithm solves the optimal depth scaling factor: In order to determine the optimal depth scaling factor, the present invention uses the Nelder-Mead optimization algorithm to optimize the error function; the algorithm is a direct search method that does not require the calculation of derivatives, and solves the minimum value of the error function through iterative convergence to find the optimal depth scaling factor. This process can effectively reduce the impact of the depth camera sensor error on the calibration results, and improve the overall accuracy and reliability of the calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the technical solution in the present invention;

[0041] Figure 2 This is the pinhole camera model diagram in this release;

[0042] Figure 3 It is a diagram of the hand-eye calibration coordinate system and its transformation relationship in the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose and advantages of the present invention more clearly understood, the present invention is specifically described below in conjunction with embodiments. It should be understood that the following text is only used to describe one or several specific embodiments of the present invention, and does not strictly limit the scope of protection of the specific claims of the present invention.

[0044] like Figure 1 As shown, an improved hand-eye calibration method based on a structured light camera includes the following steps:

[0045] Step 1: Establish a pinhole camera model, and then use the depth camera to detect the three-dimensional coordinates to obtain the physical coordinates of each point in the image coordinate system with known depth relative to the camera coordinate system;

[0046] Step 2: Build the DH model of the robot arm, and then use inverse kinematics to move to the determined coordinates;

[0047] Step 3: Collect n groups of point cloud data, substitute the depth scaling coefficient for SVD decomposition, and use Nelder-Mead to solve the optimal coefficient;

[0048] Step 4: Complete the hand-eye calibration of the robot arm based on the solved optimal coefficients.

[0049] like Figure 2 As shown, in step 1: take the camera origin O im The camera three-dimensional coordinate system is established as the coordinate origin to represent the three-dimensional coordinates of a point in space in physical length units; the image origin O pix The image coordinate system is established as the coordinate origin to represent the pixel coordinates of a point in space projected on the imaging plane, in units of pixels; the imaging origin O xy An imaging coordinate system is established for the coordinate origin to represent the two-dimensional coordinates of a point in space projected on the imaging plane, in units of physical length.

[0050] The three-dimensional coordinates of the space point and the imaging coordinate system are both based on physical length. The following equation can be established based on the principle of similar triangles:

[0051]

[0052] In the formula, x and y are the coordinates of the space point P in the imaging coordinate system, and X, Y, and Z are the coordinates of the space point P in the camera coordinate system. In the image coordinate system and the imaging coordinate system, the projection of point P can have the following conversion formula:

[0053]

[0054] Where: u and v are the coordinates of the projection of point P in the image coordinate system, α and β represent the number of pixels corresponding to the unit physical length in the x and y directions, and c x 、c y O xy Coordinates in the image coordinate system;

[0055] Combining the above two equations, we can get: Let αf = f x , βf=f y , the formula can be written as a matrix product:

[0056]

[0057] Where K is the intrinsic parameter matrix of the camera. Using the above formula, we can obtain the physical coordinates of each point in the image coordinate system of known depth relative to the camera coordinate system, given the known depth camera intrinsic parameter matrix.

[0058] In step 2, the transformation relationship between the links is represented by a model based on Dnavit-Hartenberg. The modeling follows the following rules: the origin is the intersection of the common normal of the current joint and the next joint and the axis of the current joint. If it is the last joint, it is the intersection of the common normal of the previous joint and the current joint and the axis of the current joint; the Z axis starts from the origin and coincides with the axis of the joint; the X axis is the common normal of the current joint axis pointing to the next joint axis, which is perpendicular to the axes of the two joints. When the two Z axes intersect, the direction of the X axis is rotated so that the angle between the current Z axis and the next Z axis is positive and less than 180°; the motion relationship between adjacent links can be represented as the rotation of link i around the Z axis in the coordinate system n relative to link i-1; there are 4 parameters in the DH model, namely d, a, α, and θ. The specific definitions are:

[0059] d i : Link length, the distance between adjacent joints m and joint m-1 on the Z axis in coordinate system n;

[0060] a i-1 : Link offset, the distance between the Z axis of coordinate system n and the Z axis of coordinate system n-1 on the X axis in n-1;

[0061] α i-1 : connecting rod torsion angle, the angle between the Z axis in coordinate system n and n-1 and the X axis in n-1;

[0062] θ i : Joint angle, the angle between the X-axis in coordinate system n and n-1 and the Z-axis in n;

[0063] The transformation of adjacent links between coordinate systems can be considered to follow the following steps: First, the coordinate system n-1 is rotated around the X axis by α i-1 ; Then translate along the X axis by a i-1; Then rotate θ around the Z axis i ; Finally, translate along the Z axis by d i ; Finally, the coordinate system n is completely overlapped;

[0064] In the present invention, the homogeneous transformation matrix from the m-1th joint to the mth joint can be expressed as follows according to the DH parameters:

[0065]

[0066] Assuming that the robotic arm has N links, the transformation relationship from the base coordinate system to the end coordinate system is as follows:

[0067]

[0068] The target position of the robot arm is represented by a homogeneous transformation matrix:

[0069]

[0070] Where n x 、n y 、n z is the component of the target pose n-axis relative to the x, y, and z axes of the base coordinate system, o x , o y , o z is the component of the target pose o-axis relative to the x, y, and z axes of the base coordinate system, a x 、a y 、a z It is the component of the target pose a-axis relative to the x, y, and z axes of the base coordinate system.

[0071] Let the homogeneous transformation matrix corresponding to the above target posture be equal to the transformation matrix of the end of the robot arm relative to the robot arm base coordinate system, construct a set of equations, and solve the set of equations to obtain the joint angles required to reach the target posture.

[0072] like Figure 3 As shown, in step 3, the robot arm is driven by a given set of known joint angles, and the end position of the robot arm is obtained using the robot arm kinematic model Manually measure the axial difference between a specific chessboard corner point and the end of the robotic arm Combined Repeat the above steps to obtain a set of point clouds relative to the robot base coordinate system; the depth camera obtains the pixel coordinates of the chessboard corner points at the aforementioned specific positions relative to the depth camera image coordinate system and the depth value of the chessboard corner points relative to the camera coordinate system through corner point detection. Substituting the two into the pinhole camera model, the three-dimensional coordinates of the chessboard corner points relative to the camera can be obtained. Repeat the above steps to obtain a set of point clouds relative to the depth camera coordinate system.

[0073] In the present invention, the depth of all camera-collected point clouds is processed by a first-order Kalman filter. After the depth value measured by the camera converges after filtering, the complete point cloud data is calculated through the camera's intrinsic parameter matrix, which is considered a successful point cloud collection.

[0074] In step 3 of the present invention, during the SVD decomposition process;

[0075] First, the two sets of point clouds are decentralized:

[0076]

[0077]

[0078]

[0079]

[0080] In the above formula, x i ,y i They are the coordinates of the chessboard corner points observed by the camera and the coordinates of the chessboard corner points relative to the base coordinate system of the robotic arm calculated by the forward kinematics of the robotic arm plus axial compensation; is x i The coordinate average of Yes i The coordinate average, X i is the decentralized point cloud relative to the depth camera coordinate system, Y i It is the decentralized point cloud relative to the robot base coordinate system.

[0081] Use the normalized two sets of point clouds to construct the covariance matrix: H = X T *Y; perform SVD decomposition on the covariance matrix to obtain matrices U, S, and V respectively: [USV] = SVD (H); calculate the rotation matrix and translation vector between according to the decomposition results:

[0082]

[0083] R is the rotation matrix, t is the translation vector, and the calibration result is obtained.

[0084] Considering that the depth camera has errors in measuring the depth at a long distance, but the sensor error is difficult to model and analyze, the present invention analyzes and compensates for the axial scaling of the camera depth; multiplying the depth value obtained by the depth camera by a scaling factor, the chessboard corners change as a whole relative to the point cloud data of the camera:

[0085]

[0086] z_scale is the scaling factor.

[0087] After considering the scaling factor, the formula becomes the following: the point cloud collected by the depth camera is multiplied by a scaling factor compared to its mean;

[0088]

[0089]

[0090]

[0091] H, = (X*z_scale) T *Y

[0092] According to the properties of SVD decomposition, U and V remain unchanged, and S is scaled by the same multiple as the decomposed matrix:

[0093] [US*z_scale V] = SVD(H ， )

[0094]

[0095] According to the calibration results, the depth scaling factor z_scale is optimized. The depth scaling factor of the depth camera is taken into account when solving the calibration matrix in the previous step. Here, the error function is defined with this scaling factor as the independent variable:

[0096]

[0097] The coordinates of the calibration plate relative to the robot base coordinate system are taken as the true values, and the coordinates measured by the camera and the calculated results of the calibration results are subtracted to measure the calibration error. The function independent variable is the depth scaling factor. The present invention combines the formulas here to more intuitively see the relationship between the error function and the depth scaling factor:

[0098]

[0099] The error function here has a calculated size of 3×n, where each element is a function f about z_scale i (z_scale), such a function matrix is ​​not convenient for quantitatively comparing the error size. The present invention uses the mean square error to evaluate the error size, which is calculated as follows:

[0100] rmse(z_scale)=sum(e(z_scale)*e(z_scale)) / n

[0101] The error function is multiplied element by element by itself, and the result is still 3×n. Sum means summing all elements of the 3×n matrix to get a scalar result, and finally dividing by n to get the mean square error.

[0102] Then the size of the error can be expressed as follows:

[0103]

[0104] Obviously, the optimization goal is to minimize rmse. The result of such a function combination calculation is obviously not suitable for obtaining the maximum value using the derivation method. At the same time, since the measurement error of the depth camera is related to the measurement distance, and the sampling points of the present invention are obtained at different positions of the calibration chessboard, the error function will have multiple different minimum values ​​at many sampling points of different depths. Therefore, this is still a non-convex problem. Here, a mathematical optimization algorithm such as Nelder-Mead is used to find the minimum value.

[0105] In the present invention, error optimization is performed: Here is the process using the Nelder-Mead algorithm:

[0106] Let z_scale be x i , then rmse(z_scale) is f(x i );

[0107] First, generate N initial points x2...x based on the initial point N+1 , so that x 1+i If the ith component of x1 is 5% greater than x1, the other components remain the same; if the ith component of x1 is zero, then x 1+i The i-th component of is set to 0.00025; after obtaining N+1 initial points, start looping according to the following steps until the specific accuracy condition is met and exit the loop.

[0108] First: give x i Click f(x i ) Rearrange the order from small to large so that the larger i is, the larger f(x i ) is larger;

[0109] Then: calculate the average position of the first N points as

[0110] Then: calculate x N+1 The symmetric point about point m is r = 2m-x N+1 ;

[0111] Then: If f(x1)≤f(r)<f(x N ), let x N+1 =r, and enter the next cycle;

[0112] Then: If f(r)<f(x1), calculate the expansion point as s=m+2(mx N+1 );

[0113] If f(s)<f(r), let x N+1 =s; and enter the next loop, otherwise let x N+1 =r; and enter the next cycle.

[0114] Then: If f(x N )≤f(r)<f(x N+1 ), let c1=m+(rm) / 2;

[0115] If f(c1)<f(r), let x N+1 = c1 and enter the next loop, otherwise execute the last step.

[0116] Then: If f(x N+1 )≤f(r)Let c2=m+(x N+1 -m) / 2; if f(c2)<f(x N+1 ), let x N+1 = c2 and enter the next loop, otherwise execute the last step.

[0117] The last step is to set v i =x1+(x i -x1) / 2; (i=2...N+1); and use v i Assign to x i , enter the next loop; after the loop ends, solve the optimal depth scaling factor.

[0118] In the present invention, Kalman filtering is used to calibrate the stability of data sampling: in the hand-eye calibration process based on structured light camera, the singular value decomposition (SVD) method is used for preliminary calibration; in order to reduce the noise and fluctuation in the calibration data sampling process, Kalman filtering technology is applied to smooth the data. This technology can dynamically update and correct the calibration data, significantly improve the stability and accuracy of the data, and avoid the influence of unstable factors in the sampling process on the calibration results.

[0119] In the present invention, the depth scaling factor variable is introduced: after the calibration is completed, the systematic error of the depth camera sensor is processed by introducing the depth scaling factor variable. The sensor error of the depth camera is often difficult to accurately describe by traditional modeling methods. The present invention further corrects the calibration result by adjusting the depth scaling factor to compensate for the error caused by the sensor characteristics. This variable can be adjusted dynamically to improve the accuracy of the calibration result.

[0120] In the present invention, the Nelder-Mead optimization algorithm solves the optimal depth scaling factor: In order to determine the optimal depth scaling factor, the present invention uses the Nelder-Mead optimization algorithm to optimize the error function; the algorithm is a direct search method that does not require the calculation of derivatives, and solves the minimum value of the error function through iterative convergence to find the optimal depth scaling factor. This process can effectively reduce the impact of the depth camera sensor error on the calibration results, and improve the overall accuracy and reliability of the calibration.

[0121] In the present invention, compared with the prior art, on the one hand, considering that the depth perception information of this type of active stereo vision technology sensor has random errors, a first-order Kalman filter is used to quickly stabilize and converge the information and approach the true value; on the other hand, in view of the fact that structured light cameras have many error sources and it is difficult to establish a model, the error model is simplified into an expression with a scaling coefficient for analysis, and the error caused by it is analyzed by substituting it into the SVD decomposition process, and the optimal solution is solved by the Nelder-Mead optimization algorithm.

[0122] The above are only preferred embodiments of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention shall be implemented according to conventional means in the art unless otherwise specified and limited.

Claims

1. An improved hand-eye calibration method based on structured light camera, Features: The following steps are involved: Step 1: Establish a pinhole camera model, and then use the depth camera to detect the three-dimensional coordinates to obtain the physical coordinates of each point in the image coordinate system with known depth relative to the camera coordinate system; Step 2: Build the DH model of the robot arm, and then use inverse kinematics to move to the determined coordinates; Step 3: Collect n groups of point cloud data, substitute the depth scaling coefficient for SVD decomposition, and use Nelder-Mead to solve the optimal coefficient; in step 3, the robot arm is driven by a given set of known joint angles, and the end position of the robot arm is obtained using the robot arm kinematic model Manually measure the axial difference between a specific chessboard corner point and the end of the robotic arm Combined Repeat the above steps to obtain a set of point clouds relative to the robot base coordinate system; the depth camera obtains the pixel coordinates of the chessboard corner points at the aforementioned specific positions relative to the depth camera image coordinate system and the depth value of the chessboard corner points relative to the camera coordinate system through corner point detection. Substituting the two into the pinhole camera model, the three-dimensional coordinates of the chessboard corner points relative to the camera can be obtained. Repeat the above steps to obtain a set of point clouds relative to the depth camera coordinate system. The depth of all camera-collected point clouds is processed by a first-order Kalman filter. After the depth value measured by the camera converges after filtering, it is calculated through the camera intrinsic parameter matrix to obtain the complete point cloud data. In the step 3, during the SVD decomposition process, the two groups of point clouds are first decentralized; the covariance matrix is ​​constructed using the normalized two groups of point clouds; the covariance matrix is ​​decomposed to obtain matrices U, S, and V respectively; According to the decomposition results, the rotation matrix R and the translation vector t are calculated to obtain the calibration results; Analyze and compensate for the axial scaling of the camera depth; multiply the depth value obtained by the depth camera by a scaling factor, and the chessboard corners change as a whole relative to the camera's point cloud data; the point cloud collected by the depth camera is multiplied by a scaling factor compared to its mean. According to the properties of SVD decomposition, U and V remain unchanged, and S is scaled by the same multiple of the decomposed matrix: [US*z_scale V] = SVD(H') In step 3, the depth scaling factor z_scale is optimized according to the calibration result, and the error function is defined with the scaling factor as the independent variable: The coordinates of the calibration plate relative to the robot base coordinate system are taken as the true value, and the calibration error is measured by subtracting the coordinates measured by the camera and the calculated result of the calibration result. The function argument is the depth scaling factor; combine the formulas: In order to obtain the relationship between the error function and the depth scaling factor, the error size is calculated and the optimization target is minimized. The Nelder-Mead mathematical optimization algorithm is used to find the minimum value and solve the optimal depth scaling factor. Step 4: Complete the hand-eye calibration of the robot arm based on the solved optimal coefficients.

2. According to the improved hand-eye calibration method based on structured light camera in claim 1, Features: In step 1: take the camera origin O im The camera three-dimensional coordinate system is established as the coordinate origin to represent the three-dimensional coordinates of a point in space in physical length units; the image origin O pix The image coordinate system is established as the coordinate origin to represent the pixel coordinates of a point in space projected on the imaging plane, in units of pixels; the imaging origin O xy An imaging coordinate system is established for the coordinate origin to represent the two-dimensional coordinates of a point in space projected on the imaging plane, in units of physical length.

3. The improved hand-eye calibration method based on structured light camera according to claim 2, Features: The three-dimensional coordinates of the space point and the imaging coordinate system are both based on physical length. The following equation can be established based on the principle of similar triangles: In the formula, x and y are the coordinates of the space point P in the imaging coordinate system, and X, Y, and Z are the coordinates of the space point P in the camera coordinate system. In the image coordinate system and the imaging coordinate system, the projection of point P can have the following conversion formula: Where: u and v are the coordinates of the projection of point P in the image coordinate system, α and β represent the number of pixels corresponding to the unit physical length in the x and y directions, and c x 、c y O xy Coordinates in the image coordinate system; Combining the above two equations, we can get: Let αf = f x , βf=f y , the formula can be written as a matrix product: Where K is the intrinsic parameter matrix of the camera. Using the above formula, we can obtain the physical coordinates of each point in the image coordinate system of known depth relative to the camera coordinate system, given the known depth camera intrinsic parameter matrix.

4. The improved hand-eye calibration method based on structured light camera according to claim 3, Features: In step 2, the transformation relationship between the links is represented by a model based on Dnavit-Hartenberg; the motion relationship between adjacent links can be represented as the rotation of link i relative to link i-1 around the Z axis in the coordinate system n; there are 4 parameters in the DH model, namely d, a, α, and θ; the specific definitions are: d i : Link length, the distance between adjacent joints m and joint m-1 on the Z axis in coordinate system n; a i-1 : Link offset, the distance between the Z axis of coordinate system n and the Z axis of coordinate system n-1 on the X axis in n-1; α i-1 : connecting rod torsion angle, the angle between the Z axis in coordinate system n and n-1 and the X axis in n-1; θ i : Joint angle, the angle between the X-axis in coordinate system n and n-1 and the Z-axis in n.

5. The improved hand-eye calibration method based on structured light camera according to claim 4, Features: The transformation of adjacent links between coordinate systems can be considered to follow the following steps: First, the coordinate system n-1 is rotated around the X axis by α i-1 ; Then translate along the X axis by a i-1 ; Then rotate θ around the Z axis i ; Finally, translate along the Z axis by d i ; Finally, the coordinate system n is completely overlapped; the homogeneous transformation matrix corresponding to the above target posture is equal to the transformation matrix of the end of the robot arm relative to the robot arm base coordinate system. A set of equations is constructed, and solving the set of equations can obtain the joint angles required to reach the target posture.

Images