Hand-eye calibration method considering robot body error and visual measurement error

Abstract

This invention discloses a hand-eye calibration method that takes into account both robot body errors and visual measurement errors. The method comprises the following steps: constructing a hand-eye matrix using a measuring device; constructing a visual measurement error compensation function using a multivariate linear function; identifying the coefficients of the error compensation function using principal component analysis; and correcting the coordinates of the measurement points using the visual measurement error compensation function. This method improves the conventional hand-eye calibration model, taking into account the combined influence of robot body errors and visual measurement errors during the robot's hand-eye transformation process. It constructs a mapping relationship between the coordinates of the measured point and the robot body errors and visual measurement errors. This improves the accuracy of the hand-eye matrix solution.

Description

Technical Field

[0001] The present invention relates to a hand-eye calibration method in the field of robotics, and in particular to a hand-eye calibration method that takes into account robot body errors and visual measurement errors. Background Art

[0002] At present, in order to improve the multi-station processing accuracy of mobile processing robots and ensure that the robots obtain high-precision visual measurement information, it is necessary to accurately establish the transformation relationship between the visual sensor fixed at the end of the robot and the end of the robot, that is, the "eye on the hand" hand-eye calibration.

[0003] Currently, hand-eye calibration technology has attracted considerable attention from researchers. Classic hand-eye calibration methods can be divided into two categories based on the solution model. The first model is AX = XB, where A is the homogeneous transformation matrix of the robot end-point coordinate system between two consecutive measurements, B is the homogeneous transformation matrix of the camera coordinate system between two consecutive measurements, and X is the hand-eye matrix. This model is simple in principle, but the true pose of the fixed target in the world system is unknown, and the solution process tends to converge to a local optimum, resulting in insufficient solution accuracy. The second model is AX = ZB, where A is the transformation matrix from the camera coordinate system to the fixed target, B is the transformation matrix from the robot end-point to the robot base coordinate system, Z is the pose transformation relationship from the robot base coordinate system to the fixed target, and X is the hand-eye matrix. This type of hand-eye model solves the hand-eye matrix by establishing closed-chain constraints, achieving high solution accuracy. However, due to factors such as residual distortion and external temperature, visual measurement errors are present. Furthermore, the solution process of this type of hand-eye calibration only focuses on establishing the hand-eye equation and how to avoid error propagation, without considering the impact of the robot's body errors and visual measurement errors on the solution accuracy. This is the reason why many methods do not provide high accuracy in solving the hand-eye transformation. Therefore, a hand-eye calibration method that takes into account the robot's body errors and visual measurement errors is urgently needed. Summary of the Invention

[0004] In order to solve the technical problems existing in the known technology, the present invention provides a hand-eye calibration method taking into account the robot body error and visual measurement error.

[0005] The present invention adopts a technical solution to solve the technical problems existing in the known technology: a hand-eye calibration method taking into account the robot body error and visual measurement error, characterized in that it includes the following method steps: using a measuring device to construct a hand-eye matrix; using a multivariate linear function to construct a visual measurement error compensation function; using a principal component analysis method to identify the coefficients of the error compensation function; and using the visual measurement error compensation function to correct the coordinates of the measurement point.

[0006] Furthermore, the method of constructing a hand-eye matrix using a measuring device includes the following steps:

[0007] A vision sensor is fixed to the end of the robot, a standard sphere of known size is fixed in the robot's workspace, and a laser tracker is set up to measure the robot's base coordinate system, the end position of the robot in different measurement postures, and the coordinates of the center of the standard sphere. The robot is driven so that the standard sphere is within the field of view of the vision sensor. The vision sensor collects the point cloud of the standard sphere's surface and calculates the coordinates of the sphere's center.

[0008] Assume: the robot base coordinate system is {B}, the robot end coordinate system is {T}, the visual sensor coordinate system is {V}, the standard ball center coordinate system is {W}, and the laser tracker coordinate system is {L}; based on the hand-eye calibration solution model, the following formula is obtained:

[0009]

[0010] Where: Represents the homogeneous transformation matrix from {V} to {W}; represents the hand-eye matrix, represents the homogeneous transformation matrix from {B} to {W}; Represents the homogeneous transformation matrix from {B} to {T};

[0011] The coordinates of the center of the standard ball under {V} are obtained using a visual sensor. For any robot measurement pose, we have:

[0012]

[0013] Where: represents the homogeneous transformation matrix from {B} to {T} at the i-th measurement pose; represents the homogeneous transformation matrix from {L} to {T} at the i-th measurement pose; represents the homogeneous transformation matrix from {B} to {L}; and All are measured by laser tracker; then:

[0014]

[0015] Where: P V,i P represents the coordinates of the center of the standard ball under {V} obtained by the visual sensor at the i-th measurement pose; L,B represents the coordinates of the center of the standard ball under {B}, P L,B Measured by laser tracker;

[0016] set up D={d1,d2,...,d n}; C={c1,c2,...,c n};i poseIndicates the element number in sets C and D, i pose =1, 2, ..., n; n represents the number of times the vision sensor performs visual measurements;

[0017] right Perform variable substitution and write it in the form of least squares to obtain:

[0018]

[0019] Where: Represents the rotation transformation of the hand-eye matrix; Represents the translation vector of the hand-eye matrix;

[0020] Decentralize sets C and D:

[0021]

[0022]

[0023] i pose =1~n;

[0024] Where: express Decentralized value; express Decentralized value;

[0025] Further we get:

[0026]

[0027] right Perform SVD decomposition, then and Expressed as:

[0028]

[0029] Where U represents The left singular vector obtained after SVD decomposition; V represents Right singular vectors obtained after SVD decomposition.

[0030] Furthermore, the method of constructing a visual measurement error compensation function using a multivariate linear function includes the following steps:

[0031] Assume that the coordinate measurement value of a point P to be measured in space under {V} is p V , the true value of the coordinates of point P under {V} is The coordinate measurement error of point P under {V} is δ V , then:

[0032]

[0033] Let P T,S Represents the coordinates of point P measured by the visual sensor under {T}, P T,S The calculation formula is as follows:

[0034]

[0035] Let P T,L Indicates the coordinate value of point P measured by laser tracker under {T}; T,L As a benchmark, P T,S Relative P T,L The deviation is expressed as:

[0036]

[0037] Where: △P T,S Indicates P T,S Relative P T,L Deviation;

[0038] Hand-eye matrix after vision sensor installation The error is a constant, then △P T,S With the measured value p V , so △P T,S is the measured value p V function;

[0039] Use p V Multivariate linear function to approximate △P T,S , obtain the error compensation function, and use multiple linear regression to estimate the coefficients of the error compensation function.

[0040] Furthermore, using p V The quadratic response surface is used to approximate △P T,S , and use multiple linear regression to estimate the coefficients of the response surface.

[0041] Furthermore, using p V The method for obtaining the error compensation function from the multivariate linear function includes the following steps:

[0042] Assume: l Indicates △P T,S The error compensation function of the lth coordinate; l represents the coordinate axis number corresponding to the error compensation function; l = 1 to 3;

[0043] Use linear function to express ρ l ,set up:

[0044] in:

[0045]

[0046]

[0047] Where: n l Represents ρ l The order of the terms in the linear function polynomial, n l ≥0; represents the error compensation function ρ l The nth linear function polynomial of l Monomial, n l =0, Represents a monomial The coefficient of Q l Indicated by 1, The vector composed of Ω l Indicated by The vector of components;

[0048] Assume: k is the measurement pose number of the robot end, k = 1, 2, ..., K t ;K t represents the total number of measured poses at the end of the robot; ρ l,k Represents ρ l The kth measured pose observation value; Indicates the corresponding ρ l The calculated value of the kth measured pose; let Q l,k Represents Q l The vector corresponding to the kth measured pose;

[0049] set up:

[0050]

[0051]

[0052]

[0053] Then we get the following corresponding p V Error compensation function: y l =X l Ω l +E l .

[0054] Furthermore, the measurement positions are evenly distributed within the field of view of the visual sensor; and the coordinates of the measured points are decentralized and dimensionless.

[0055] Furthermore, the multivariate linear function y is reconstructed with weights according to the following formula: l =X l Ω l +E l:

[0056] y l ′=D l y l ;

[0057] X′ l =D l X l ;

[0058] E l ′=D l E l ;

[0059] in:

[0060]

[0061]

[0062]

[0063] Where: ρ l,j,k Represents ρ l The jth repeated measurement value at the kth measurement pose, j represents the value of ρ l The repeated measurement number at the kth measurement pose, n t Represents ρ l The number of repeated measurements at the kth measurement pose; Represents ρ l,j,k The mean of Represents ρ l,j,k Variance of D l represents the weighted reconstruction matrix; y l ′ represents the reconstructed y l ; X′ l Represents the reconstructed X l ;E l ′ represents the reconstructed E l ;E l ′ is considered to obey a normal distribution with a mean of zero and a fixed variance.

[0064] Furthermore, the principal component analysis method is used to identify the coefficient Ω of the error compensation function. l The method comprises the following steps:

[0065] According to the cumulative contribution rate threshold Determine the minimum number of principal components m; the formula for calculating the cumulative contribution rate of variance is as follows:

[0066]

[0067] Where μ l Indicates the cumulative contribution rate of variance; express After the eigenvalues ​​of the mth i eigenvalues; l express The set of all eigenvalues; m i express The eigenvalue numbers after sorting from largest to smallest, m represents the minimum number of principal components;

[0068] Based on this, construct the matrix V l :

[0069] V l =[v l,1 v l,2 ...v l,m ];

[0070] Where: Represents eigenvalues The corresponding eigenvector; m i =1, 2, ..., m;

[0071] Through linear transformation, let H l =X′ l V l , α l =V l T Ω l ,get:

[0072] y′ l =H l α l +E l ';

[0073] The least squares method is used to obtain α l Unbiased estimate of After that, we calculate

[0074] Where: H l , α l is an intermediate variable; Represents α l The unbiased estimation matrix of V l express The matrix formed.

[0075] Furthermore, after identifying Ω l After that, the coordinates of the point to be measured under {T} are obtained through an online visual measurement; the coordinates of the point to be measured under {T} P T The calculation formula is as follows:

[0076]

[0077] Furthermore, the following operations are repeated: constructing a visual measurement error compensation function using a multivariate linear function; identifying coefficients of the error compensation function using principal component analysis; until the absolute value change rate of the coefficients of the error compensation function is lower than a given threshold.

[0078] The advantages and positive effects of the present invention are: (1) By analyzing the influence of the robot body error on the hand-eye matrix solution, the present invention proposes a hand-eye matrix solution method based on external measurement information, thereby improving the solution accuracy of the hand-eye matrix.

[0079] (2) The present invention improves the conventional hand-eye calibration model, considers the comprehensive influence of the robot's body error and visual measurement error in the robot's hand-eye transformation process, proposes a hand-eye transformation compensation method taking into account the robot's body error and visual measurement error, and constructs a mapping relationship between the coordinates of the measured point and the robot's body error and visual measurement error.

[0080] (3) The present invention uses a parameter identification method based on principal component analysis to identify the coefficients of the error compensation function offline, thereby achieving accurate and real-time compensation of hand-eye transformation errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 It is a structural schematic diagram of the present invention.

[0082] In the figure: 1. Laser tracker 1; 2. Standard ball; 3. Robot end calibration plate; 4. Visual sensor; 5. Robot. DETAILED DESCRIPTION

[0083] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0084] See Figure 1 A hand-eye calibration method taking into account the robot's body error and visual measurement error is characterized by comprising the following method steps: using a measuring device to construct a hand-eye matrix; using a multivariate linear function to construct a visual measurement error compensation function; using principal component analysis to identify the coefficients of the error compensation function; and using the visual measurement error compensation function to correct the coordinates of the measurement point.

[0085] Preferably, the method of constructing a hand-eye matrix using a measurement device may include the following steps:

[0086] A visual sensor 4 is fixed to the end of an industrial robot 5, a robot end calibration plate 3 is set at the end of the robot 5, a standard sphere 2 of known size is fixed in the robot workspace, and a laser tracker 1 is set to measure the robot base coordinate system, the end posture of the robot 5 under different measurement postures, and the center coordinates of the standard sphere 2; the robot 5 is driven so that the standard sphere 2 is within the field of view of the visual sensor 4; the visual sensor 4 collects the surface point cloud of the standard sphere 2 and calculates the center coordinates of the sphere; the measurement posture represents the measurement position and measurement posture of the end of the robot 5 when the visual sensor 4 measures.

[0087] The robot base coordinate system can be set as {B}, the robot end coordinate system as {T}, the visual sensor coordinate system as {V}, the standard ball center coordinate system as {W}, and the laser tracker coordinate system as {L}. The following formula can be obtained based on the hand-eye calibration solution model:

[0088]

[0089] Where: Represents the homogeneous transformation matrix from {V} to {W}; represents the hand-eye matrix, represents the homogeneous transformation matrix from {B} to {W}; Represents the homogeneous transformation matrix from {B} to {T}.

[0090] The visual sensor 4 is used to obtain the coordinates of the center of the standard ball under {V}. For any robot measurement posture, we have:

[0091]

[0092] Where: represents the homogeneous transformation matrix from {B} to {T} at the i-th measurement pose; represents the homogeneous transformation matrix from {L} to {T} at the i-th measurement pose; represents the homogeneous transformation matrix from {B} to {L}; and are all measured by laser tracker 1; then:

[0093]

[0094] Where: P V,i P represents the coordinates of the center of the standard ball under {V} obtained by the visual sensor 4 at the i-th measurement posture; L,B represents the coordinates of the center of the standard ball under {B}, P L,B Measured by laser tracker 1.

[0095] Can be set D={d1,d2,...,dn}; C={c1,c2,...,c n};i pose Indicates the element number in sets C and D, i pose =1, 2, ..., n; n represents the number of times the visual sensor 4 performs visual measurement.

[0096] right We can substitute variables and write it in the form of least squares to get:

[0097]

[0098] Where: Represents the rotation transformation of the hand-eye matrix; Represents the translation vector of the hand-eye matrix.

[0099] Sets C and D can be decentralized:

[0100]

[0101]

[0102] i pose =1~n;

[0103] Where: express Decentralized value; express Decentralized value.

[0104] We can further obtain:

[0105]

[0106] Can Perform SVD decomposition, then and Expressed as:

[0107]

[0108] Where U represents The left singular vector obtained after SVD decomposition; V represents Right singular vectors obtained after SVD decomposition.

[0109] SVD: Singular Value Decomposition, for any matrix A with m rows and n columns, there exists an orthogonal matrix U with m rows and m columns and an orthogonal matrix V with n rows and n columns such that where ∑ r =diag(σ1,...,σ r ), σr are the singular values ​​of the matrix A.

[0110] Preferably, the method of constructing a visual measurement error compensation function using a multivariate linear function may include the following steps:

[0111] It can be assumed that the coordinate measurement value of a point P to be measured in space under {V} is p V , the true value of the coordinates of point P under {V} is The coordinate measurement error of point P under {V} is δ V , then:

[0112]

[0113] P can be set T,S represents the coordinates of point P measured by the visual sensor 4 under {T}, P T,S The calculation formula is as follows:

[0114]

[0115] P can be set T,L Indicates the coordinate value of point P measured by laser tracker 1 under {T}; T,L As a benchmark, P T,S Relative P T,L The deviation is expressed as:

[0116]

[0117] Where: △P T,S Indicates P T,S Relative P T,L Deviation;

[0118] Hand-eye matrix after vision sensor 4 is installed The error is a constant, then △P T,S With the measured value p V , so △P T,S is the measured value p V function.

[0119] Available p V Multivariate linear function to approximate △P T,S , obtain the error compensation function, and use multiple linear regression to estimate the coefficients of the error compensation function.

[0120] Preferably, p V The quadratic response surface is used to approximate △P T,S , and use multiple linear regression to estimate the coefficients of the response surface.

[0121] Preferably, use p VThe method for obtaining an error compensation function from a multivariate linear function may include the following steps:

[0122] We can set: l Indicates △P T,S Error compensation function of the lth coordinate; l=1~3; l=1~3, l=1 can represent the horizontal axis coordinate; l=2 can represent the vertical axis coordinate; l=3 can represent the vertical axis coordinate.

[0123] Use linear function to express ρ l , you can set:

[0124] in:

[0125]

[0126]

[0127] Where: n l Represents ρ l The order of the terms in the linear function polynomial, n l ≥0; represents the error compensation function ρ l The nth linear function polynomial of l Monomial, n l =0, Represents a monomial The coefficient of Q l Indicated by 1, The vector composed of Ω l Indicated by Vector composed of.

[0128] It can be assumed that: k is the measurement pose number of the robot end, k = 1, 2, ..., K t ;K t represents the total number of measured poses at the end of the robot; ρ l,k Represents ρ l The kth measured pose observation value; Indicates the corresponding ρ l The calculated value of the kth measured pose; let Q l,k Represents Q l The vector corresponding to the kth measured pose.

[0129] Can be set:

[0130]

[0131]

[0132]

[0133] Then we get the following corresponding p V Error compensation function: y l =X l Ω l +E l .

[0134] Preferably, the measurement positions can be evenly distributed within the field of view of the visual sensor; and the coordinates of the points to be measured are decentralized and dimensionless.

[0135] Preferably, the multivariate linear function y can be reconstructed by weight according to the following formula: l =X l Ω l +E l :

[0136] y l ′=D l y l ;

[0137] X′ l =D l X l ;

[0138] E l ′=D l E l ;

[0139] in:

[0140]

[0141]

[0142]

[0143] Where: ρ l,j,k Represents ρ l The jth repeated measurement value at the kth measurement pose, j represents the value of ρ l The repeated measurement number at the kth measurement pose, n t Represents ρ l The number of repeated measurements at the kth measurement pose; Represents ρ l,j,k The mean of Represents ρ l,j,k Variance of D l represents the weighted reconstruction matrix; y l ′ represents the reconstructed y l ; X′ l Represents the reconstructed X l ;E l ′ represents the reconstructed E l;E l ′ is considered to obey a normal distribution with a mean of zero and a fixed variance.

[0144] Preferably, the principal component analysis method is used to identify the coefficient Ω of the error compensation function l The method comprises the following steps:

[0145] According to the cumulative contribution rate threshold Determine the minimum number of principal components m; the formula for calculating the cumulative contribution rate of variance is as follows:

[0146]

[0147] Where μ l Indicates the cumulative contribution rate of variance; express After the eigenvalues ​​of the mth i eigenvalues; l express The set of all eigenvalues; m i express The eigenvalue numbers after sorting from large to small, m represents the minimum number of principal components.

[0148] Based on this, construct the matrix V l :

[0149] V l =[v l,1 v l,2 ... v l,m ];

[0150] Where: Represents eigenvalues The corresponding eigenvector; m i =1, 2, ..., m;

[0151] Through linear transformation, let H l =X′ l V l , α l =V l T Ω l ,get:

[0152] y′ l =H l α l +E l ';

[0153] The least squares method is used to obtain α l Unbiased estimate of After that, we calculate

[0154] Where: H l , α l is an intermediate variable; Represents α l The unbiased estimation matrix of V l express The matrix formed.

[0155] Principal component analysis is based on the premise of minimizing information loss, and integrates the original variables into a few new variables through linear combination; using new variables to replace the original variables in data modeling can greatly reduce the computational workload in the analysis process; the selection of new variables by principal components is not a simple choice of the original variables, but the result of the reorganization of the original variables, so it will not cause a large amount of loss of the original variable information and can represent most of the information of the original variables; at the same time, the selected new variables are uncorrelated with each other, which can effectively solve many problems brought to the analysis application by variable information overlap, multicollinearity, etc.

[0156] Considering that X may be caused by insufficient measurement points, etc. l There is complex collinearity, so the principal component analysis method is used to identify the coefficient Ω l .

[0157] Preferably, after identifying Ω l After that, the coordinates of the point to be measured under {T} can be obtained through one online visual measurement; the coordinates of the point to be measured under {T} P T The calculation formula is as follows:

[0158]

[0159] Preferably, the following operations can be repeated: constructing a visual measurement error compensation function using a multivariate linear function; identifying coefficients of the error compensation function using principal component analysis; until the absolute value change rate of the coefficients of the error compensation function is lower than a given threshold.

[0160] A hand-eye calibration method taking into account robot body error and visual measurement error may include the following steps:

[0161] Step 1: Fix the visual sensor 4 at the end of the robot, fix a standard sphere 2 of known size in the robot workspace, and use the laser tracker 1 to measure the robot base coordinate system, the end position of the robot in different postures, and the coordinates of the standard sphere 2; drive the robot so that the standard sphere 2 is within the field of view of the visual sensor 4; use the visual sensor 4 to collect the surface point cloud of the standard sphere 2 and calculate the coordinates of the sphere center.

[0162] Step 2: Assume: the robot base coordinate system is {B}, the robot end coordinate system is {T}, the visual sensor 4 coordinate system is {V}, the standard ball 2 coordinate system is {W}, and the laser tracker 1 coordinate system is {L}; Assume the hand-eye matrix is The following formula is constructed based on the hand-eye calibration solution model:

[0163]

[0164] Where: Represents the homogeneous transformation matrix from {V} to {W}; represents the hand-eye matrix, represents the homogeneous transformation matrix from {B} to {W}; Represents the homogeneous transformation matrix from {B} to {T}.

[0165] Step 3: When performing error compensation measurement on the center point P of the standard ball, k is the measurement pose number of the robot end, k = 1, 2, ..., K t ;K t represents the total number of measured poses at the end of the robot; p V,k is the measurement value of the visual sensor 4 after the robot moves to a certain measurement posture k, △P T,S,k When P is the kth measured pose T,S,k Relative P T,L,k Deviation of P T,L,k is the coordinate value of point P measured by the laser tracker at {T} during the k-th measurement pose; P T,S,k The coordinates of point P under {T} measured by the visual sensor 4 at the kth measurement posture are obtained by the laser tracker 1. T,L,k , the coordinates p of point P in {V} are obtained by visual sensor 4 in the measurement posture k state V,k .

[0166] Step 4: Let ρ l,k Indicates the error compensation value of the lth coordinate axis of point P in the measurement posture k state; combined with the hand-eye matrix structure but

[0167] Step 5: After the error compensation value calculation is completed, the robot moves to the next measurement pose and continues to measure until it traverses the K t The multivariate linear regression equation is constructed based on the measured poses.

[0168] The working principle of the present invention is further described below with reference to a preferred embodiment of the present invention:

[0169] A hand-eye calibration method taking into account robot body errors and visual measurement errors, the main features of which include the following steps:

[0170] Step 1: Use external measurement equipment to construct the hand-eye matrix.

[0171] For ease of description, the coordinate systems involved in the hand-eye calibration process are defined as follows: the robot base coordinate system is {B}, the robot end coordinate system is {T}, the visual sensor 4 coordinate system is {V}, the standard sphere 2 coordinate system is {W}, and the laser tracker 1 coordinate system is {L}; based on the hand-eye calibration solution model: AX = ZB, the following homogeneous transformation matrix formula can be obtained:

[0172]

[0173] Where, represents the homogeneous transformation matrix from system {V} to system {W}; represents the hand-eye matrix, represents the homogeneous transformation matrix from system {B} to system {W}; Denotes the homogeneous transformation matrix from system {T} to system {B}. Note that the robot body still has errors even after calibration. If represents the true value of the homogeneous transformation matrix from system {T} to system {B}, then Therefore, there is an error in the hand-eye matrix X solved by the model AX=ZB.

[0174] A visual sensor 4 is fixed to the end of the robot, and a standard sphere 2 of known dimensions is fixed within the robot's workspace. The robot is then driven so that the standard sphere 2 is within the visual sensor's field of view. The visual sensor 4 then collects a point cloud from the surface of the standard sphere 2 and calculates the coordinates of the sphere's center. After completion, the robot moves to the next pose and continues measuring until n (n>10) sets of experimental data have been collected. Since the positional relationship between the standard sphere 2 and the robot base is fixed, for any measured pose, we have:

[0175]

[0176] Where, represents the homogeneous transformation matrix from system {B} to system {T} when the i-th robot measures its posture; represents the homogeneous transformation matrix from system {L} to system {T} when the i-th robot measures the pose; represents the homogeneous transformation matrix from system {B} to system {L}. At the same time, and All are measured by a laser tracker 1.

[0177]

[0178] Where, PV,i P represents the coordinates of the center of the standard ball obtained by the visual sensor 4 when the i-th robot measures the posture in the frame {V}; L,B It represents the coordinates of the center of the standard ball in the frame {B}, which is measured by the laser tracker 1.

[0179] Substituting variables in formula (3) and writing it in the form of least squares, we have:

[0180]

[0181] Where, express And D={d1,d2,...,d n}; Indicates P V,i And C={c1,c2,...,c n};i pose Indicates the i-th pose elements; n represents the number of times the visual sensor 4 performs visual measurements; Represents the rotation transformation of the hand-eye matrix; Represents the translation vector of the hand-eye matrix.

[0182] In order to solve the rotation transformation and translation vector of the hand-eye matrix respectively, the sets C and D are decentralized:

[0183]

[0184] Where, express Decentralized value; express Decentralized value.

[0185] Substituting formula (4) into formula (3) and expanding the square, we can obtain:

[0186]

[0187] Considering the difficulty of directly solving equation (6), Perform SVD decomposition, then and It can be expressed as:

[0188]

[0189] Where U represents The left singular vector obtained after SVD decomposition; V represents Right singular vectors obtained after SVD decomposition.

[0190] Step 2: Use multivariate linear function to construct visual measurement error compensation function.

[0191] In order to express the visual measurement error, an image coordinate system and a pixel coordinate system are established on the images of the left and right cameras of the visual sensor 4. Without loss of generality, a point to be measured p(x v ,y v ,z v ) is expressed as (u, v) in the pixel coordinates of a single camera and (x, y) in the image coordinate system, then the two satisfy:

[0192]

[0193] In the formula, let: u and v represent the horizontal and vertical coordinate values ​​of the point to be measured in the current camera pixel coordinate system; x and y represent the horizontal and vertical coordinate values ​​of the point to be measured in the current camera image coordinate system; dx represents the physical size of each pixel in the horizontal axis of the image coordinate system, dy represents the physical size of each pixel in the vertical axis of the image coordinate system, u0 and v0 represent the horizontal and vertical coordinate values ​​of the origin of the image coordinate system in the pixel coordinate system; x v 、y v 、z v Corresponding to the measured coordinate values ​​of the horizontal, vertical, and vertical axes of the point to be measured under {V}; (u, v) represents the coordinates of point P in the current camera pixel coordinate system; (x, y) represents the coordinates of point P in the current camera image coordinate system; dx represents the physical size of each pixel in the x-axis of the image coordinate system, dy represents the physical size of each pixel in the y-axis of the image coordinate system, and (u0, v0) represents the coordinates of the origin of the image coordinate system in the pixel coordinate system.

[0194] After the visual sensor 4 is installed, the geometric relationship between the left and right cameras is fixed, and the point to be measured p(x v ,y v ,z v ) under the system {V} can be expressed as:

[0195]

[0196] In the formula, (x v ,y v ,z v ) represents the coordinates of point P in the system {V}; L represents the baseline length; f represents the focal length of the camera; (x1, y1) and (x2, y2) represent the coordinates of the point P to be measured in the left and right camera image coordinate systems, respectively.

[0197] Because the pinhole imaging model is an ideal linear model, the visual sensor 4 still has residual distortion even after internal parameter calibration. In addition, due to the limited pixel size of the image, combining equations (8) and (9), it can be seen that the coordinates (x1, y1) and (x2, y2) of the test point in the image coordinate system have errors. Therefore, the true value of the test point P in the system {V} can be expressed as:

[0198]

[0199] Where, Indicates the true value of the test point P under the system {V}; δ V Represents the visual measurement error.

[0200] Based on this, for a given robot measurement pose and a fixed point P in the field of view of the visual sensor 4, P T,S The position information of the fixed point P determined by the visual sensor 4 under the system {T} can be expressed as:

[0201]

[0202] Where, P T,S Denotes the coordinates of the point P to be measured in the frame {T} determined by the visual sensor 4. In addition, let P T,L It represents the position information of point P in the system {T} determined by laser tracker 1. Theoretically, P T,L =P T,S However, from equations (3) and (10), we can see that there are errors in both the hand-eye matrix and the visual sensor 4, so P T,L ≠P T,S Therefore, if P T,L As a benchmark, P T,S Relative P T,L The deviation is expressed as:

[0203]

[0204] Where, △P T,S Indicates P T,S Relative P T,L deviation.

[0205] Hand-eye matrix after vision sensor 4 is installed The error is a constant. Combining equations (8), (9) and (12), we can know that △P T,S As the coordinates of the measuring point P in the visual field of the visual sensor 4 change, △P T,S is the measured value p V function.

[0206] Because △P T,S With p VThe functional relationship between them is very complex, so we can consider using p V The quadratic response surface is used to approximate △P T,S , and use multiple linear regression to estimate the coefficients of the response surface. For convenience, △P T,S and p V Denoted as:

[0207]

[0208] Where ρ1, ρ2, ρ3 represent △P T,S Coordinates △P x ,△P y ,△P z .

[0209] Let ρ r To approximate △P T,S About p V The quadratic response surface function of r Expressed as:

[0210]

[0211] Where, α 0,r , α s,r , α s,t,r , α o,r Represents the coefficients of each polynomial.

[0212] Substituting variables in formula (13) and writing it in the format of a multivariate linear function, we have:

[0213]

[0214] in:

[0215]

[0216]

[0217] Where n l Represents ρ l The order of the terms in the linear function polynomial, n l ≥0; represents the error compensation function ρ l The nth linear function polynomial of l Monomial, n l =0, Represents a monomial The coefficient of Q l Indicated by 1, The vector composed of Ω l Indicated by Vector composed of.

[0218] Now, p V,k is the measurement value of the visual sensor 4 after the robot moves to a certain measurement posture k, k is the measurement posture number of the robot end, k = 1, 2, ..., K t ;K t Indicates the total number of measured poses of the robot end.

[0219] △P can be constructed in three steps T,S,k ,△P T,S,k When P is the kth measured pose T,S,k Relative P T,L,k Deviation of P T,L,k is the coordinate value of point P measured by the laser tracker at {T} during the k-th measurement pose; P T,S,k Represents the coordinates of point P measured by the visual sensor 4 under {T} at the kth measurement posture.

[0220] 1. Given a measuring point in the field of view, the coordinates P of the measuring point in the system {T} are obtained by measuring with the laser tracker 1. T,L,k ;

[0221] 2. Obtain the coordinates of the point to be measured p by the visual sensor 4 V,k , and combined with the hand-eye matrix structure

[0222] 3. T,L,k and Subtract and get △P T,S,k .

[0223] Finally, by variable substitution, we get ρ l The observed value at this measurement point According to formula (15), the measured point ρ can be obtained l,k The calculated value of According to this, the mobile robot terminal makes it traverse K in the field of view of the visual sensor 4 t The error compensation measurement pose can be used to construct the following multivariate linear regression equation:

[0224] y l =X l Ω l +E l (18)

[0225]

[0226] Where K t represents the total number of error compensation measurement poses; k represents the kth error compensation measurement pose; y l Represents the observed value ρ l,k The vector composed ofl,k Represents Q l The vector corresponding to the kth measured pose; X l Represents Q l,k The matrix composed of l,k Represents ρ l The kth observation value ρ l,k Calculated value with its model The residual between l Indicates e l,k Vector composed of.

[0227] Step 3: Use principal component analysis to identify the coefficients of the error compensation function.

[0228] In order to ensure that the multiple linear regression model has excellent compensation ability, when estimating Ω l The following measures are taken:

[0229] (1) Visual measurement point planning. In order to improve the mapping ability of sampling points to the distribution law of visual measurement errors, the sampling points should be distributed as evenly as possible in the field of view.

[0230] (2) Decentralization and dimensionless processing of measurement coordinates. In order to reduce the influence of different characteristic scales of original data on fitting parameters, η j Decentralized and dimensionless processing:

[0231]

[0232] Where η l Indicates p V The lth coordinate of 0,l represents η l The mean of ; △η l represents η l The value range of η l ' indicates that η l The lth coordinate obtained after decentralized and dimensionless processing.

[0233] (3) Parameter identification algorithm selection. Note that although the residual e l is considered to have zero mean, but its variance may vary with the measurement point. Therefore, if the model is homoscedastic, no weighting is required; otherwise, a weighted reconstruction of the multivariate linear regression equation such as Equation (16) is required.

[0234]

[0235] in:

[0236]

[0237]

[0238]

[0239] Where, ρ l,j,k Represents ρ l The j-th repeated measurement value at the k-th measurement position, n t represents the number of repeated measurements; Represents ρ l,j,k The mean of Represents ρ l,j,k Variance of D l represents the weighted reconstruction matrix; y l ′ represents the reconstructed y l ;X l ′ represents the reconstructed X l ;E l ′ represents the reconstructed E l ;E l ′ can be regarded as obeying a normal distribution with a mean of zero and a fixed variance.

[0240] Considering that insufficient measurement points may cause X l There is complex collinearity, so the principal component analysis method is used to identify Ω l First, solve The eigenvalue λ l and its corresponding eigenvector v l , and sort by eigenvalue from large to small; then according to the variance cumulative contribution rate threshold Determine the minimum number of principal components m. The cumulative contribution rate of variance can be expressed as:

[0241]

[0242] Where μ l Indicates the cumulative contribution rate of variance; express After the eigenvalues ​​of the mth i eigenvalues.

[0243] Based on this, using λ l,1 ,λ l,2 ,...,λ l,m The corresponding eigenvector can construct the matrix V l :

[0244] V l =[v l,1 v l,2 ... v l,m ] (twenty two)

[0245] Where, Represents eigenvalues The corresponding eigenvector.

[0246] By linear transformation H l =X l ′V l and α l =V l T Ω l , formula (16) can be rewritten as:

[0247] y l =H l α l +E l (twenty three)

[0248] Then we use the least squares method to get α l Unbiased estimate of After that, we can calculate Generally, [μ l ]=99.5%. In addition, the measurement error is considered to be nonlinear, while this method is based on the linear assumption.

[0249] In order to improve the recognition accuracy, the “measurement-identification” process needs to be repeated until ||Ω l The rate of change of || is below a given threshold.

[0250] Step 4: Use the visual measurement error compensation function to correct the coordinates of the measurement point.

[0251] After identifying Ω l After that, the coordinates of the measured point in the system {T} can be obtained through an online visual measurement. First, a measured point is given in the measured area. The mobile robot makes the measured point enter the field of view of the visual sensor 4 and takes a photo of the measured point. Then, the image processing constructs p V , note that the coordinates of the point to be measured are P under the system {T} T satisfy:

[0252]

[0253] Therefore, the identified Ω l Compensation correction P T .

[0254] The embodiments described above are only used to illustrate the technical ideas and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. The scope of the patent of the present invention cannot be limited by these embodiments alone. That is, any equivalent changes or modifications made to the spirit disclosed by the present invention still fall within the scope of the patent of the present invention.

Claims

1. A hand-eye calibration method taking into account robot body error and visual measurement error, characterized in that: The method comprises the following steps: constructing a hand-eye matrix using a measuring device; constructing a visual measurement error compensation function using a multivariate linear function; identifying coefficients of the error compensation function using a principal component analysis method; and correcting the coordinates of the measurement point using the visual measurement error compensation function; The method of constructing a visual measurement error compensation function using a multivariate linear function includes the following steps: set up: The coordinate measurement value of a point P to be measured in space under {V} is p V , the true value of the coordinates of point P under {V} is The coordinate measurement error of point P under {V} is δ V , then: Let P T,S Represents the coordinates of point P measured by the visual sensor under {T}, P T,S The calculation formula is as follows: Let P T,L Indicates the coordinate value of point P measured by laser tracker under {T}; T,L As a benchmark, P T,S Relative P T,L The deviation is expressed as: Where: ΔP T,S Indicates P T,S Relative P T,L Deviation; Hand-eye matrix after vision sensor installation The error is a constant, then ΔP T,S With the measured value p V , so ΔP T,S is the measured value p V function; Use p V Multivariate linear function to approximate ΔP T,S , obtain the error compensation function, and use multiple linear regression to estimate the coefficients of the error compensation function.

2. The hand-eye calibration method taking into account robot body error and visual measurement error according to claim 1 is characterized in that: The method of constructing a hand-eye matrix using a measurement device includes the following steps: A vision sensor is fixed to the end of the robot, a standard sphere of known size is fixed in the robot's workspace, and a laser tracker is set up to measure the robot's base coordinate system, the end position of the robot in different measurement postures, and the coordinates of the center of the standard sphere. The robot is driven so that the standard sphere is within the field of view of the vision sensor. The vision sensor collects the point cloud of the standard sphere's surface and calculates the coordinates of the sphere's center. Assume: the robot base coordinate system is {B}, the robot end coordinate system is {T}, the visual sensor coordinate system is {V}, the standard ball center coordinate system is {W}, and the laser tracker coordinate system is {L}; based on the hand-eye calibration solution model, the following formula is obtained: Where: Represents the homogeneous transformation matrix from {V} to {W}; represents the hand-eye matrix, represents the homogeneous transformation matrix from {B} to {W}; Represents the homogeneous transformation matrix from {B} to {T}; The coordinates of the center of the standard ball under {V} are obtained using a visual sensor. For any robot measurement pose, we have: Where: represents the homogeneous transformation matrix from {B} to {T} at the i-th measurement pose; represents the homogeneous transformation matrix from {L} to {T} at the i-th measurement pose; represents the homogeneous transformation matrix from {B} to {L}; and All are measured by laser tracker; then: Where: P V,i P represents the coordinates of the center of the standard ball under {V} obtained by the visual sensor at the i-th measurement pose; L,B represents the coordinates of the center of the standard ball under {B}, P L,B Measured by laser tracker; set up D={d1,d2,...,d n }; C={c1,c2,...,c n };i pose Indicates the element number in sets C and D, i pose =1, 2, ..., n; n represents the number of times the vision sensor performs visual measurements; right Perform variable substitution and write it in the form of least squares to obtain: Where: Represents the rotation transformation of the hand-eye matrix; Represents the translation vector of the hand-eye matrix; Decentralize sets C and D: i pose =1~n; Where: express Decentralized value; express Decentralized value; Further we get: right Perform SVD decomposition, then and Expressed as: Where U represents The left singular vector obtained after SVD decomposition; V represents Right singular vectors obtained after SVD decomposition.

3. The hand-eye calibration method taking into account robot body error and visual measurement error according to claim 1 is characterized in that: Use p V The quadratic response surface is used to approximate ΔP T,S , and use multiple linear regression to estimate the coefficients of the response surface.

4. The hand-eye calibration method taking into account robot body error and visual measurement error according to claim 1 is characterized in that: Use p V The method for obtaining the error compensation function from the multivariate linear function includes the following steps: set up: ρ l Indicates ΔP T,S The error compensation function of the lth coordinate; l represents the coordinate axis number corresponding to the error compensation function; l = 1 to 3; Use linear function to express ρ l ,set up: in: Where: n l Represents ρ l The order of the terms in the linear function polynomial, n l ≥0; represents the error compensation function ρ l The nth linear function polynomial of l Monomial, n l =0, Represents a monomial The coefficient of Q l Indicated by The vector composed of Ω l Indicated by The vector of components; Assume: k is the measurement pose number of the robot end, k = 1, 2, ..., K t ;K t represents the total number of measured poses at the end of the robot; ρ l,k Represents ρ l The kth measured pose observation value; Indicates the corresponding ρ l The calculated value of the kth measured pose; let Q l,k Represents Q l The vector corresponding to the kth measured pose; set up: Then we get the following corresponding p V Error compensation function: y l =X l Ω l +E l .

5. The hand-eye calibration method taking into account robot body error and visual measurement error according to claim 4 is characterized in that: The measurement positions are evenly distributed within the field of view of the visual sensor; and the coordinates of the measured points are decentralized and dimensionless.

6. The hand-eye calibration method according to claim 4, wherein: Reconstruct the multivariate linear function y according to the following weighted formula l =X l Ω l +E l : y l ′=D l y l ; X′ l =D l X l ; AND l ′=D l AND l ; in: Where: ρ l,j,k Represents ρ l The jth repeated measurement value at the kth measurement pose, j represents the value of ρ l The repeated measurement number at the kth measurement pose, n t Represents ρ l The number of repeated measurements at the kth measurement pose; Represents ρ l,j,k The mean of Represents ρ l,j,k Variance of D l represents the weighted reconstruction matrix; y l ′ represents the reconstructed y l ; X′ l Represents the reconstructed X l ;E l ′ represents the reconstructed E l ;E l ′ is considered to obey a normal distribution with a mean of zero and a fixed variance.

7. The hand-eye calibration method according to claim 6, wherein: The principal component analysis method is used to identify the coefficient Ω of the error compensation function. l The method comprises the following steps: According to the cumulative contribution rate threshold Determine the minimum number of principal components m; The calculation formula for the cumulative contribution rate of variance is as follows: Where μ l Indicates the cumulative contribution rate of variance; express After the eigenvalues ​​of the mth i eigenvalues; λ l express The set of all eigenvalues; m i express The eigenvalue numbers after sorting from largest to smallest, m represents the minimum number of principal components; Based on this, construct the matrix V l : V l =[v l,1 v l,2 ...v l,m ]; Where: Represents eigenvalues The corresponding eigenvector; m i =1, 2, ..., m; Through linear transformation, let H l =X′ l V l , α l =V l T Ω l ,get: y′ l =H l a l +E l ′; The least squares method is used to obtain α l Unbiased estimate of After that, we calculate Where: H l , α l is an intermediate variable; Represents α l The unbiased estimation matrix of V l express The matrix formed.

8. The hand-eye calibration method according to claim 4, wherein: After identifying Ω l After that, the coordinates of the point to be measured under {T} are obtained through an online visual measurement; the coordinates of the point to be measured under {T} P T The calculation formula is as follows:

9. The hand-eye calibration method taking into account robot body error and visual measurement error according to claim 1 is characterized in that: Repeat the following operations: construct a visual measurement error compensation function using a multivariate linear function; identify the coefficients of the error compensation function using principal component analysis; until the absolute value change rate of the coefficients of the error compensation function is lower than a given threshold.

Images