A robot automatic calibration and three-dimensional measurement method for large aerospace components

Abstract

The present invention discloses a robot automatic calibration and three-dimensional measurement method for large aerospace components. Firstly, the hand-eye relationship equation is linearly solved based on the Kronecker product and Moore-Penrose inverse, which can reduce the influence of system noise and calculation errors in the calibration process while ensuring a more accurate closed solution, and the Rodrigues formula is used to solve the hand-eye relationship matrix H. cg The rotation part is orthogonalized to eliminate the influence of measurement noise and improve the efficiency of the algorithm. Then, in order to further improve the hand-eye calibration accuracy and improve the point cloud registration error, the differential evolution (DE) algorithm based on minimizing the Euclidean transformation error is introduced and applied to the hand-eye calibration to solve the instability of the solution of the hand-eye equation due to the lack of constraints on the rotation matrix. Finally, multiple unit measurement point cloud data of large aerospace components are collected to complete the 3D morphology measurement task of large aerospace components as the object to be measured.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional shape measurement, and more specifically, relates to a robot automatic calibration and three-dimensional measurement method for large aerospace components. Background Art

[0002] Key parts in the aerospace field are usually large aerospace components with large sizes and complex shapes. The field of view of general three-dimensional measurement equipment such as structured light cameras or laser scanners is limited, and only local point cloud data of large measurement objects can be collected from a single perspective. In order to reconstruct the three-dimensional morphology of large aerospace components, it is necessary to unify the local point cloud data obtained by measuring each unit in the same reference coordinate system. Therefore, the three-dimensional reconstruction technology based on the constrained posture of industrial robots provides a new idea for non-contact measurement of large aerospace components.

[0003] The three-dimensional reconstruction technology based on industrial robots uses industrial robots as motion carriers and determines the position relationship of the unit measurement coordinate system of each perspective through the constraints between the coordinate systems of the robot's axis joints. While retaining the non-contact and fast characteristics of visual measurement technology, the robot's fast and flexible characteristics enhance the flexibility of the entire measurement system.

[0004] The current hand-eye calibration method for industrial robots mainly uses cameras to shoot calibration objects for posture recognition, obtain camera parameters and robot posture information, establish hand-eye calibration equations, and then solve the hand-eye equations. At present, different mathematical tools such as spinors, quaternions, and dual quaternions are mainly used to describe spatial motion, or different motion constraints such as pure movement or pure rotation are added to decouple rotation and translation to obtain the hand-eye relationship matrix. However, the above method only considers minimizing algebraic errors to calculate the hand-eye relationship matrix and complete the hand-eye calibration, and does not consider the impact of image noise on the hand-eye calibration accuracy. This leads to low calibration accuracy of the posture relationship obtained by linear solution, which affects the accuracy and completeness of three-dimensional shape measurement.

[0005] In addition, the problems caused by the accuracy of solving the hand-eye relationship and the efficiency of completing related measurement tasks are very serious problems currently faced by this type of measurement device. In order to further solve the accuracy requirements of three-dimensional measurement tasks, it is necessary to compensate for the settlement results of the hand-eye equation. Due to the error between the actual world coordinates and the world coordinates calculated by the calibration results, such as according to the Chinese invention patent application "A robot-based three-dimensional shape measurement method for complex structural parts" published on December 30, 2022 with publication number CN115546289A, only the mean compensation of the error cannot meet the measurement task requirements. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a robot automatic calibration and three-dimensional measurement method for large aerospace components, so as to linearly solve a more accurate closed solution, simplify the calculation and improve the efficiency, and more specifically reduce the influence of system noise and errors caused by calculations during the calibration process, while correcting the hand-eye relationship accuracy to ensure the registration accuracy of multi-view point cloud data, thereby greatly improving the efficiency of multi-view three-dimensional shape measurement.

[0007] To achieve the above-mentioned purpose, the present invention provides a robot automatic calibration and three-dimensional measurement method for large aerospace components, which is characterized by comprising:

[0008] (1) Establishing the hand-eye calibration equation

[0009] The binocular camera is installed on the end flange of the robot, and the robot is controlled to move to the i-th shooting posture. A corner point of the checkerboard calibration plate is photographed, and the calibration plate image is obtained at this shooting posture. At the same time, the robot posture information and the position P of the corner point in the camera coordinate system are recorded. i , in this way, n shooting postures are photographed to obtain n sets of calibration plate images and corresponding robot posture information;

[0010] According to Zhang's calibration method, the rotation vector R of the calibration plate relative to the camera in each set of calibration plate images is obtained. ci and the translation vector T ci , and converted into a rotation and translation matrix, that is, the camera external parameter matrix H is obtained ci , i = 1, 2, ..., n, calculate the rotation vector R relative to the base according to the robot posture information gi and the translation vector T gi , get the robot pose matrix H gi ,i=1,2,...,n, where:

[0011]

[0012] For any two transformed shooting poses u and v, the camera extrinsic matrix and robot pose matrix are used to establish the hand-eye calibration equation:

[0013]

[0014] in:

[0015]

[0016] Among them, R gu,v Representation Matrix The rotation matrix in , T gu,v Representation Matrix The translation vector in R cg Represents the hand-eye relationship matrix Hcg The rotation matrix in , T cg represents the translation vector in the hand-eye relationship matrix, R cu,v Representation Matrix The rotation matrix, T cu,v Representation Matrix The translation vector in ;

[0017] (2) Use Kronecker product and Moore-Penrose inverse to transform the hand-eye calibration equation into a least squares problem and use singular value decomposition to calculate the hand-eye relationship matrix H cg

[0018] Set up a system of linear equations:

[0019]

[0020] Where I is the identity matrix, represents Kronecker product, vec represents vectorized operation;

[0021] Transform all two shots under the pose u, v matrix:

[0022]

[0023] Place them by columns to get a matrix R;

[0024] Perform singular value decomposition on the matrix R to obtain the U matrix, i.e. the left singular matrix, the singular value matrix ∑, and the V matrix, i.e. the right singular matrix;

[0025] Transform all two shots under the pose u, v matrix:

[0026]

[0027] Place them by column and get a matrix T g ;

[0028] Take the column vector V∑ -1 U T T g The first 9 elements of are reduced to a 3×3 matrix form, and the matrix

[0029] Then, for the matrix Orthogonalized singular value decomposition is Among them, U R is a right singular matrix, ∑ R is the singular value matrix, V R is the left singular matrix, and the hand-eye relationship matrix H is obtained cg The rotation matrix R cg Right now

[0030] The matrix (R gu,v -I) Place them by columns to get a matrix R g , the matrix (R cg T cu,v -T gu,v ) are placed column by column to obtain a matrix T,

[0031] According to the rotation matrix R cg Calculate the hand-eye relationship matrix H cg The translation vector T cg :

[0032] T cg =R g -1 T

[0033] (3) The calculated hand-eye relationship matrix H cg Correction is performed to obtain the hand-eye relationship matrix for coarse registration of multi-view point clouds

[0034] 3.1) Obtain the corresponding point set between the real coordinates of the J groups of corner points and the point cloud coordinates

[0035] 3.1.1) Fix the probe device on the end flange of the robot, establish the tool coordinates through the five-point method and the teaching pendant, and obtain the position coordinates ΔP of the probe end relative to the robot end flange coordinate system 0 The robot drives the probe to move to the top of the four corner points of the first square in the upper left corner of the calibration plate, and records the coordinates of the robot end flange relative to the base coordinate system from the teach pendant. The probe is continuously driven to move to the top of the four corner points of the square on the calibration plate, and the coordinates of the robot end flange relative to the base coordinate system at the corner points are obtained. And calculate the true coordinates of all corner points of the calibration plate:

[0036]

[0037] in, is the three-dimensional coordinate of the robot end flange directly above the lth corner point relative to the base coordinate system, Δx 0 ,Δy 0 ,Δz 0 is the position coordinate of the probe end relative to the robot end flange coordinate system, L is the number of corner points on the calibration plate, T represents transposition, and B represents the robot base coordinate system;

[0038] 3.1.2) Shoot and reconstruct the calibration plate 3D point cloud at a certain robot posture, reconstruct the calibration plate 3D point cloud, click and obtain the point cloud coordinates of the four corner points of the first square in the upper left corner of the calibration plate relative to the binocular camera coordinate system in sequence, and calculate the point cloud coordinates P of all the calibration plate corner points corresponding to the real coordinates of the corner points. 1 l :

[0039]

[0040] in, is the point cloud coordinate P 1 l The three-dimensional coordinates of are represented by, and C represents the binocular camera coordinate system;

[0041] 3.1.3) According to the robot posture information, the robot posture matrix when shooting the calibration plate point cloud is obtained as H g , from the known real coordinates of the corner point P 0 l , get the corner point coordinates in the flange coordinate system converted to the robot posture

[0042]

[0043] in, is the corner point coordinate The three-dimensional coordinates of G represent the robot end flange coordinate system;

[0044] This gives us a set of corresponding points

[0045] 3.1.4) Change the robot posture J-1 times, repeat steps 3.1.2) and 3.1.3), and get the jth set of corresponding points, which is re-recorded as In this way, we get the set P of corresponding points between the real coordinates of the J groups of corner points and the point cloud coordinates;

[0046] 3.2) Optimize the hand-eye relationship matrix H based on minimizing the Euclidean transformation error cg

[0047] 3.2.1) Hand-eye relationship matrix H cg Initialize the population

[0048] Construct a population of size M and initialize each individual x h for:

[0049]

[0050] Where h represents the individual number, h = 1, 2, ... M, 0 represents the 0th generation population, and the subscript number is the component number in the individual, [r ox ,r oy ,r oz ] is the hand-eye relationship matrix H cg The rotation matrix R cg The corresponding rotation vector, [x o ,y o ,z o ] is the hand-eye relationship matrix H cg The translation vector T in cg The transpose of , that is:

[0051]

[0052] Among them, Rodrigues() represents the rotation vector to rotation matrix operation;

[0053] rand(0,1) means to take a random number between 0 and 1;

[0054] r xmaxmin ,r ymaxmin ,r zmaxmin ,x maxmin ,y maxmin ,z maxmin They respectively represent the difference between the maximum and minimum values ​​of the rotation vectors x, y, z in the hand-eye relationship matrix and the translation vectors x, y, z in the hand-eye relationship matrix;

[0055] 3.2.2) Individual variation and crossover

[0056] Perform mutation and crossover on individuals in a population:

[0057]

[0058]

[0059] in, is the kth component of the hth individual in the tth generation population, Mutation() is the mutation operation, is the kth component of the hth mutant individual of the tth generation, Crossover() is the crossover operation, is the kth component of the hth experimental individual of the tth generation, k = 1, 2, …, 6;

[0060] 3.2.3) Use the greedy algorithm to select individuals x to enter the next generation population h (t+1):

[0061]

[0062] in:

[0063]

[0064]

[0065]

[0066]

[0067] in, are the hth individual x in the tth generation population h (t) The corresponding rotation matrix:

[0068]

[0069]

[0070] in, are the hth individual u in the tth generation population h (t) The corresponding rotation matrix:

[0071]

[0072]

[0073] 3.2.4) t = t + 1, return to step 3.2.2) until the set termination condition is reached;

[0074] 3.2.5) Compare M individuals x h (t+1),h=1,2,…,the fitness function value f(x h+1 (t),P), and the individual corresponding to the minimum fitness function value is recorded as x′(t)={x′ 1 (t),x′ 2 (t),x′ 3 (t),x′ 4 (t),x′ 5 (t),x′ 6 (t)}, then the corrected hand-eye calibration matrix for:

[0075] R′ cg =Rodrigues([x′ 1 (t),x′ 2 (t),x′ 3 (t)]), T′ cg =[x′ 4 (t),x′ 5 (t),x′ 6 (t)]T

[0076] (4) Three-dimensional shape measurement

[0077] Debug the binocular camera so that it can clearly capture the measurement object, i.e., the large aerospace component to be measured, so that the left and right cameras are kept at the same horizontal position and a certain distance is maintained, and binocular positioning is completed; debug the robot so that it can carry the binocular camera to perform three-dimensional measurement tasks, ensuring that the full picture of the measurement object is captured;

[0078] Structural light images are taken at each measurement position, and the unit measurement point cloud data of the measured object is reconstructed. Record the measured posture and calculate the posture matrix of the robot Where m is the number of measurement poses for three-dimensional shape measurement of the measured object.

[0079] Measure point cloud data based on the obtained units Each view pose matrix And the optimized hand-eye relationship matrix obtained in step (4) Calculate the rigid body transformation matrix of each unit measurement point cloud data from the binocular camera coordinate system to the robot base coordinate system:

[0080]

[0081] The unit measurement point cloud data obtained by the binocular camera is converted to the unified coordinate system of the robot base to obtain the single-view point cloud under the qth measurement pose after registration

[0082]

[0083] Realize the registration of multi-view point cloud data, thereby completing the three-dimensional shape measurement of large aerospace components.

[0084] The object of the invention of the present invention is achieved in this way:

[0085] In order to improve the accuracy and completeness of the three-dimensional shape measurement of large aerospace components, the robot automatic calibration and three-dimensional measurement method for large aerospace components of the present invention comprises: firstly, a linear solution is performed on the hand-eye relationship equation based on the Kronecker product and the Moore-Penrose inverse, and the error influence caused by the system noise and calculation in the calibration process is more targeted while ensuring a more accurate closed solution, and the Rodrigues formula is used to solve the hand-eye relationship matrix H cgThe rotation part of the calibration plate is orthogonalized to eliminate the influence of measurement noise and improve the efficiency of the algorithm. Then, in order to further improve the accuracy of hand-eye calibration and improve the point cloud registration error, the differential evolution (DE) algorithm based on minimizing the Euclidean transformation error is introduced and applied to hand-eye calibration to solve the instability of the hand-eye equation solution due to the lack of constraints on the rotation matrix. The point cloud coordinates P of the corner points of the calibration plate are converted into 1 l and the real coordinates of the corresponding corner points The rigid body transformation relationship between them is used as the source of composition, and the cost function of minimizing the Euclidean transformation error is established. The initial hand-eye matrix is ​​converged and iterated, and its rotation and translation parts are corrected at the same time to obtain the hand-eye relationship matrix for coarse registration of multi-view point clouds. Finally, multiple unit measurement point cloud data of large aerospace components are collected, and the point cloud registration of the unit measurement point cloud data to the robot base coordinate system is completed, so as to achieve the function of obtaining the overall point cloud with a contour closer to the real one more quickly and efficiently, and complete the three-dimensional morphology measurement task of large aerospace components as the objects to be measured. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 It is a flow chart of a specific implementation method of the robot automatic calibration and three-dimensional measurement method for large aerospace components of the present invention;

[0087] Figure 2 It is a schematic diagram of the Euclidean transformation of the corner points of the calibration plate;

[0088] Figure 3 It is a schematic diagram of the real coordinates of the corner points and the point cloud coordinates;

[0089] Figure 4 This is a comparison chart of the results before and after hand-eye calibration correction;

[0090] Figure 5 It is a schematic diagram of point cloud data measured by a local unit;

[0091] Figure 6 It is a schematic diagram of converting multiple unit measurement point cloud data into the same coordinate system;

[0092] Figure 7 This is a schematic diagram of the three-dimensional morphology measurement results of a large aerospace component. DETAILED DESCRIPTION

[0093] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0094] Figure 1It is a flow chart of a specific implementation method of the robot automatic calibration and three-dimensional measurement method for large aerospace components of the present invention.

[0095] In this embodiment, if Figure 1 As shown, the robot automatic calibration and three-dimensional measurement method for large aerospace components of the present invention includes:

[0096] Step S1: Establish hand-eye calibration equation

[0097] The binocular camera is installed on the end flange of the robot, and the robot is controlled to move to the i-th shooting posture. A corner point of the chessboard calibration plate is photographed, and the calibration plate image is obtained at this shooting posture. At the same time, the robot posture information and the position P of the corner point in the camera coordinate system are recorded. i In this way, n shooting postures are shot to obtain n sets of calibration plate images and corresponding robot posture information.

[0098] According to Zhang's calibration method, the rotation vector R of the calibration plate relative to the camera in each set of calibration plate images is obtained. ci and the translation vector T ci , and converted into a rotation and translation matrix, that is, the camera external parameter matrix H is obtained ci , i = 1, 2, ..., n, calculate the rotation vector R relative to the base according to the robot posture information gi and the translation vector T gi , get the robot pose matrix H gi ,i=1,2,...,n, where:

[0099]

[0100] Since any two transformation shooting postures u, v in n sets of shooting postures, where u, v = 1, 2, ..., n, and u ≠ v, according to the relative posture relationship H between the robot base and the calibration plate BW The relative position relationship between the camera and the robot end flange is the hand-eye relationship matrix H cg Fixed, combined with the coordinate transformation relationship between the camera extrinsic matrix and the robot posture matrix, the following relationship can be obtained:

[0101] H gu H cg H cu =H BW =H BW =H gv H cg H cv

[0102] Right now:

[0103] H gu Hcg H cu =H gv H cg H cv

[0104] Make a matrix change, that is, multiply the left and right sides by the matrix You can get:

[0105]

[0106] That is, the hand-eye calibration equation:

[0107]

[0108] in:

[0109]

[0110] Among them, R gu,v Representation Matrix The rotation matrix in , T gu,v Representation Matrix The translation vector in R cg Represents the hand-eye relationship matrix H cg The rotation matrix in , T cg represents the translation vector in the hand-eye relationship matrix, R cu,v Representation Matrix The rotation matrix, T cu,v Representation Matrix The translation vector in .

[0111] make X=H cg , then the hand-eye calibration equation can be expressed as:

[0112] AX=XB

[0113] From n sets of calibration data, we can establish The hand-eye calibration equation.

[0114] Step S2: Use Kronecker product and Moore-Penrose inverse to transform the hand-eye calibration equation into a least squares problem and use singular value decomposition to calculate the hand-eye relationship matrix H cg

[0115] Based on the properties of the Kronecker product, the hand-eye calibration equation AX = XB to be solved is expanded to obtain the homogeneous equation:

[0116] AX=XB

[0117]

[0118]

[0119]

[0120] That is, formula 1 and formula 2:

[0121]

[0122] Among them, R cu,v ,R cg ,R gu,v They are all rotation matrices, belonging to the special orthogonal group SO (3) , is closed under multiplication.

[0123] For formulas 1 and 2, we use the properties of matrix direct product to transform the matrix equations into a system of linear algebraic equations, and then solve them to obtain the following two formulas:

[0124]

[0125] Therefore, the above equations can be constructed into the following matrix equation form:

[0126]

[0127] According to the properties of the contradictory system of equations, the coefficient matrix on the left side of the system of equations is an irreversible matrix. The idea of ​​generalized inverse will be used to solve the solution of the contradictory system of equations. Therefore, it is necessary to find the minimum norm least squares solution of the system of equations. The minimum norm least squares solution of the system of equations is equal to C + d. Coupling the rotation matrix with the translation vector will reduce the influence of system noise and calculation errors during the calculation process. The specific steps are as follows.

[0128] The original matrix equation is converted into a linear least squares problem:

[0129]

[0130]

[0131] By SVD decomposition, we can get According to the orthogonality of matrices U and V, we can get:

[0132]

[0133] Re-order Substitute into the above formula and expand the matrix to get:

[0134]

[0135] Among them, σ i ,z i ,u iis the vector part of each matrix. Therefore, the original problem can be summarized as:

[0136]

[0137] Obviously, when When the problem achieves the minimum value, the least squares solution and the minimum value are:

[0138]

[0139]

[0140] Depend on Available Therefore, the formula for solving the linear least squares solution through SVD is:

[0141]

[0142] Therefore, the hand-eye relationship matrix H is calculated in the present invention cg for:

[0143] Set up a system of linear equations:

[0144]

[0145] Where I is the identity matrix, represents Kronecker product, vec represents vectorized operation;

[0146] Transform all two shots under the pose u, v matrix:

[0147]

[0148] Place them by columns to get a matrix R;

[0149] Perform singular value decomposition on the matrix R to obtain the U matrix, i.e. the left singular matrix, the singular value matrix ∑, and the V matrix, i.e. the right singular matrix;

[0150] Transform all two shots under the pose u, v matrix:

[0151]

[0152] Place them by column and get a matrix T g ;

[0153] Take the column vector V∑ -1 U T T g The first 9 elements of are reduced to a 3×3 matrix form, and the matrix

[0154] Then, for the matrix Orthogonalized singular value decomposition is Among them, U R is a right singular matrix, ∑ R is the singular value matrix, V R is the left singular matrix, and the hand-eye relationship matrix H is obtained cg The rotation matrix R cg Right now

[0155] The matrix (R gu,v -I) Place them by columns to get a matrix R g , the matrix (R cg T cu,v -T gu,v ) are placed column by column to obtain a matrix T,

[0156] According to the rotation matrix R cg Calculate the hand-eye relationship matrix H cg The translation vector T cg :

[0157] T cg =R g -1 T

[0158] In this way, the rotation and translation matrix of the camera coordinate system relative to the robot end flange coordinate system is obtained, that is, the hand-eye relationship matrix H cg , and use this to initialize and get the 0th generation population of the DE algorithm.

[0159] Step S3: Calculate the hand-eye relationship matrix H cg Correction is performed to obtain the hand-eye relationship matrix for coarse registration of multi-view point clouds

[0160] Step S3.1: Obtain the corresponding point sets between the real coordinates of the J sets of corner points and the point cloud coordinates

[0161] Step S3.1.1: Fix the probe device on the end flange of the robot, establish the tool coordinates through the five-point method and the teaching pendant, and obtain the position coordinates ΔP of the probe end relative to the robot end flange coordinate system 0 The robot drives the probe to move to the top of the four corner points of the first square in the upper left corner of the calibration plate, and records the coordinates of the robot end flange relative to the base coordinate system from the teach pendant. The probe is continuously driven to move to the top of the four corner points of the square on the calibration plate, and the coordinates of the robot end flange relative to the base coordinate system at the corner points are obtained. And calculate the true coordinates of all corner points of the calibration plate:

[0162]

[0163] in, is the three-dimensional coordinate of the robot end flange directly above the lth corner point relative to the base coordinate system, Δx 0 ,Δy 0 ,Δz 0 is the position coordinate of the probe end relative to the robot end flange coordinate system, L is the number of corner points on the calibration plate, T represents transposition, and B represents the robot base coordinate system;

[0164] Step S3.1.2: Shoot and reconstruct the three-dimensional point cloud of the calibration plate at a certain robot posture, reconstruct the three-dimensional point cloud of the calibration plate, click and obtain the point cloud coordinates of the four corner points of the first square in the upper left corner of the calibration plate relative to the binocular camera coordinate system in sequence, and calculate the point cloud coordinates P of all the corner points of the calibration plate corresponding to the real coordinates of the corner points. 1 l :

[0165]

[0166] in, is the point cloud coordinate P 1 l The three-dimensional coordinates of are represented by, and C represents the binocular camera coordinate system;

[0167] Step S3.1.3: According to the robot posture information, the robot posture matrix when shooting the calibration plate point cloud is obtained as H g , from the known real coordinates of the corner point P 0 l , get the corner point coordinates in the flange coordinate system converted to the robot posture

[0168]

[0169] in, is the corner point coordinate The three-dimensional coordinates of G represent the robot end flange coordinate system;

[0170] This gives us a set of corresponding points

[0171] Step S3.1.4: The set is rewritten as In this way, we get the set P of corresponding points between the real coordinates of the J groups of corner points and the point cloud coordinates;

[0172] Step S3.2: Optimize the hand-eye relationship matrix H based on minimizing the Euclidean transformation error cg

[0173] Step S3.2.1: Take the hand-eye relationship matrix H cg Initialize the population

[0174] Construct a population of size M and initialize each individual x h for:

[0175]

[0176] Where h represents the individual number, h = 1, 2, ... M, 0 represents the 0th generation population, and the subscript number is the component number in the individual, [r ox ,r oy ,r oz ] is the hand-eye relationship matrix H cg The rotation matrix R cg The corresponding rotation vector, [x o ,y o ,z o ] is the hand-eye relationship matrix H cg The translation vector T in cg The transpose of , that is:

[0177]

[0178] Among them, Rodrigues() represents the rotation vector to rotation matrix operation;

[0179] rand(0,1) means to take a random number between 0 and 1;

[0180] r xmaxmin ,r ymaxmin ,r zmaxmin ,x maxmin ,y maxmin ,z maxmin They respectively represent the difference between the maximum and minimum values ​​of the rotation vectors x, y, z in the hand-eye relationship matrix and the translation vectors x, y, z in the hand-eye relationship matrix;

[0181] Step S3.2.2: Individual variation and crossover

[0182] Perform mutation and crossover on individuals in a population:

[0183]

[0184]

[0185] in, is the kth component of the hth individual in the tth generation population, Mutation() is the mutation operation, is the kth component of the hth mutant individual of the tth generation, Crossover() is the crossover operation, is the kth component of the hth experimental individual of the tth generation, k = 1, 2, …, 6;

[0186] Step S3.2.3: Use the greedy algorithm to select individuals x to enter the next generation population h (t+1):

[0187]

[0188] in:

[0189]

[0190]

[0191]

[0192]

[0193] in, are the hth individual x in the tth generation population h (t) The corresponding rotation matrix:

[0194]

[0195]

[0196] in, are the hth individual u in the tth generation population h (t) The corresponding rotation matrix:

[0197]

[0198]

[0199] Step S3.2.4: t=t+1, return to step S3.2.2 until the set termination condition is reached;

[0200] Step S3.2.5: Compare M individuals x h (t+1),h=1,2,…,the fitness function value f(x h+1 (t),P), and the individual corresponding to the minimum fitness function value is recorded as x′(t)={x′ 1 (t),x′ 2 (t),x′ 3 (t),x′ 4 (t),x′ 5 (t),x′ 6 (t)}, then the corrected hand-eye calibration matrix for:

[0201] R′ cg =Rodrigues([x′ 1 (t),x′ 2 (t),x′ 3 (t)]), T′ cg =[x′ 4 (t),x′ 5 (t),x′ 6 (t)] T

[0202] Step S4: 3D shape measurement

[0203] Debug the binocular camera so that it can clearly capture the measurement object, i.e., the large and complex parts to be measured, so that the left and right cameras are kept at the same horizontal position and a certain distance is maintained, and binocular positioning is completed; debug the robot so that it can carry the binocular camera to perform three-dimensional measurement tasks, ensuring that the full picture of the measurement object is captured;

[0204] Structural light images are taken at each measurement position, and the unit measurement point cloud data of the measured object is reconstructed. Record the measured posture and calculate the posture matrix of the robot Where m is the number of measurement poses for three-dimensional shape measurement of the measured object.

[0205] According to the obtained unit point cloud data Each view pose matrix And the optimized hand-eye relationship matrix obtained in step (4) Calculate the rigid body transformation matrix of each unit point cloud data from the binocular camera coordinate system to the robot base coordinate system:

[0206]

[0207] The unit measurement point cloud data obtained by the binocular camera is converted to the unified coordinate system of the robot base to obtain the single-view point cloud under the qth measurement pose after registration

[0208]

[0209] Realize the registration of multi-view point cloud data, thereby completing the 3D shape measurement of large and complex parts.

[0210] By controlling the robot's movement and carrying the binocular camera, the unit measurement point cloud information of the parts to be measured is collected at multiple viewing angles and postures; based on the optimized hand-eye matrix, multiple point clouds are converted to the same coordinate system to complete the multi-view point cloud registration function, realizing the three-dimensional shape measurement of the object to be measured, so as to improve the efficiency of three-dimensional shape measurement of large and complex parts.

[0211] Example

[0212] In an embodiment, a large-scale aerospace component is used as a specimen to perform three-dimensional shape measurement. This embodiment obtains three-dimensional point cloud data of the specimen except for the bottom, thereby restoring the shape characteristics of the specimen.

[0213] The test specimen used in the embodiment is a 1600mm×1525mm rectangular plate, and the calibration plate used is a 8×11, 15mm standard black and white chessboard calibration plate, such as Figure 2 shown.

[0214] In this embodiment, the robot is first controlled to drive the binocular camera to collect 9 calibration plate images with a pixel resolution of 3000×4096 at different shooting positions, and then the homogeneous expression H of the camera extrinsic parameter matrix of the 9 calibration plate images is solved by Zhang's calibration method. c ; and record the robot posture information of the corresponding shooting position, and similarly calculate the second expression H of its robot posture matrix g .

[0215] Then, the hand-eye equation to be solved is transformed into a system of linear algebraic equations using the properties of Kronecker product and Moore-Penrose inverse. Then, the rotation and translation matrix of the camera coordinate system relative to the robot end flange coordinate system, i.e., the hand-eye relationship matrix H, is solved by the least squares method. cg .

[0216] The initial population of the differential evolution (DE) algorithm is generated according to the hand-eye calibration results; in this example, the initial population is:

[0217]

[0218] In this embodiment, M=200,

[0219] Get the corresponding point set between the real coordinates of the J groups of corner points and the point cloud coordinates, such as Figure 3 Shown

[0220] Corrected hand-eye calibration matrix for:

[0221]

[0222] like Figure 4 As shown, the hand-eye relationship matrix H cgAfter correction by the DE algorithm, the position of the corresponding calibration plate origin in the robot base coordinate system is also calculated. Comparing the distribution of the calibration plate origins under n shooting poses before and after optimization, it can be seen that the calibration plate origins (solid origins) after optimization are more concentrated and basically overlapped, while the calibration plate origins (hollow origins) before optimization are more scattered. This result proves the improvement of the hand-eye calibration results, and its effect will be verified in the morphology measurement.

[0223] For the test piece to be measured in this embodiment, it is necessary to collect unit measurement point cloud data from multiple positions. During the collection process, ensure that the test piece is photographed in a complete range. The mobile robot drives the binocular camera to complete the structured light image collection of the left and right cameras of the measurement object in multiple viewing angles and positions, and reconstruct the unit measurement point cloud data. like Figure 5 As shown in the figure, the circled part is the reconstruction result of the measurement point cloud of a unit. And record the robot posture information of each view to obtain the posture matrix

[0224] According to the obtained unit point cloud data And each view pose matrix And the corrected hand-eye relationship matrix Then calculate the rigid body transformation matrix of each unit measurement point cloud data from the binocular camera coordinate system to the robot base coordinate system The unit measurement point cloud obtained by the binocular camera is converted into a unified coordinate system, such as Figure 6 The figure shows a schematic diagram of converting multiple unit measurement point cloud data into the same coordinate system. It can be seen that multiple unit measurement point cloud data all achieve rigid body transformation, and the overall point cloud remains in its original state during the transformation process without distortion.

[0225] In this example, the three-dimensional morphology measurement results of large aerospace components as the test pieces are as follows: Figure 7 As shown. Figure 7 It can be seen that the three-dimensional morphology measurement results of the specimen are complete, and the registration between multiple unit measurement point cloud data is good.

[0226] Although the above describes an illustrative specific embodiment of the present invention to facilitate the understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiment. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

Claims

1. A robot automatic calibration and three-dimensional measurement method for large aerospace components, characterized in that: include: (1) Establish the hand-eye calibration equation The binocular camera is installed on the end flange of the robot, and the robot is controlled to move to the i-th shooting posture. A corner point of the chessboard calibration plate is photographed, and the calibration plate image is obtained at this shooting posture. At the same time, the robot posture information and the position P of the corner point in the camera coordinate system are recorded. i , in this way, n shooting postures are photographed to obtain n sets of calibration plate images and corresponding robot posture information; According to Zhang's calibration method, the rotation vector R of the calibration plate relative to the camera in each set of calibration plate images is obtained. ci and the translation vector T ci , and converted into a rotation and translation matrix, that is, the camera external parameter matrix H is obtained ci , i = 1, 2, ..., n, calculate the rotation vector R relative to the base according to the robot posture information gi and the translation vector T gi , get the robot pose matrix H gi ,i=1,2,...,n, where: For any two transformed shooting poses u and v, the camera extrinsic matrix and robot pose matrix are used to establish the hand-eye calibration equation: in: Among them, R gu,v Representation Matrix The rotation matrix in , T gu,v Representation Matrix The translation vector in R cg Represents the hand-eye relationship matrix H cg The rotation matrix in , T cg represents the translation vector in the hand-eye relationship matrix, R cu,v Representation Matrix The rotation matrix, T cu,v Representation Matrix The translation vector in ; (2) Use Kronecker product and Moore-Penrose inverse to transform the hand-eye calibration equation into a least squares problem and use singular value decomposition to calculate the hand-eye relationship matrix H cg Set up a system of linear equations: Where I is the identity matrix, represents Kronecker product, vec represents vectorized operation; Transform all two shots under the pose u, v matrix: Place them by columns to get a matrix R; Perform singular value decomposition on the matrix R to obtain the U matrix, i.e. the left singular matrix, the singular value matrix ∑, and the V matrix, i.e. the right singular matrix; Transform all two shots under the pose u, v matrix: Place them by column and get a matrix T g ; Take the column vector V∑ -1 U T T g The first 9 elements of are reduced to a 3×3 matrix form, and the matrix Then, for the matrix Orthogonalized singular value decomposition is Among them, U R is a right singular matrix, ∑ R is the singular value matrix, V R is the left singular matrix, and the hand-eye relationship matrix H is obtained cg The rotation matrix R cg Right now The matrix (R gu,v -I) Place them by columns to get a matrix R g , the matrix (R cg T cu,v -T gu,v ) are placed column by column to obtain a matrix T, According to the rotation matrix R cg Calculate the hand-eye relationship matrix H cg The translation vector T cg : T cg =R g -1 T (3) The calculated hand-eye relationship matrix H cg Correction is performed to obtain the hand-eye relationship matrix for coarse registration of multi-view point clouds 3.1) Obtain the corresponding point set between the real coordinates of the J groups of corner points and the point cloud coordinates 3.1.1) Fix the probe device on the end flange of the robot, establish the tool coordinates through the five-point method and the teach pendant, and obtain the position coordinate ΔP0 of the probe end relative to the robot end flange coordinate system. The robot drives the probe to move to the top of the four corner points of the first square in the upper left corner of the calibration plate, and record the coordinates of the robot end flange relative to the base coordinate system from the teach pendant. Continue to drive the probe to move to the top of the four corner points of the square on the calibration plate to obtain the coordinates of the robot end flange relative to the base coordinate system at the top of the corner points. And calculate the true coordinates of all corner points of the calibration plate: in, is the three-dimensional coordinate of the robot end flange directly above the lth corner point relative to the base coordinate system, Δx0, Δy0, Δz0 are the position coordinates of the probe end relative to the robot end flange coordinate system, L is the number of corner points on the calibration plate, T represents transposition, and B represents the robot base coordinate system; 3.1.2) Shoot and reconstruct the calibration plate 3D point cloud at a certain robot posture, reconstruct the calibration plate 3D point cloud, click and obtain the point cloud coordinates of the four corner points of the first square in the upper left corner of the calibration plate relative to the binocular camera coordinate system in sequence, and calculate the point cloud coordinates of all the calibration plate corner points corresponding to the real coordinates of the corner points. in, Point cloud coordinates The three-dimensional coordinates of are represented by, and C represents the coordinate system of the binocular camera; 3.1.3) According to the robot posture information, the robot posture matrix when shooting the calibration plate point cloud is obtained as H g , from the known real coordinates of the corner points Get the corner coordinates in the flange coordinate system converted to the robot posture in, is the corner point coordinate The three-dimensional coordinates of G represent the robot end flange coordinate system; This gives us a set of corresponding points 3.1.4) Change the robot posture J-1 times, repeat steps 3.1.2) and 3.1.3), and get the jth set of corresponding points, which is re-recorded as In this way, we get the set P of corresponding points between the real coordinates of the J groups of corner points and the point cloud coordinates; 3.2) Optimize the hand-eye relationship matrix H based on minimizing the Euclidean transformation error cg 3.2.1) Hand-eye relationship matrix H cg Initialize the population Construct a population of size M and initialize each individual x h for: in, h represents the individual number, h = 1, 2, ... M, 0 represents the 0th generation population, and the subscript number is the component number in the individual, [r ox ,r oy ,r oz ] is the hand-eye relationship matrix H cg The rotation matrix R cg The corresponding rotation vector, [x o ,y o ,z o ] is the hand-eye relationship matrix H cg The translation vector T in cg The transpose of , that is: Among them, Rodrigues() represents the rotation vector to rotation matrix operation; rand(0,1) means to take a random number between 0 and 1; r xmaxmin ,r ymaxmin ,r zmaxmin ,x maxmin ,y maxmin ,z maxmin They respectively represent the difference between the maximum and minimum values ​​of the rotation vectors x, y, z in the hand-eye relationship matrix and the translation vectors x, y, z in the hand-eye relationship matrix; 3.2.2) Individual variation and crossover Perform mutation and crossover on individuals in a population: in, is the kth component of the hth individual in the tth generation population, Mutation() is the mutation operation, is the kth component of the hth mutant individual of the tth generation, Crossover() is the crossover operation, is the kth component of the hth experimental individual of the tth generation, k = 1, 2, …, 6; 3.2.3) Use the greedy algorithm to select individuals x to enter the next generation population h (t+1): in: in, are the hth individual x in the tth generation population h (t) The corresponding rotation matrix: in, are the hth individual u in the tth generation population h (t) The corresponding rotation matrix: 3.2.4) t = t + 1, return to step 3.2.2) until the set termination condition is reached; 3.2.5) Compare M individuals x h (t+1),h=1,2,…,the fitness function value f(x h+1 (t), P), the individual corresponding to the minimum fitness function value is recorded as x′(t) = {x′1(t), x′2(t), x′3(t), x′4(t), x′5(t), x′6(t)}, then the corrected hand-eye calibration matrix for: (4) Three-dimensional shape measurement Debug the binocular camera so that it can clearly capture the measurement object, i.e., the large aerospace component to be measured, so that the left and right cameras are kept at the same horizontal position and a certain distance is maintained, and binocular positioning is completed; debug the robot so that it can carry the binocular camera to perform three-dimensional measurement tasks, ensuring that the full picture of the measurement object is captured; Structural light images are taken at each measurement position, and the unit measurement point cloud data of the measured object is reconstructed. Record the measured posture and calculate the posture matrix of the robot in, m is the number of measurement poses for three-dimensional shape measurement of the measured object Measure point cloud data based on the obtained units Each view pose matrix And the optimized hand-eye relationship matrix obtained in step (3) Calculate the rigid body transformation matrix of each unit measurement point cloud data from the binocular camera coordinate system to the robot base coordinate system: The unit measurement point cloud data obtained by the binocular camera is converted to the unified coordinate system of the robot base to obtain the single-view point cloud under the qth measurement pose after registration Realize the registration of multi-view point cloud data, thereby completing the three-dimensional shape measurement of large aerospace components.

Images