A robot-assisted multi-view three-dimensional scanning measurement method

Abstract

The application discloses a kind of robot-assisted multi-view three-dimensional scanning measurement methods, first, acquisition checkerboard calibration plate image, obtain n group camera information and corresponding shooting position robot posture information, then based on Kronecker product without special mathematical tool and effectively solve linear equation closed solution, reduce the error influence caused by system noise, calculation in calibration process, complete preliminary hand-eye calibration;Then with hand-eye relationship matrix as initial value, according to photogrammetry bundle adjustment as optimization model, establish minimization re-projection error as cost function, introduce particle swarm algorithm iteration to hand-eye relationship matrix optimization, further improve the hand-eye calibration precision, ensure the registration quality of point cloud image;Finally, by controlling robot motion carries binocular camera under multiple measurement poses Acquisition large complex parts unit measurement point cloud information, based on the optimized hand-eye matrix conversion to same coordinate system, complete multi-view point cloud registration function, to improve the three-dimensional topography measurement efficiency of large complex parts.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of three-dimensional topography measurement, and more particularly relates to a robot-assisted multi-view three-dimensional scanning measurement method. BACKGROUND

[0002] With the pace of modernization development of mechanical manufacturing industry, higher requirements are put forward for the size of complex parts. The traditional measurement method adopts three-coordinate measuring machine or laser scanner and other three-dimensional measurement equipment, which is not only high in cost but also low in measurement efficiency. For large and complex parts, the space of some parts is limited, so that the field of view of the detection equipment is difficult to reach, so that the topography measurement cannot be completed.

[0003] The three-dimensional reconstruction technology based on industrial robot uses industrial robot as a moving carrier, determines the pose relationship of the unit measurement coordinate system of each view through the constraint between the joint coordinate systems of each axis of the robot, so that the measurement system retains the characteristics of non-contact and rapid of visual measurement technology, and the flexible characteristics of the robot enhance the flexibility of the whole measurement system.

[0004] At present, in the related technology of industrial robot, the hand-eye calibration is performed by collecting and recording the camera information and the robot posture information of the robot in different motion postures, establishing a hand-eye calibration equation, and obtaining a hand-eye relationship matrix according to a linear solution closed solution. However, the above method only considers minimizing the algebraic error to calculate the hand-eye relationship matrix to complete the hand-eye calibration, and does not consider the influence of image noise on the hand-eye calibration accuracy, which leads to low calibration accuracy of the pose relationship obtained by linear solution, thereby affecting the accuracy and integrity of three-dimensional topography measurement. SUMMARY

[0005] The present application aims to overcome the shortcomings of the prior art and provide a robot-assisted multi-view three-dimensional scanning measurement method to optimize the hand-eye form precision, ensure the registration accuracy of multi-view point cloud data, and improve the accuracy and integrity of three-dimensional topography measurement.

[0006] To achieve the above-mentioned purpose of the application, the robot-assisted multi-view three-dimensional scanning measurement method of the present application has the following characteristics:

[0007] (1) Establishing a hand-eye calibration equation

[0008] The binocular camera is installed at the end flange of the robot, the robot is controlled to move to the i-th shooting pose, and a corner point on the checkerboard calibration plate is shot. The calibration plate picture is obtained at the shooting pose, and the robot pose information and the position P of the corner point in the camera coordinate system are recorded. i In this way, n shooting poses are shot to obtain n sets of calibration plate pictures and corresponding robot posture information.

[0009] According to Zhang's calibration method, the rotation vector R of the calibration board relative to the camera in each group of calibration board pictures is obtained ci and the translation vector T ci , and is converted into a rotation translation matrix, that is, the camera extrinsic matrix H ci , i = 1, 2,..., n, the rotation vector R of the robot relative to the base is calculated according to the robot pose information gi and the translation vector T gi , the robot pose matrix H gi , i = 1, 2,..., n, wherein:

[0010]

[0011] For any two times of transformation, the camera extrinsic matrix and the robot pose matrix between the shooting poses u and v are established to establish a hand-eye calibration equation:

[0012]

[0013] Wherein:

[0014]

[0015] Wherein, wherein, R gu,v represents the rotation matrix in the matrix , T gu,v represents the translation vector in the matrix , R cg represents the rotation matrix in the hand-eye relationship matrix H cg , T cg represents the translation vector in the hand-eye relationship matrix, R cu，v represents the rotation matrix of the matrix , T cu，v represents the translation vector in the matrix ;

[0016] (2) The hand-eye calibration equation is converted into a least squares problem by using Kronecker product, and the hand-eye relationship matrix H cg

[0017] A linear equation group is established: Wherein, I is a unit matrix, represents the Kronecker product, and vec represents the vectorization operation;

[0018] The matrices under all two times of transformation and shooting poses u and v are placed by columns to obtain a matrix R;

[0019] The singular value decomposition is performed on the matrix R, and the last column of the obtained V matrix, i.e. the right singular matrix, is taken out and restored into a 3x3 matrix form to obtain a matrix

[0020] The matrix is subjected to orthogonal singular value decomposition, i.e. wherein U R is a right singular matrix, ∑ R is a singular value matrix, and V R is a left singular matrix, to obtain a rotation matrix R cg of the hand-eye relationship matrix H cg .

[0021] The matrix (R gu，v -I) at the shooting poses u and v after the two transformations is placed column by column to obtain a matrix R g , and the matrix (R cg T cu，v -T gu，v ) at the shooting poses u and v after the two transformations is placed column by column to obtain a matrix T,

[0022] The translation vector T cg of the hand-eye relationship matrix H cg is calculated according to the rotation matrix R cg .

[0023]

[0024] (3) Optimizing the hand-eye relationship matrix based on minimizing the reprojection error

[0025] 3.1) Construct a particle swarm with a population size of K, and the position of each particle is p k and the velocity is v k , and then initialize the position p k and the velocity v k of the K particles:

[0026] p k = H cg

[0027] v k = rand(v min , v max )

[0028] wherein v min , v max represent the upper and lower thresholds of the velocity set at each iteration of the particle, and rand is a random value operation, so that the initial value of the position of each particle is the hand-eye relationship matrix H cgeach particle velocity is randomly valued, the population size of the particle swarm is K;

[0029] 3.2) Establish a re-projection error cost function as the fitness function of the particle swarm optimization algorithm:

[0030]

[0031] wherein s i is a scale factor, K is the camera intrinsic matrix, exp[·] 3×4 represents the 3x4 sub-matrix in the upper left corner of the matrix in the parentheses, ||||2 represents the two-norm operation, P j is the position of the corner point in the camera coordinate system for any one shot of the i-th shot pose;

[0032] 3.3) According to the calculated fitness function value of the particle find the current individual extremum position of each particle and update it as the historical optimal position of the particle and the group extremum position, that is, the minimum particle position of the fitness function, and update it as the global optimal position g * :

[0033]

[0034]

[0035] 3.4) Update the velocity v k and position p k of each particle:

[0036]

[0037] p k = p k + v k

[0038] wherein ω is an inertia factor, c1 and c2 are acceleration constants, and then return to step 3.3) until the set termination condition is reached;

[0039] 3.5) Take the global optimal position g * as the optimized hand-eye relationship matrix

[0040] (4) Three-dimensional topography measurement

[0041] The binocular camera is debugged to clearly capture the measuring object, i.e. a large and complex part to be measured, so that the left and right cameras are kept in the same horizontal position and are kept at a certain distance, and the double target positioning is completed; the robot is debugged to carry the binocular camera to perform a three-dimensional measurement task, and to ensure that the full appearance of the measuring object is captured;

[0042] The structure light image is captured at each measuring position, and the unit measuring point cloud data of the measuring object is reconstructed The measuring position is recorded, and the position matrix of the robot at the measuring position is calculated Wherein m is the number of measuring positions for three-dimensional appearance measurement of the measuring object, and the unit measuring point cloud data is obtained according to the obtained unit measuring point cloud data Each visual angle position matrix And the optimized hand-eye relationship matrix obtained in step (4) The rigid body transformation matrix of each unit measuring point cloud data from the binocular camera coordinate system to the robot base coordinate system is calculated:

[0043]

[0044] The unit measuring point cloud data obtained by the binocular camera is converted to the robot base unified coordinate system, and the registered single visual angle point cloud at the qth measuring position is obtained

[0045]

[0046] The registration of the multi-view point cloud data is realized, and thus the three-dimensional appearance measurement of the large and complex part is completed.

[0047] The application object of the application is achieved in the following manner:

[0048] The robot-assisted multi-view three-dimensional scanning measurement method of the application first collects the checkerboard calibration plate image, obtains n groups of camera information and the robot posture information corresponding to the shooting position, then effectively solves the linear equation closed solution based on the Kronecker product without special mathematical tools, reduces the error influence caused by system noise and calculation in the calibration process, and completes the preliminary hand-eye calibration; then, the hand-eye relationship matrix calculated by the singular value decomposition based on the Kronecker product parameterization method is used as the initial value, the photogrammetry bundle adjustment is used as the optimization model, the re-projection error minimization is used as the cost function, the particle swarm (PSO) algorithm is introduced to iteratively update the best particle, and the hand-eye relationship matrix is optimized, so that the hand-eye calibration accuracy is further improved, and the point cloud image registration quality is ensured; finally, the robot motion is controlled to carry the binocular camera to collect the unit measurement point cloud information of the complex part to be measured at multiple view positions; the multiple point clouds are converted to the same coordinate system based on the optimized hand-eye matrix, the multi-view point cloud registration function is completed, the three-dimensional appearance measurement of the measured object is realized, and the three-dimensional appearance measurement efficiency of the large and complex part is improved. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a flow chart of one specific embodiment of the robot-assisted multi-view three-dimensional scanning measurement method of the application;

[0050] Figure 2 is a schematic diagram of the i, jth shooting position transformation relationship;

[0051] Figure 3 is a comparison diagram of the hand-eye calibration results before and after optimization;

[0052] Figure 4 is a schematic diagram of a unit measurement point cloud data;

[0053] Figure 5 is a schematic diagram of converting multiple unit measurement point cloud data to a unified coordinate system;

[0054] Figure 6 is a three-dimensional appearance measurement result diagram. DETAILED DESCRIPTION

[0055] The specific embodiments of the application will be described below with reference to the accompanying drawings, so that those skilled in the art can better understand the application. It should be particularly noted that in the following description, when the detailed description of the known functions and designs may obscure the main content of the application, these descriptions will be omitted here.

[0056] Figure 1 is a flow chart of one specific embodiment of the robot-assisted multi-view three-dimensional scanning measurement method of the application.

[0057] In this embodiment, asFigure 1 As shown, the robot-assisted multi-view 3D scanning measurement method of the present invention includes:

[0058] Step S1: Establish hand-eye calibration equation

[0059] A binocular camera is mounted on the end flange of the robot. The robot is controlled to move to the i-th shooting pose and take a picture of a corner point on the chessboard calibration board. An image of the calibration board is obtained at this shooting pose. At the same time, the robot's pose information and the position P of the corner point in the camera coordinate system are recorded. i By taking pictures of n different shooting positions, n sets of calibration board images and corresponding robot posture information are obtained.

[0060] The rotation vector R of the calibration board relative to the camera in each set of calibration board images is obtained using Zhang's calibration method. ci Translation vector T ci This is then converted into a rotation and translation matrix, thus obtaining the camera extrinsic parameter matrix H. ci For i = 1, 2, ..., n, calculate the rotation vector R relative to the base based on the robot's pose information. gi Translation vector T gi The robot pose matrix H is obtained. gi ,i=1,2,...,n, where:

[0061]

[0062] Since between any two pose changes u,v in the n sets of shooting poses, where u,v=1,2,...,n and u≠v, according to the relative pose relationship H between the robot base and the calibration plate... BW The relative pose relationship between the camera and the robot's end effector flange, i.e., the hand-eye relationship matrix H, remains constant. cg By keeping these parameters constant and combining the coordinate transformation relationship between the camera extrinsic matrix and the robot pose matrix, we can obtain the following relationship:

[0063] H gu H cg H cu =H BW =H BW =H gv H cg H cv

[0064] Right now:

[0065] H gu H cg H cu =H gv H cg H cv

[0066] Perform a matrix transformation, that is, multiply both sides by the matrix before and after. We can obtain:

[0067]

[0068] That is, the hand-eye calibration equation:

[0069]

[0070] in:

[0071]

[0072] Among them, R gu,v Representation matrix The rotation matrix T in gu v represents a matrix The translation vector in R cg Represents the hand-eye relationship matrix H cg The rotation matrix T in cg R represents the translation vector in the hand-eye relationship matrix. cu,v Representation matrix The rotation matrix, T cu,v Representation matrix Translation vector in;

[0073] make The hand-eye calibration equation can then be expressed as:

[0074] AX = XB

[0075] A system can be established from n sets of calibration data. The equation is calibrated by hand and eye.

[0076] Step S2: Transform the hand-eye calibration equation into a least squares problem using the Kronecker product and calculate the hand-eye relationship matrix H using singular value decomposition. cg

[0077] Expanding the hand-eye calibration equation AX = XB based on the properties of the Kronecker product, we obtain the homogeneous equation:

[0078] AX = XB

[0079]

[0080]

[0081] That is, Formula 1 and Formula 2:

[0082]

[0083] Among them, Rcu,v ,R cg ,R gu,v They are all rotation matrices, belonging to the special orthogonal group SO. (3) It is closed in multiplication.

[0084] Convert and calculate Formula 1:

[0085] R gij R cg I = IR cg R cij

[0086] vec(R gu.v R cg I)=vec(IR cg R cu,v )

[0087]

[0088]

[0089] have to:

[0090]

[0091] Where I is the identity matrix, The Kronecker product is represented by vec, which represents the vectorization operation.

[0092] Thus, we obtain a system of linear equations of the form Cx = 0. This system can be transformed into a least squares problem using singular value decomposition (SVD), since:

[0093]

[0094] The following least squares problem is obtained:

[0095]

[0096]

[0097] have to:

[0098]

[0099] st||vec(R cg )||2=1

[0100] At this point, the original problem becomes in:

[0101]

[0102] And due to the constraint ||y||² = 1, therefore y = [0 0...1] T The minimum solution, i.e., the solution where Cx = 0, is the last column of V:

[0103]

[0104] Therefore, in this invention, the hand-eye relationship matrix H is calculated. cg for:

[0105] First, establish a system of linear equations: Where I is the identity matrix, The Kronecker product is represented by vec, which represents the vectorization operation.

[0106] Then, combine the matrices under both transformations of the shooting pose u,v Arrange the columns to obtain a matrix R, and then use singular value decomposition to calculate the hand-eye relationship matrix H. cg :

[0107] Then, singular value decomposition is performed on matrix R. The last 9 elements of the resulting V matrix (the right singular matrix) are extracted and restored to a 3×3 matrix form, yielding the matrix...

[0108] Then, for the matrix Perform orthogonalized singular value decomposition, i.e. Among them, U R For a right singular matrix, ∑ R V is a singular value matrix. R Given a left singular matrix, we obtain the hand-eye relationship matrix H. cg rotation matrix R cg Right now

[0109] Therefore, the rotation matrix R is calculated using Equation 1. cg To ensure the identity orthogonality of the rotation matrix, the rotation part of the solved hand-eye matrix needs to be orthogonalized using the Rodrigues formula to eliminate the influence of measurement noise; where ∑ R To a certain extent, it can reflect the quality of the calibration results; a good calibration... R Their diagonal elements should be very close, or even completely equal.

[0110] Finally, using Formula 2 above, we can obtain:

[0111] R gu,v T cg +T gu,v =R cg T cu,v +T cg

[0112] (R gu,v -I)T cg =R cg T cu,v -T gu,v

[0113] The matrix (R) under both transformed shooting poses u,v gu,v -I) Arrange the columns to obtain a matrix R g The matrix (R) under all two transformed shooting poses u,v cg T cu,v -T gu,v Arrange them column-wise to obtain a matrix T.

[0114] According to the rotation matrix R cg The hand-eye relationship matrix H was calculated. cg Translation vector T cg :

[0115] T cg =R g -1 T

[0116] Thus, the rotation and translation matrix of the camera coordinate system relative to the robot's end effector flange coordinate system, i.e., the hand-eye relationship matrix H, can be obtained. cg This is used as the initial particle position for the PSO algorithm.

[0117] Step S3: Optimize the hand-eye relationship matrix based on minimizing reprojection error

[0118] Step S3.1: Construct a particle swarm and use the hand-eye relationship matrix H cg As the initial position of all particles

[0119] Construct a particle swarm with a population size of K, where each particle is at position p. k and velocity v k Then initialize the positions p of the K particles. k and speed v k :

[0120] p k =H cg

[0121] v k =rand(v min ,v max )

[0122] Among them, v min ,v max This represents the upper and lower threshold values ​​for the particle's velocity set in each iteration, with `rand` representing a random value selection operation. This results in the initial position of each particle being the hand-eye relationship matrix H.cg A swarm of particles, where the velocity of each particle is randomly assigned, has a population size of K.

[0123] Step S3.2: Establish the reprojection error cost function as the fitness function of the particle swarm optimization algorithm:

[0124]

[0125] Among them, s i Let be the scale factor, K be the camera intrinsic parameter matrix, and exp[·]. 3×4 The parentheses represent the top-left 3×4 submatrix of the matrix within the brackets, ||||2 represents the L2 norm operation, and P j The position of the corner point in the camera coordinate system for any shot taken from the i-th shooting pose.

[0126] P = [x, y, z] T Let B be the coordinate system of the robot's base, and P be the coordinate system of the robot's base. (The text also mentions a corner point on the chessboard (taking the top left corner as an example) in the world coordinate system W.) j For the j-th shooting pose g j The position P of the lower position in the camera coordinate system is expressed by the following relationship:

[0127]

[0128] like Figure 2 As shown, the corresponding i-th shooting pose g i The position of position P in the camera coordinate system is P i . Figure 2 In the middle, c i ,c j For the i-th and j-th shooting poses g i g j The coordinate system below is used for addition. Thus, through two sets of poses H... gi H gj The hand-eye relationship between them can be used to determine the position P. j Coordinate transformation to position P in the i-th shooting pose ij And according to hand-eye relationship:

[0129] H gi XH ci =H cj XH gj

[0130]

[0131] Position P can be obtained by sorting. ij The following expression:

[0132]

[0133] Where X = H cg .

[0134] The position P can then be determined using the camera's calibration parameters. ij The image coordinates P′ are obtained by projecting the robot's i-th pose onto the image. ij Let P′ be the denoted P′. ij The corresponding corner position P on the i-th captured image i For P i =[u i ,v i ,1] T ,Right now:

[0135]

[0136]

[0137] Among them, s i Let be the scale factor, K be the camera intrinsic parameter matrix, and exp[·]. 3×4 This represents a 3×4 submatrix of the matrix within the brackets.

[0138] Let position P′ be... ij If the reprojection error is err, then the position P′ ij That is, the reprojection error of the corner point is:

[0139]

[0140] Substituting the expression for the reprojection error of the corner point into the above equation, we get:

[0141]

[0142] Extending this to the n sets of data in the entire calibration dataset, we can obtain the reprojection error cost function:

[0143]

[0144] Reprojection error cost function err * Fitness function of particle swarm optimization algorithm By iteratively solving the problem, the optimized hand-eye relationship matrix can be obtained.

[0145] Step S3.3: Calculate the fitness function value of the particle. Find the current individual extreme value of each particle. Position and update to the particle's historical best position. and group extreme values The position is the minimum particle position in the fitness function and is updated to the globally optimal position g. * :

[0146]

[0147]

[0148] Step S3.4: Update the velocity v of each particle k and position p k :

[0149]

[0150] p k =p k +v k

[0151] Where ω is the inertia factor, c1 and c2 are acceleration constants, and then return to step 4.3) until the set termination condition is reached;

[0152] Step S3.5: Set the global optimal position g * As the optimized hand-eye relationship matrix

[0153] Particle swarm optimization (PSO) is existing technology and will not be elaborated upon here. The innovation of this invention lies in using PSO to process the hand-eye relationship matrix H calculated in step S2, based on the constructed reprojection error cost function. cg Optimization was performed. Using the hand-eye relationship matrix calculated based on the Kronecker product parameterization method as the initial value, and photogrammetric bundle adjustment as the optimization model, the minimum reprojection error was established as the cost function. A particle swarm optimization (PSO) algorithm was introduced to iteratively update the optimal particles, optimizing the rotation and translation vectors. This further improved the hand-eye calibration accuracy and ensured the registration quality of the point cloud images.

[0154] Step S4: Three-dimensional topography measurement

[0155] Debug the binocular camera so that it can clearly capture the object to be measured, i.e., the large and complex parts to be measured, keep the left and right cameras at the same horizontal position and maintain a certain distance, and complete the binocular positioning; debug the robot so that it can carry the binocular camera to perform the three-dimensional measurement task, and ensure that the whole picture of the object to be measured is captured.

[0156] Structured light images are captured at each measurement pose, and the unit measurement point cloud data of the measured object is reconstructed. Record the measured pose and calculate the pose matrix of the robot. Where m is the number of measurement poses for the three-dimensional shape measurement of the object, based on the obtained unit point cloud data. Pose matrices from various viewpoints and the optimized hand-eye relationship matrix obtained in step (4) Calculate the rigid transformation matrix of each unit point cloud data from the stereo camera coordinate system to the robot base coordinate system:

[0157]

[0158] The unit measurement point cloud data acquired by the binocular camera is converted to the unified coordinate system of the robot base to obtain the single-view point cloud in the q-th measurement pose after registration.

[0159]

[0160] The registration of point cloud data from multiple perspectives is achieved, thereby enabling the three-dimensional morphological measurement of large and complex components.

[0161] By controlling the robot's movement to carry a binocular camera, unit measurement point cloud information of the parts under test is collected from multiple perspective poses. Based on the optimized hand-eye matrix, multiple point clouds are transformed into the same coordinate system to complete the multi-view point cloud registration function, thereby realizing the three-dimensional shape measurement of the parts under test and improving the efficiency of three-dimensional shape measurement of large and complex parts.

[0162] Example

[0163] In this embodiment, the specimen sample used as a large and complex component is 600mm×450mm×200mm in size. It is necessary to collect unit measurement point cloud data from multiple measurement poses. The calibration plate used is an 8×11, 15mm standard black and white checkerboard calibration plate.

[0164] During the data acquisition process, an operator controls a robot to drive a binocular camera to acquire structured light images of the specimen sample from multiple perspectives and poses. To ensure the completeness of the sample's image coverage, some overlap between the image coverage of adjacent measurement units is considered. This embodiment obtains 3D point cloud data of the specimen, excluding the bottom, thus reconstructing the morphological features of the real object.

[0165] In this embodiment, firstly, the robot is controlled to drive the binocular camera to acquire nine calibration board images with a pixel resolution of 3000×4096 at nine different shooting poses. Then, the homogeneous expression H of the camera extrinsic parameter matrix for capturing the nine calibration board images is calculated using Zhang's calibration method. ci The robot pose information at the corresponding shooting positions is recorded, and the secondary expression H of its pose matrix is ​​calculated. gi Let i = 1, 2, ..., 9. Then, establish 36 hand-eye calibration equations to be solved.

[0166] Then, the hand-eye equations to be solved are transformed into a system of linear algebraic equations using the properties of the Kronecker product and the Moore-Penrose inverse. Finally, the rotation and translation matrix of the camera coordinate system relative to the robot end flange coordinate system, i.e., the hand-eye relationship matrix H, is solved using the least squares method. cg In this embodiment, the hand-eye relationship matrix H is solved by the Kronecker product and the Moore-Penrose inverse. cg The expression is as follows:

[0167]

[0168] Among them, the hand-eye relationship matrix H cg rotation matrix R cg The top-left corner is a 3×3 matrix, i.e.:

[0169]

[0170] Hand-eye relationship matrix H cg Translation vector T cg The top right corner is a 3×1 column, i.e.

[0171]

[0172] Based on the calculated hand-eye relationship matrix H cg This sets the initial positions of the search space for the Particle Swarm Optimization (PSO) algorithm. In this example, the initial positions of the K particles in the swarm are all H. cg ,

[0173] In this example, the optimized hand-eye relationship matrix for:

[0174]

[0175] like Figure 3 As shown, after optimizing the hand-eye matrix using the particle swarm optimization algorithm, the origin position of the calibration plate in the robot base coordinates can be calculated. Comparing the distribution of the calibration plate origin in n shooting poses before and after optimization, it can be seen that the optimized calibration plate origin (solid origin) is more concentrated and basically overlaps; while the unoptimized calibration plate origin (hollow origin) is more dispersed. This result proves the improvement of the hand-eye calibration result, and its effect will be verified in topography measurement.

[0176] For the specimen sample in this embodiment, a mobile robot is needed to drive a binocular camera to complete the structured light image acquisition of the left and right cameras of the specimen sample under measurement poses from multiple perspectives, and reconstruct the unit measurement point cloud data. The robot pose information from various viewpoints is recorded to obtain the pose matrix. Unit measurement point cloud data such as Figure 4 As shown.

[0177] Using the obtained unit point cloud data and pose matrices from each viewpoint, and the hand-eye relationship matrix obtained through the method proposed in this invention, the rigid body transformation matrix from the binocular camera coordinate system to the robot base coordinate system for each unit point cloud data is calculated:

[0178]

[0179] like Figure 5 As shown, rigid body transformation was used to convert multiple unit measurement point cloud data to a unified coordinate system, and the overall point cloud maintained its original shape without distortion during the transformation process.

[0180] In this embodiment, the three-dimensional morphology measurement results of the specimen sample as a large and complex component are as follows: Figure 6 As shown, in this measurement, overall point cloud data excluding the bottom of the specimen sample was obtained. Multiple unit measurement point cloud data were then used to achieve point cloud registration using the method proposed in this invention. The resulting overall point cloud shows the complete morphology of the specimen sample and closely approximates its actual size. Furthermore, no manually marked points were used as features for point cloud registration throughout the entire measurement process, significantly improving the efficiency of the overall measurement task.

[0181] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A robot-assisted multi-view three-dimensional scanning measurement method, characterized in that, include: (1) Establish hand-eye calibration equation A binocular camera is mounted on the end flange of the robot. The robot is controlled to move to the i-th shooting pose and take a picture of a corner point on the chessboard calibration board. An image of the calibration board is obtained at this shooting pose. At the same time, the robot's pose information and the position P of the corner point in the camera coordinate system are recorded. i By taking pictures of n shooting positions, n sets of calibration board images and corresponding robot posture information are obtained; The rotation vector R of the calibration board relative to the camera in each set of calibration board images is obtained using Zhang's calibration method. ci Translation vector T ci This is then converted into a rotation and translation matrix, thus obtaining the camera extrinsic parameter matrix H. ci For i = 1, 2, ..., n, calculate the rotation vector R relative to the base based on the robot's pose information. gi Translation vector T gi The robot pose matrix H is obtained. gi ,i=1,2,...,n, where: For any two changes in shooting pose u, v, the camera extrinsic parameter matrix H cu H cv and robot pose matrix H gu H gv Establish hand-eye calibration equations: in: Among them, R gu,v Representation matrix The rotation matrix T in gu,v Representation matrix The translation vector in R cg Represents the hand-eye relationship matrix H cg The rotation matrix T in cg R represents the translation vector in the hand-eye relationship matrix. cu,v Representation matrix The rotation matrix, T cu,v Representation matrix Translation vector in; (2) The hand-eye calibration equation is transformed into a least squares problem using the Kronecker product, and the hand-eye relationship matrix H is calculated using singular value decomposition. cg Establish a system of linear equations: Where I is the identity matrix, The Kronecker product is represented by vec, which represents the vectorization operation. The matrix under both captured pose changes u and v Arranged column-wise, we obtain a matrix R; Perform singular value decomposition on matrix R, extract the last 9 elements of the right singular matrix V, and restore it to a 3×3 matrix form to obtain the matrix. For matrix Perform orthogonalized singular value decomposition, i.e. Among them, U R For a right singular matrix, ∑ R V is a singular value matrix. R Given a left singular matrix, we obtain the hand-eye relationship matrix H. cg rotation matrix R cg Right now The matrix (R) under both transformed shooting poses u,v gu,v -I) Arrange the columns to obtain a matrix R g The matrix (R) under all two transformed shooting poses u,v cg T cu,v -T gu,v Arrange them column-wise to obtain a matrix T. According to the rotation matrix R cg The hand-eye relationship matrix H was calculated. cg Translation vector T cg : T cg =R g -1 T (3) Optimize the hand-eye relationship matrix based on minimizing reprojection error 3.1) Construct a particle swarm with a population size of K, where each particle is at position p. k and velocity v k Then initialize the positions p of the K particles. k and speed v k : p k =H cg v k =rand(v min ,v max ) Among them, v min ,v max This represents the upper and lower threshold values ​​for the particle's velocity set in each iteration, with `rand` representing a random value selection operation. This results in the initial position of each particle being the hand-eye relationship matrix H. cg A swarm of particles, where the velocity of each particle is randomly assigned, has a population size of K. 3.2) Establish the reprojection error cost function as the fitness function of the particle swarm optimization algorithm: Among them, s i Let be the scale factor, K be the camera intrinsic parameter matrix, and exp[·]. 3×4 The parentheses represent the top-left 3×4 submatrix of the matrix within the brackets, || ||2 represents the L2 norm operation, and P j The position of the corner point in the camera coordinate system for any shot not taken from the i-th shooting pose; 3.3) Calculate the fitness function value of the particle. Find the current individual extreme value of each particle. Position and update to the particle's historical best position. and group extreme values The position is the minimum particle position in the fitness function and is updated to the globally optimal position g. * : 3.4) Update the velocity v of each particle. k and position p k : p k =p k +v k in, ω is the inertia factor, c1 and c2 are acceleration constants, and then return to step 3.3) until the set termination condition is reached; 3.5) Set the global optimal position g * As the optimized hand-eye relationship matrix (4) Three-dimensional topography measurement Debug the binocular camera so that it can clearly capture the object to be measured, i.e., the large and complex parts to be measured, keep the left and right cameras at the same horizontal position and maintain a certain distance, and complete the binocular positioning; debug the robot so that it can carry the binocular camera to perform the three-dimensional measurement task, and ensure that the whole picture of the object to be measured is captured. Structured light images are captured at each measurement pose, and the unit measurement point cloud data of the measured object is reconstructed. Record the measured pose and calculate the pose matrix of the robot. Where m is the number of measurement poses for measuring the three-dimensional shape of the object. Based on the obtained unit measurement point cloud data Pose matrices from various viewpoints and the optimized hand-eye relationship matrix obtained in step (4) Calculate the rigid transformation matrix of each unit measurement point cloud data from the binocular camera coordinate system to the robot base coordinate system: The unit measurement point cloud data acquired by the binocular camera is converted to the unified coordinate system of the robot base to obtain the single-view point cloud in the q-th measurement pose after registration. The registration of point cloud data from multiple perspectives is achieved, thereby enabling the three-dimensional morphological measurement of large and complex components.

Images