Single-target-point shooting hand-eye calibration method based on genetic algorithm

Abstract

The invention provides a single-target-point shooting hand-eye calibration method based on a genetic algorithm, and belongs to the technical field of robot vision calibration, and the method comprises the steps: keeping the position of a calibration ball unchanged, and shooting K groups of original point clouds of the calibration ball through a camera; performing color segmentation and filtering processing on each group of original point clouds to extract K groups of calibration ball surface partial point clouds; template matching is carried out on part of the point clouds on the surfaces of all the sets of calibration balls, so that the spatial positions of the centers of all the calibration balls corresponding to the K sets of original point clouds in a camera coordinate system are obtained; constructing an equation set formed by K nonlinear equations; and according to the spatial position invariance of the calibration balls in the robot base coordinate system in multiple times of shooting, an optimization problem for minimizing the spatial position difference of the centers of the calibration balls in the camera coordinate system is obtained, and the optimization problem is calculated by using a genetic algorithm to obtain an optimal hand-eye matrix. The method does not need extra auxiliary equipment, can effectively resist environmental noise, and improves the calibration robustness.

Description

Technical Field

[0001] The invention belongs to the technical field of robot vision calibration, and in particular relates to a single-target point shooting hand-eye calibration method based on a genetic algorithm. Background Art

[0002] With the rapid development of industrial robots and robot vision technology, they have been widely used in scenarios such as part recognition, positioning, and measurement. When using the eye-in-hand camera installation method, the accuracy of the target object's pose estimation depends on the hand-eye calibration accuracy and the robot's repeated positioning accuracy. Therefore, robot hand-eye calibration is a necessary step in the use of robot vision.

[0003] When using 3D targets for robot hand-eye calibration, spherical targets offer advantages such as more relaxed camera pose requirements and good contour continuity. Therefore, they are widely used in both single-camera hand-eye calibration and multi-camera system calibration. However, existing methods using calibration spheres for hand-eye calibration rely heavily on geometric fitting during image processing to obtain the coordinates of the sphere's center. This method, which relies solely on a single geometric shape to locate spatial points, is not robust against environmental noise, such as reflections and partial occlusions.

[0004] Existing methods typically use a point set consisting of four or more target points to solve the hand-eye calibration matrix, placing high precision requirements on the processing and installation of calibration tools. Furthermore, during calibration, the operator must operate the robotic arm to capture target points in different poses. Simultaneously capturing multiple target points limits the robotic arm's ability to capture certain poses, resulting in excessive concentration of calibration data and a decrease in calibration accuracy. Summary of the Invention

[0005] The purpose of this invention is to propose a single-target point shooting hand-eye calibration method based on genetic algorithm, which does not require additional auxiliary equipment, can effectively resist environmental noise and improve calibration robustness.

[0006] The present invention is achieved through the following technical solutions: A single-target point shooting hand-eye calibration method based on genetic algorithm includes the following steps: Step S1: Keep the calibration ball in the same position, and drive the robot to use the camera to shoot the calibration ball in the motion space. K A set of original point clouds, the camera is set at the end of the robot; Step S2: perform color segmentation and filtering on each group of original point clouds to extract K Group calibration sphere surface partial point cloud; Step S3: perform template matching on each group of calibration sphere surface point clouds to obtain KThe spatial position of the center of each calibration sphere corresponding to the original point cloud of the group in the camera coordinate system, wherein a part of the complete point cloud of the calibration sphere is taken as the template point cloud; Step S4: Establish a kinematic chain between the robot base coordinate system, the robot end dynamic coordinate system, the camera coordinate system and the calibration ball center position, and construct K The nonlinear equations involve the homogeneous transformation matrix between the robot's end-moving coordinate system and the robot's base coordinate system, the hand-eye matrix as the target quantity, the center position of the calibration ball, and the spatial position of the center of the calibration ball in the robot's base coordinate system. Step S5: Convert the hand-eye matrix in the nonlinear equation into Euler angles, and based on the invariance of the spatial position of the calibration ball in the robot base coordinate system during multiple shots, obtain an optimization problem of minimizing the difference in the spatial position of the center of each group of calibration balls in the camera coordinate system. Use a genetic algorithm to calculate the optimization problem to obtain the optimal hand-eye matrix.

[0007] Furthermore, in step S1, the calibration sphere is fixed on a black background plate via a rigid target holder, the surface of the calibration sphere is coated with a highly reflective white coating, and the camera is a binocular structured light camera; Furthermore, in step S2, i Group original point cloud Filter to remove outliers to obtain a preliminary processed point cloud , traverse the initial processing point cloud , judge whether the RGB attributes of each preliminary processed point cloud are greater than the set color segmentation threshold, if so, retain the point cloud, otherwise filter out the point cloud to obtain the segmented point cloud , perform secondary filtering on the segmented point cloud to remove the local fragments retained by the reflection, and obtain the first i Group calibration sphere surface partial point cloud , where 1≤ i ≤ K .

[0008] Furthermore, in step S3, a cross section is taken so that the surface shell of the calibration sphere is divided into two parts with a height ratio of 1:3, and the point cloud of the smaller part is taken as the template point cloud. .

[0009] Furthermore, in step S3, the first i The spatial position of the center of the calibration ball corresponding to the original point cloud in the camera coordinate system includes the following steps: Step S31: Use the point cloud center to make the template point cloud With the i Group calibration sphere surface partial point cloud Align and use normal distribution transformation for rough matching, transform the aligned template point cloud to the rough matching point cloud, and obtain the initial homogeneous transformation matrix that records the displacement information during the transformation process ; Step S32: Use the iterative closest point algorithm to perform fine registration, transform the coarse matching point cloud into the fine matching point cloud, and obtain the homogeneous transformation matrix that records the displacement information during the transformation process. ; Step S33: According to the formula Calculate the composite transformation matrix , the composite transformation matrix involves the displacement component p i,x3 、 p i,y3 、 p i,z3 , No. i The spatial position of the center of the calibration ball corresponding to the original point cloud in the camera coordinate system is expressed as .

[0010] Furthermore, in step S4, K The nonlinear equations are expressed as ,in, Indicates obtaining the i When assembling the original point cloud, the homogeneous transformation matrix of the robot end relative to the robot base coordinate system is obtained using the forward kinematics method. represents the hand-eye matrix as the target quantity, Indicates the spatial position of the calibration ball in the robot base coordinate system.

[0011] Furthermore, step S5 includes the following steps: Step S51: convert the hand-eye matrix in the nonlinear equation into the Euler angle expression ,in, Indicates that each moves along the X, Y, and Z axes of the robot's end motion coordinate system x '、 y '、 z 'The homogeneous transformation matrix, 、 、 Respectively represent the Z, Y, and X axis rotations around the robot's end moving coordinate system a '、 b '、 c 'Homogeneous transformation matrix; Step S52: The optimization problem is expressed as ,in, , norm() is a vector modulus calculation function.

[0012] Step S53: Use a genetic algorithm to calculate the optimization problem to obtain the optimal hand-eye matrix.

[0013] Furthermore, in step S2, if If it is established, it is determined that the RGB attributes of each preliminary processed point cloud are greater than the set color segmentation threshold ,in, Point.R Represents a single preliminary processed point cloud Point The R attribute, Point.G Represents a single preliminary processed point cloud Point The G attribute, Point.B Represents a single preliminary processed point cloud Point The B attribute.

[0014] The present invention has the following beneficial effects: 1. The present invention uses only one calibration ball as the target point, keeps the position of the calibration ball unchanged, and drives the robot to use the camera to shoot the calibration ball in the motion space. K The original point clouds of each group are segmented and filtered to improve the quality of original data processing. Then, template matching is performed on the surface point clouds of each group of calibration spheres to obtain K The spatial position of each calibration ball center corresponding to the original point cloud in the camera coordinate system is obtained, thereby improving the positioning accuracy of the ball center. Then, a motion chain is established between the robot base coordinate system, the robot end dynamic coordinate system, the camera coordinate system and the calibration ball center position to construct a kinematic chain composed of K The hand-eye matrix in the nonlinear equation is finally converted into Euler angle expression. According to the invariance of the spatial position of the calibration ball in the robot base coordinate system during multiple shots, the optimization problem of minimizing the spatial position difference of the center of each group of calibration balls in the camera coordinate system is obtained. The genetic algorithm is used to calculate the optimization problem to obtain the optimal hand-eye matrix. By converting the equation established by the kinematic model into the above optimization problem and combining it with the genetic algorithm to calculate the optimization problem, the global optimal solution can be obtained from the algorithm level, avoiding the local optimal solution, thereby effectively improving the calibration robustness and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention will be further described in detail below with reference to the accompanying drawings.

[0016] Figure 1 Flowchart of the present invention.

[0017] Figure 2 This is the overall structural diagram of the calibration system of the present invention.

[0018] Figure 3 This is a point cloud processing flowchart of the present invention.

[0019] Figure 4Schematic diagram of point cloud transformation of the present invention.

[0020] Figure 5 This is the robot coordinate system conversion chain of the present invention.

[0021] Figure 6 The present invention provides a set of equations for construction and optimization process.

[0022] Figure 7 It is the fitness convergence curve of the genetic algorithm of the present invention. DETAILED DESCRIPTION

[0023] like Figure 1 As shown in FIG, the hand-eye calibration method for single target point shooting based on genetic algorithm includes the following steps: Step S1: Keep the calibration ball in the same position, and drive the robot to use the camera to shoot the calibration ball in the motion space. K A set of original point clouds, the camera is set at the end of the robot; The overall structure of the calibration system is shown in the figure below: Figure 2 The camera is a binocular structured light camera, which is installed at the end of the robot (i.e., robotic arm). The calibration ball is fixed to the black background plate through a rigid target base, and the surface of the calibration ball is coated with a highly reflective white coating. According to the spatial position of the target base, calibration ball, and black background plate, the robot motion space is calculated and the robot is selected in the motion space. K = 14 groups of shooting points, the staff drives the robot to move in the motion space, and controls the robot's mechanical arm posture through the robot control system to obtain K = 14 groups of original point clouds, each group of original point cloud images includes part of the target base, background plate and the visible part of the calibration sphere. i The original point cloud is as follows Figure 4 As shown in (a).

[0024] Step S2: perform color segmentation and filtering on each group of original point clouds to extract K Group calibration sphere surface partial point cloud; In order to eliminate the influence of background and other noise, such as Figure 3 As shown in the figure, set the parameters of the statistical filter such as the nearest point and distance threshold, and i Group original point cloud Filter to remove outliers and obtain a preliminary processed point cloud ; Set the color segmentation threshold based on the RGB attributes of each preliminary processed point cloud , traverse the initial processing point cloud , for a single preliminary processed point cloud Point ,like If the point is closer to white in the RGB color space, then the RGB attributes of each preliminarily processed point cloud are greater than the set color segmentation threshold. , then retain the point cloud, otherwise filter out the point cloud, and then obtain the segmented point cloud , perform secondary filtering on the segmented point cloud to remove the local fragments retained by the reflection, and obtain the first i Group calibration sphere surface partial point cloud ,in, Point.R Represents a single preliminary processed point cloud Point The R attribute, Point.G Represents a single preliminary processed point cloud Point The G attribute, Point.B Represents a single preliminary processed point cloud Point B attribute, 1≤ i ≤ K .like Figure 4 (b) shows the i Part of the point cloud on the surface of the group calibration sphere shows that the outline of the surface point group is relatively complete and there are no obvious outliers.

[0025] Step S3: perform template matching on each group of calibration sphere surface point clouds to obtain K The spatial position of the center of each calibration sphere corresponding to the original point cloud of the group in the camera coordinate system, wherein a part of the complete point cloud of the calibration sphere is taken as the template point cloud; The specific steps include: Step S30: Take a cross section so that the surface shell of the calibration sphere is divided into two parts with a height ratio of 1:3, and take the point cloud of the smaller part as the template point cloud ,like Figure 4 (c) Step S31: Use the point cloud center to make the template point cloud With the i Group calibration sphere surface partial point cloud Perform alignment and obtain the template point cloud along the robot base coordinate system x 、 y 、 z Displacement component in the direction p i,x 、 p i,y 、 p i,z , and use the normal distribution transformation for rough matching, transform the aligned template point cloud to the rough matching point cloud, and obtain the initial homogeneous transformation matrix , to record the aligned template point cloud Move to coarse matching point cloud The displacement information, 、 、 Represent the coarse registration point cloud coordinate system x 、 y 、 z The unit direction vector of the axis in the template point cloud coordinate system, represents the displacement component of the origin of the coarse registration point cloud coordinate system in the template point cloud coordinate system; where, Figure 5 As shown, the robot base coordinate system is , the camera coordinate system is , the robot end moving coordinate system is In this embodiment, a six-joint robot is taken as an example, but it can also be extended to serial robots or serial structures of any configuration.

[0026] Step S32: Use the iterative closest point algorithm to perform fine registration, set the maximum number of iterations, mean square error and other parameters, and obtain the homogeneous transformation matrix of the displacement information in the process of transforming the coarse matching point cloud to the fine registration point cloud. ,in, 、 、 Respectively represent the unit direction vectors of the x, y, and z axes of the fine registration point cloud coordinate system in the coarse registration point cloud coordinate system, It represents the displacement component of the origin of the fine registration point cloud coordinate system in the coarse registration point cloud coordinate system. The point cloud registration is as follows: Figure 4 (d) Step S33: According to the formula Calculate the composite transformation matrix , the composite transformation matrix involves the displacement component p i,x3 、 p i,y3 、 p i,z3 , No. i The spatial position of the center of the calibration ball corresponding to the original point cloud in the camera coordinate system is expressed as .

[0027] Step S4: Establish a kinematic chain between the robot base coordinate system, the robot end dynamic coordinate system, the camera coordinate system and the calibration ball center position, and construct K The nonlinear equations involve the homogeneous transformation matrix between the robot's end-moving coordinate system and the robot's base coordinate system, the hand-eye matrix as the target quantity, the center position of the calibration ball, and the spatial position of the center of the calibration ball in the robot's base coordinate system. Robot end moving coordinate system With the robot base coordinate system The conversion relationship between them is obtained by the robot control system and is generally expressed as Euler angles Convert the Euler angle into the corresponding homogeneous transformation matrix in the form of (This embodiment is a six-joint robot, so the homogeneous transformation matrix is ), which means getting the i When forming the original point cloud, the homogeneous transformation matrix of the robot end relative to the robot base coordinate system is obtained using the forward kinematics method.

[0028] Define the hand-eye matrix as , from the previous steps, the calibration sphere center can be expressed in the camera coordinate system as , then each shot establishes an equation through the kinematic chain, expressed as ,but K Group shooting formed a total of K Group equations to form a system of equations ,in, and It changes and is known every time you shoot, Indicates the spatial position of the calibration ball in the robot's base coordinate system. It is different and unknown for each shot. The shooting position remains unchanged each time. Minimize the difference and find the invariant .

[0029] Step S5: Convert the hand-eye matrix in the nonlinear equation into Euler angles. Based on the invariance of the spatial position of the calibration balls in the robot base coordinate system during multiple shots, obtain an optimization problem to minimize the difference in the spatial positions of the centers of each group of calibration balls in the camera coordinate system. Use a genetic algorithm to calculate this optimization problem to obtain the optimal hand-eye matrix. like Figure 6 As shown, the specific steps include: Step S51: Use the Euler angle method of xyzabc to convert the optimization target from 12 unknowns of the homogeneous transformation matrix to 6 unknowns. , in order to reduce the complexity of the optimization process, the hand-eye matrix is expressed in Euler angles as ,in, Indicates that each moves along the X, Y, and Z axes of the robot's end motion coordinate system x '、 y '、 z 'The homogeneous transformation matrix, 、 、 Respectively represent the Z, Y, and X axis rotations around the robot's end moving coordinate system a '、 b '、 c 'Homogeneous transformation matrix; Step S52: The optimization problem is expressed as ,in, , Indicates smallest; Step S53: using a genetic algorithm to calculate the optimization problem to obtain the optimal hand-eye matrix; The process of calculation using genetic algorithm includes: Initializing the genetic algorithm: determining the initial population size N , constraints and iteration termination conditions, generate the initial population, and use real number coding; among them, in order to balance the computational cost and avoid local optimality, combined with the current variable dimension d=6, take the number of candidate solutions, that is, the population size N The upper and lower limits of the optimization variables are roughly estimated based on the hand-eye installation position, and the floating range of the variables is given for selection. The tolerance of the constraint conditions and the decrease tolerance of the objective function between two adjacent iterations are set, and the initial population (candidate solutions) are randomly initialized according to the above conditions.

[0030] Determine the fitness function, select suitable individuals to enter the next generation according to the fitness function, and perform crossover and mutation on the remaining individuals in the current population; the fitness function is designed to be , select the top K parents with the highest fitness and merge and sort them with the offspring individuals after crossover mutation, and then retain the best N Individuals; arithmetic crossover is used to generate offspring. , represents the offspring code, and Denotes the two parents of the crossover, α is the crossover operator, which can be taken as 0.5. The mutation method uses Gaussian mutation, that is, a random perturbation with a mean of 0 and a controllable standard deviation is applied to a single optimization variable.

[0031] Repeat the iterative process until the number of iterations or the iteration accuracy reaches the preset threshold, decode the final chromosome, and obtain the result This is the optimization result of the hand-eye matrix calculated by the genetic algorithm. Figure 7 is the fitness convergence curve of the genetic algorithm.

[0032] The above description is merely a preferred embodiment of the present invention and therefore cannot be used to limit the scope of the present invention. In other words, equivalent changes and modifications made according to the scope of the patent application and the contents of the specification should still fall within the scope of the patent of the present invention.

Claims

1. A single-target point shooting hand-eye calibration method based on genetic algorithm, characterized by: The steps include: Step S1: Keep the calibration ball in the same position, and drive the robot to use the camera to shoot the calibration ball in the motion space. K A set of original point clouds, the camera is set at the end of the robot; Step S2: perform color segmentation and filtering on each group of original point clouds to extract K Group calibration sphere surface partial point cloud; Step S3: perform template matching on each group of calibration sphere surface point clouds to obtain K The spatial position of the center of each calibration sphere corresponding to the original point cloud of the group in the camera coordinate system, wherein a part of the complete point cloud of the calibration sphere is taken as the template point cloud; Step S4: Establish a kinematic chain between the robot base coordinate system, the robot end dynamic coordinate system, the camera coordinate system and the calibration ball center position, and construct K The nonlinear equations involve the homogeneous transformation matrix between the robot's end-moving coordinate system and the robot's base coordinate system, the hand-eye matrix as the target quantity, the center position of the calibration ball, and the spatial position of the center of the calibration ball in the robot's base coordinate system. Step S5: Convert the hand-eye matrix in the nonlinear equation into Euler angles, and based on the invariance of the spatial position of the calibration ball in the robot base coordinate system during multiple shots, obtain an optimization problem of minimizing the difference in the spatial position of the center of each group of calibration balls in the camera coordinate system. Use a genetic algorithm to calculate the optimization problem to obtain the optimal hand-eye matrix.

2. The genetic algorithm-based hand-eye calibration method for single-target point shooting according to claim 1, characterized in that: In step S1, the calibration sphere is fixed on a black background plate through a rigid target holder, the surface of the calibration sphere is coated with a highly reflective white coating, and the camera is a binocular structured light camera; The method for hand-eye calibration based on genetic algorithm for single target point shooting according to claim 1 is characterized in that: in step S2, i Group original point cloud Filter to remove outliers and obtain a preliminary processed point cloud , traverse the initial processing point cloud , judge whether the RGB attributes of each preliminary processed point cloud are greater than the set color segmentation threshold, if so, retain the point cloud, otherwise filter out the point cloud to obtain the segmented point cloud , perform secondary filtering on the segmented point cloud to remove the local fragments retained by the reflection, and obtain the first i Group calibration sphere surface partial point cloud , where 1≤ i ≤ K .

3. The method for hand-eye calibration based on a genetic algorithm for single-target point shooting according to claim 3, characterized in that: In step S3, a cross section is taken so that the surface shell of the calibration sphere is divided into two parts with a height ratio of 1:3, and the point cloud of the smaller part is taken as the template point cloud. .

4. The method for hand-eye calibration based on a genetic algorithm for single-target point shooting according to claim 4, characterized in that: In step S3, the first i The spatial position of the center of the calibration ball corresponding to the original point cloud in the camera coordinate system includes the following steps: Step S31: Use the point cloud center to make the template point cloud With the i Group calibration sphere surface partial point cloud Align and use normal distribution transformation for rough matching, transform the aligned template point cloud to the rough matching point cloud, and obtain the initial homogeneous transformation matrix that records the displacement information during the transformation process ; Step S32: Use the iterative closest point algorithm to perform fine registration, transform the coarse matching point cloud into the fine matching point cloud, and obtain the homogeneous transformation matrix that records the displacement information during the transformation process. ; Step S33: According to the formula Calculate the composite transformation matrix , the composite transformation matrix involves the displacement component p i,x3 、 p i,y3 、 p i,z3 , No. i The spatial position of the center of the calibration ball corresponding to the original point cloud in the camera coordinate system is expressed as .

5. The method for hand-eye calibration based on a genetic algorithm for single-target point shooting according to claim 5, characterized in that: In the step S4, the K The nonlinear equations are expressed as ,in, Indicates obtaining the i When assembling the original point cloud, the homogeneous transformation matrix of the robot end relative to the robot base coordinate system is obtained using the forward kinematics method. represents the hand-eye matrix as the target quantity, Indicates the spatial position of the calibration ball in the robot base coordinate system.

6. The method for hand-eye calibration based on a genetic algorithm for single-target point shooting according to claim 6, characterized in that: The step S5 comprises the following steps: Step S51: convert the hand-eye matrix in the nonlinear equation into the Euler angle expression ,in, Indicates that each moves along the X, Y, and Z axes of the robot's end motion coordinate system x '、 y '、 z 'The homogeneous transformation matrix, 、 、 Respectively represent the Z, Y, and X axis rotations around the robot's end moving coordinate system a '、 b '、 c 'Homogeneous transformation matrix; Step S52: The optimization problem is expressed as ,in, , norm() is a vector modulus calculation function.

7. Step S53: Use a genetic algorithm to calculate the optimization problem to obtain the optimal hand-eye matrix.

8. The genetic algorithm-based hand-eye calibration method for single-target point shooting according to claim 3, characterized in that: In step S2, if If it is established, it is determined that the RGB attributes of each preliminary processed point cloud are greater than the set color segmentation threshold ,in, Point.R Represents a single preliminary processed point cloud Point The R attribute, Point.G Represents a single preliminary processed point cloud Point The G attribute, Point.B Represents a single preliminary processed point cloud Point The B attribute.

Images